JP2020135099A - State recognition device - Google Patents

Abstract

To provide a state recognition device which can select beforehand a measurement range or a measurement target of state recognition processing to be executed and can execute high-level state recognition processing while reducing the processing resources of the entire device.SOLUTION: The state recognition device includes: a motion information generation unit for generating motion information of measurement targets on the basis of information acquired by a measuring device; an interaction extraction unit for extracting interaction information of a space and a measurement target or interaction information of measurement targets on the basis of the motion information; a recognition target determination unit for determining a recognition target on which the state recognition processing is to be executed, on the basis of the interaction information; and a state recognition unit of executing the state recognition processing on the recognition target.SELECTED DRAWING: Figure 1

Description

The present invention relates to a state recognition device that recognizes the state of a measurement target based on the output of the measurement device, and a state recognition method.

In recent years, state recognition that recognizes the state within the measurement range of the measuring device and the state of the measurement target (person, vehicle, etc.) based on the information such as image data and distance data acquired by the measuring device such as a surveillance camera and a distance sensor. The need for technology is increasing. For example, there are technologies for detecting abnormal conditions (traffic jams, etc.) such as traffic flow and traffic flow, and technologies for detecting persons and vehicles with abnormal behavior (fallen person, rampage person, broken vehicle, etc.).

With this type of state recognition technology, processing resources are small if only the amount of change in state in the measurement range is detected, but when recognizing the detailed behavior of a person or vehicle using an advanced algorithm. Requires a large amount of processing resources, so measures to reduce the processing resources are required. Therefore, in general, processing resources are often reduced by setting a part of a wide measurement range in the recognition area for executing state recognition in advance.

However, when a surveillance camera installed in a public place recognizes an abnormal state, it is unclear in which area the abnormality occurs, so a preset recognition area may not be appropriate. Therefore, in the state recognition technology that uses an advanced algorithm, there are high expectations for a technology that determines a recognition area according to the actual situation.

As a method of determining the recognition area according to the actual situation, there is a method described in Patent Document 1. In this document, the measurement target is a vehicle, and by analyzing time change information such as the amount of movement of the vehicle, it is possible to identify a recognition area in which an abnormal state is presumed to occur. Then, by executing the state recognition process by advanced image analysis after identifying the recognition area, it is possible to recognize the abnormal state and the situation of the vehicle or person within the measurement range in detail while suppressing the processing resources, and issue a stepwise alarm. Highly accurate abnormality detection such as reduction of reports and false alarms can be realized.

JP-A-2002-190090

However, Patent Document 1 has a low degree of freedom in the direction of travel because it identifies an area where an abnormality in the traffic flow is expected by analyzing the image of the camera and detecting the gradual change of the traffic volume over time. When the measurement target is limited to the vehicle and the measurement target is a person with various traveling directions and actions, there is a problem that an appropriate recognition area cannot be set.

Therefore, according to the present invention, in a state recognition device that recognizes a measurement range and a state of a measurement target, a state recognition process is performed based on interaction information between measurement targets extracted from the general movement flow information of the measurement target. It is an object of the present invention to provide a state recognition device capable of executing advanced state recognition processing while reducing the processing resources of the entire device by selecting the measurement range and the measurement target to be executed in advance.

In order to achieve the above object, the state recognition device of the present invention has a motion information generation unit that generates motion information of a measurement target based on the information acquired by the measurement device, and a space and the measurement object based on the motion information. The interaction extraction unit that extracts the interaction information of the above or the interaction information between the measurement targets, the recognition target determination unit that determines the recognition target that executes the state recognition process based on the interaction information, and the recognition target determination unit. It is assumed to have a state recognition unit that executes a state recognition process for the recognition target.

According to the state recognition device of the present invention, a measurement range and a measurement target for executing the state recognition process can be selected in advance, and advanced state recognition processing can be executed while reducing the processing resources of the entire device.

Functional block diagram of the state recognition device of the first embodiment Functional block diagram of the human flow information acquisition unit The figure explaining the process in the person position measurement part The figure explaining the process in the flow line creation part The figure explaining the process in the person flow information generation part Functional block diagram of the recognition area candidate selection unit The figure explaining the processing in the person flow information comparison part Functional block diagram of the human flow interaction extraction unit The figure explaining the process in the positional relationship calculation part Functional block diagram of the state recognition device of the second embodiment

Hereinafter, specific embodiments of the state recognition device of the present invention will be described with reference to the drawings.

First, the outline of the state recognition device 1 according to the first embodiment of the present invention will be described with reference to the functional block diagram of FIG. In the present embodiment, the state recognition device 1 in which the connected measuring device is the monocular camera 2 and the measurement target is the person H will be described, but the application target of the present invention is limited to these measuring devices and the measurement target. The measuring device may be another sensor such as a stereo camera or a distance sensor, and the measurement target may be a vehicle or the like. The state recognition device 1 is actually a computer including a computing device such as a CPU, a main storage device such as a semiconductor memory, an auxiliary storage device such as a hard disk, and hardware such as a communication device. Then, while referring to the database recorded in the auxiliary storage device, the arithmetic unit executes the program loaded in the main storage device to realize each function described later. In the following, such a well-known technique will be used. The description will be omitted as appropriate.

The state recognition device 1 extracts the interaction information Ii from the human flow information Ih of the entire measurement range of the image data P captured by the camera 2, and based on this, determines in advance a narrow recognition area A to which advanced state recognition processing is applied. After that, it is a device that reduces processing resources by executing advanced state recognition processing for the determined recognition area A. Although the state recognition device 1 and the camera 2 are separately configured in FIG. 1, the state recognition device 1 may be incorporated into the camera 2 to integrate the two.

As shown in FIG. 1, the state recognition device 1 of this embodiment is output from the human flow information acquisition unit 3 and the human flow information acquisition unit 3 that acquire the human flow information Ih by analyzing the image data P captured by the camera 2. Based on the human flow information Ih acquired by the recognition area candidate selection unit 4 and the human flow information acquisition unit 3, which selects the recognition area candidate C as a candidate for executing the state recognition process by calculating the amount of change in the human flow from the human flow information Ih. In addition, the human flow interaction extraction unit 5 and the human flow interaction extraction unit that extract the interaction information Ii between the recognition area candidate C selected by the recognition area candidate selection unit 4 and the person H in the recognition area candidate C and other persons. A recognition area determination unit 6 that determines the recognition area A from the interaction information Ii acquired in 5 and a state recognition unit 7 that executes a state recognition process for the determined recognition area A are provided. The details of each part will be described below.
<People flow information acquisition department 3>
The details of the human flow information acquisition unit 3 (movement information generation unit) that generates the human flow information Ih, which is the movement information when the measurement target is a person, will be described with reference to the functional block diagram shown in FIG. As shown here, the flow information acquisition unit 3 measures the position of the person H by executing image analysis or the like on the image acquisition unit 31 that acquires the image data P from the camera 2 and the acquired image data P. From the person position measurement unit 32, the flow line creation unit 33 that creates the flow line L from the measured position information of the person H, the position information and the flow line information of the person H acquired from the person position measurement unit 32 and the flow line creation unit 33. The person flow information generation unit 34 for generating the person flow information Ih is provided. Hereinafter, each unit of the person position measurement unit 32, the flow line creation unit 33, and the person flow information generation unit 34 will be sequentially described.
<Person position measurement unit 32>
First, the person position measuring unit 32 will be described with reference to FIG. In FIG. 3, P is image data acquired from the camera 2, H is a person to be measured, F is a rectangular person area surrounding the person H, B is the foot position of the person H on image coordinates, and S ₃ is three-dimensional. Space, B ₃ is the foot position of the person H on the three-dimensional space S ₃ , S ₂ is the xz plane at the position of y = 0 on the three-dimensional space S ₃ , and B ₂ is the person on the xz plane S _2. The foot position of H is shown.

The person position measuring unit 32 first detects the person H in the image by executing an image processing algorithm such as person detection on the image data P. As a person detection method, a well-known person detection method such as saving a large number of sample images of person H in advance and searching for whether or not a part similar to the sample image exists in the acquired video may be used. , Not particularly limited. Further, instead of detecting the entire person, a method capable of detecting a part of the area of the person H from the image, such as face detection, may be used. Next, the foot position B (u, v) of the person H on the image coordinates is determined from the person area F in the image extracted by the person detection. As a determination method, a method in which the lowermost center of the circumscribing rectangle of the person area F is set as the foot position, or v from the image coordinates (u', v') of the highest apex of the head area of the person H detected by face detection. There is a method of using the image coordinates (u', v'-h) obtained by adding a constant value h in the direction as the foot position, and the present invention is not particularly limited. After that, the image coordinates (u, v) of the foot position B in the image data P are converted into the foot position B ₃ (x, y, z) in the three-dimensional space S ₃ , and on the xz plane S ₂ where y = 0. The foot position B ₂ (x, z) of is measured as the position of the person H. Conversion from foot position B (u, v) to foot position B ₃ (x, y, z) generally uses camera parameters of internal parameters (focal length, etc.) and external parameters (installation position, installation angle, etc.). Utilize various methods. As a method of deriving the camera parameters, general camera calibration may be used, and the camera parameters can be estimated by manually giving the world coordinates corresponding to the image coordinates in advance.
<Flow line creation unit 33>
FIG. 4 shows a processing flow in which the flow line creating unit 33 generates the flow line L of the person H. The flow line creating unit 33 generates the flow line L by combining the positions 28 of the same person measured by the person position measuring unit 32.

For example, when the person H is imaged by the camera 2 for t seconds from the time T, the flow line L can be generated by plotting the foot position B ₂ of the person H at each time on the same xz plane S ₂ and connecting them. it can. In addition to this example, any method other than the above may be used as long as it can generate a flow line L capable of grasping the position information and the traveling direction of the person H in a certain time zone.
<People flow information generation unit 34>
Next, the human flow information generation unit 34 will be described with reference to FIG. The human flow information generation unit 34 generates the human flow information Ih within the measurement range by using the flow lines La, Lb, ..., Ln for each person Ha, Hb, ..., Hn created by the flow line creation unit 33.

The method of creating the human flow information Ih will be specifically described. First, as shown in the upper part of FIG. 5, the xz plane S ₂ of the measurement range is divided into a plurality of areas (for example, 16 areas of 4 × 4). Then, the movement directions of the person H in each area are classified into the eight directions shown in the middle of FIG. 5 from the flow line information, and finally, the statistics of the movement directions of all the people H are calculated and the flow direction D (each area) for each area. By acquiring the moving direction (moving direction that frequently appears in), the human flow information Ih in the measurement range shown in the lower part of FIG. 5 is obtained. In this person flow information Ih, the area where the person flow direction D does not exist indicates that the area is not passed by a person, and from the person flow information Ih, not only the movement direction of the person H within the measurement range but also the person H is It is possible to grasp areas that do not pass. In addition to this example, as a method for generating the human flow information Ih, another method may be used as long as it is a method capable of grasping a rough human flow information Ih. Further, as the person flow information Ih, not only the person flow direction but also the number of persons H who have passed through each area may be counted and the person flow rate for each area may be added.
<Recognition area candidate selection unit 4>
FIG. 6 is a functional block diagram of the recognition area candidate selection unit 4. The recognition area candidate selection unit 4 stores the person flow information Ih acquired from the person flow information acquisition unit 3 in the person flow information history database 42, and the latest person flow information Ih _new acquired from the person flow information acquisition unit 3. And the past human flow information Ih _old stored in the human flow information history database 42 are compared with the human flow information comparison unit 43, and from the comparison result of the human flow information comparison unit 43, the area where the amount of change in the human flow is present is output as the recognition area candidate C. The recognition area candidate output unit 44 is provided. Hereinafter, the human flow information comparison unit 43 and the recognition area candidate output unit 44 will be described in detail.
<People flow information comparison unit 43, recognition area candidate output unit 44>
The human flow information comparison unit 43 will be described with reference to FIG. 7. The person flow information comparison unit 43 compares the latest person flow information Ih _new acquired from the person flow information acquisition unit 3 with the past person flow information Ih _old stored in the person flow information history database 42, and calculates the amount of change in person flow ΔIh.

As a comparison method used in the human flow information comparison unit 43, for example, there is a method of quantifying the human flow direction of each area of the human flow information Ih and comparing the appearance frequency of each numerical value between old and new. Specifically, as illustrated in the upper part of FIG. 7, after creating the human flow information Ih'which is a numerical value of the human flow information Ih, the appearance frequency for each human flow direction in the entire measurement range is calculated based on the human flow information Ih'. A represented histogram (FIG. 7 (a)) is created. Then, a method of calculating 'and the histogram based on the _new, the past of the people flow information Ih' latest pedestrian flow information Ih the people flow amount of change ΔIh by calculating the difference between the histogram based on the _old.

Further, as another comparison method used by the person flow information comparison unit 43, for example, there is a method of comparing old and new people flow information Ih'in which the direction of people flow in each area is quantified. Specifically, as shown in FIG. 7 (b), by comparing the latest human stream information Ih _{'new new} and past human stream information Ih' _old, 1 the area there is a change in the human stream, a change is no area 0 This is a method of calculating the amount of change in human flow ΔIh.

The recognition area candidate output unit 44 outputs a region where there is a change in the human flow as the recognition area candidate C by using the human flow change amount ΔIh calculated by the human flow information comparison unit 43. For example, when the human flow change amount ΔIh shown in FIG. 7B is used, two areas, C _{1 on} the upper left and C _{2 on} the lower right, are output as recognition area candidates C.

As the past human flow information Ih _old acquired from the human flow information history database 42, a method of using the latest information in the past as it is or an average is calculated every time the human flow information Ih is acquired from the human flow information acquisition unit 3. There is a method of using the personal flow information Ih _ave updated by doing so, a method of using the average of the personal flow information Ih at regular intervals, and the like, and there is no particular limitation. Further, the human flow information Ih may be expressed not only by the human flow direction but also by using the human flow direction. For example, in the case of FIG. 7A, the human flow for each human flow direction is used as a weighting coefficient and multiplied by the frequency of each human flow direction. A method of using the combined values as a histogram may also be used. In addition to the method described in this example, another method may be used as long as it can compare the human flow information Ih. In addition to the method of comparing the person flow information Ih based on the position of the person H as in this example, another method is used as long as it is a method that can roughly detect the area where the person flow change is expected to occur. You may. For example, a method such as comparing the optical flow of the entire image may be used.
<Human flow interaction extraction unit 5>
FIG. 8 shows the block configuration of the human flow interaction extraction unit 5. As shown here, the human flow interaction extraction unit 5 recognizes from the latest human flow information Ih _new acquired from the human flow information acquisition unit 3 and the position information of the recognition area candidate C output from the recognition area candidate selection unit 4. It includes a human flow interaction calculation unit 51 that calculates the human flow interaction with respect to the area candidate C, and a human flow interaction output unit 52 that outputs the calculated human flow interaction. Further, the human flow interaction calculation unit 51 includes a positional relationship calculation unit 51a, a flow line change calculation unit 51b, and an attention level calculation unit 51c in order to calculate the human flow interaction. Hereinafter, each function of the human flow interaction calculation unit 51 and the human flow interaction output unit 52 will be described.
<Positional relationship calculation unit 51a>
The positional relationship calculation unit 51a will be described with reference to FIG. In FIG. 9, B ₃ a, B ₃ b, B ₃ c are the latest foot positions of the persons Ha, H b, H c in the three-dimensional space S ₃ , and B ₂ a, B ₂ b, B ₂ c are the xz planes. The latest foot positions of people Ha, Hb, and Hc in S ₂ , C ₁ , C ₂ are recognition area candidates, O ₁ , O ₂ are the centers of each recognition area candidate, and O ₁ Ha, O ₁ Hb, and O ₁ Hc are The foot positions B ₂ a, B ₂ b, B ₂ c of each person, the Euclidean distance of the center O ₁ of the recognition area candidate C ₁ , and th indicate an example of the Euclidean distance threshold.

The positional relationship calculation unit 51a calculates and accumulates the Euclidean distance between each person and the center of each recognition area candidate at a certain time, and generates a graph as shown in FIG. 9C. For example, as shown by the solid line in FIG. 9C, since there is a period in which the Euclidean distance O ₁ Ha between the person Ha and the center O ₁ of the recognition area candidate C ₁ is equal to or less than the threshold value th, the person Ha is the center O _1. It can be seen that there is a period that is quite close to. Further, as indicated by a dashed line, since the Euclidean distance _O 1 Hb between the center _{O 1} of the person Hb recognition area candidate _{C 1} there is a point where the 0, the center _{O 1} of the person Hb recognition area candidates _{C 1} You can see that it has passed.

The positional relationship calculation unit 51a calculates the positional relationship between the person H and the recognition area candidate from the mutation information of these Euclidean distances. The calculation method is not particularly limited as long as it can calculate a value that can quantitatively express the degree of approach of the person H to the recognition area candidate C. For example, there is a method in which the points plotted at a position smaller than the predetermined Euclidean distance threshold th are used as they are as the approach condition. Another method is to use the slope information of the graph between certain plots. As illustrated in O ₁ Ha and O ₁ Hb of FIG. 9 (c), it can be determined whether or not the person H is approaching the recognition area candidate C by finding the positive or negative of the slope of the graph, and from the magnitude of the slope. Since it is possible to determine whether or not the movement of the person H with respect to the recognition area candidate C is positive by calculating the movement speed of the person H, the sum total can be used as a criterion for determining the degree of approach. Also, instead of using all plots in the graph of each person, set multiple thresholds different from the above-mentioned threshold th, and do not use plot points above a certain threshold when calculating the degree of approach, or 2 A method such as using only plot points between two thresholds may be used.
<Flow line change calculation unit 51b>
The flow line change calculation unit 51b calculates the amount of change in the flow line L of the person H with respect to the recognition area candidate C from the flow line information acquired from the flow line information acquisition unit 3. As a method of calculating the change of the flow line L, there is a method of utilizing the direction of the flow line L as shown in the middle part of FIG. For example, the direction of the flow line L of each person is held every time, and the change in the direction of the flow line L in the vicinity of the recognition area candidate C is defined as the amount of change in the flow line L, and the person moving in the back direction. By setting a value range in which the amount of change is 0 if the moving direction of H remains in the back direction, and the maximum value is taken if the moving direction changes in the opposite direction, the flow line related to the person H. The amount of change in L can be calculated. Further, the past human flow information Ih _old in the measurement range may also be used, and when the recognition area candidate C is far away, the person H who has moved in the same direction as the past human flow information Ih _old is the recognition area candidate C. There is also a method of using a criterion that the amount of change in the flow line L becomes large when the person moves in a direction significantly different from the direction of the past human flow information Ih _{old in} the vicinity of.
<Attention calculation unit 51c>
The attention level calculation unit 51c is a function of calculating the attention level to the recognition area candidate C from the information in the line-of-sight direction of the person H with respect to the recognition area candidate C. The method of acquiring the line-of-sight direction information is not particularly limited, such as a method of estimating the line-of-sight direction from the image data P or a method of using a line-of-sight sensor.

As a method of estimating the line-of-sight direction from the image data P, for example, a method of executing head detection on the image, extracting the head region, and estimating from the ratio of black to the head region may be used. If the black area is extremely small, the direction is the direction of the camera, if the black area is large, the direction is opposite to the camera, if the black area is large on the left side, the direction is right, if the black area is large on the right side, the direction is left. It is possible to estimate the viewpoint direction.

In this way, the attention degree calculation unit 51c determines that the more the estimated viewpoint direction is the direction of the recognition area candidate C with respect to the person H, the more the corresponding person H is paying attention to the recognition area candidate C. The degree of attention to the recognition area candidate C can be quantitatively calculated.
<Human flow interaction output unit 52>
The human flow interaction output unit 52 aggregates the human flow interaction information Ii for each recognition area candidate calculated by the human flow interaction calculation unit 51, and outputs the final degree of human flow interaction for each recognition area candidate C.

As a method of calculating the degree of interaction, the interaction information Ii calculated by each of the positional relationship calculation unit 51a, the flow line change calculation unit 51b, and the attention degree calculation unit 51c is calculated using all or part of the information. For example, from the positional relationship calculation unit 51a, the sum of the Euclidean distances of the recognition area candidate C and the person H in the measurement range, and from the movement line change calculation unit 51b, the degree of change in the movement direction of each person in the vicinity of the recognition area candidate C. The sum of the attention of the person H with respect to the recognition area candidate C is received from the attention calculation unit 51c, and the sum of each is rounded to a value of 0 to 5 from a preset range or the like, and the sum of those values is simply summed. Alternatively, there is a method of determining the sum obtained by using a weighting coefficient or the like as the degree of human flow interaction with the recognition area candidate C.

Further, the final interaction information Ii may be determined by using the correlation of the human-flow interaction output from each of the positional relationship calculation unit 51a, the flow line change calculation unit 51b, and the attention level calculation unit 51c. Basically, it is assumed that the person H has a line of sight in the same direction as the traveling direction, and the recognition area candidate C, which is presumed that the person H cannot approach from the information of the positional relationship calculation unit 51a and the flow line change calculation unit 51b, is also included. Regardless, the recognition area candidate C having a high degree of attention output from the attention degree calculation unit 51c may use a method of increasing the final degree of human flow interaction.
<Recognition area determination unit 6>
The recognition area determination unit 6 is a processing target determination unit that determines the execution target of the state recognition process in the state recognition unit 7, and when the target of the state recognition process is an area as in this embodiment, human flow interaction The recognition area candidate C having a high degree of human flow interaction output by the output unit 52 is determined as the recognition area A to be processed by the state recognition unit 7. The recognition area A determined by the recognition area determination unit 6 is not limited to one, and not only the recognition area candidate C having the maximum degree of human flow interaction but also a plurality of recognition area candidates having the subsequent degree of human flow interaction. C may be determined as the recognition area A, while the recognition area A may not be determined when the maximum degree of human-flow interaction is smaller than a predetermined threshold. Further, the number of recognition area candidates to be output may be set with reference to spec information such as the CPU and memory of the state recognition device 1.
<State recognition unit 7>
The state recognition unit 7 executes advanced state recognition processing on the recognition area A determined by the recognition area determination unit 6. As a result, the processing target area of the state recognition unit 7 is narrowed from the entire measurement range to the recognition area A, so that the processing resources required for state recognition can be significantly reduced even when an advanced algorithm is used. The contents of the state recognition process are the process of detecting an abnormal state (congestion, etc.) in the measurement range, the process of detecting an abnormal behavior of the measurement target (fallen person, rampaging person, broken vehicle, etc.), and the surroundings of the measurement target. It is one of the processes of detecting an object and determining the attributes (age and gender of the person, belongings of the person, etc.), but since these processes themselves are the same in the past, detailed description thereof will be omitted.

In the above embodiment, the human flow interaction extraction unit 5 and the recognition area determination unit 6 extract the degree of interaction between the recognition area candidate C and the person H, and determine the recognition area A to execute the state recognition unit 7. Although the method has been described, the person H who executes the state recognition unit 7 may be determined by extracting the degree of interaction between the persons. For example, in the recognition area candidate selection unit 4, the person H in the recognition area candidate C immediately after the change in the human flow is detected is set as the center position, and the person H in the recognition area candidate C and the surroundings are set by the human flow interaction extraction unit 5. A method may be adopted in which the degree of interaction with the person H is calculated, and the person H whose final sum of the degree of interaction is higher than the threshold value is output as the execution target of the state recognition unit 7. In this method, the person H, which is the central position of the human flow interaction extraction unit 5, does not always have to be in the recognition area candidate C, and the recognition area A finally output by the recognition area determination unit 6 selects the recognition area candidate. Instead of the area selected by the unit 4, a range within a certain distance from the person H, which is the central position of the human flow interaction extraction unit 5, may be output as the final recognition area A. Further, it is not necessary for the human flow interaction calculation unit 51 to output the interaction information Ii for all the persons H, and when a plurality of persons are newly detected at the same time, the interaction information Ii is randomly detected so as to be within a predetermined number of persons. A method of selecting the person H or the like may be used.

In the first embodiment of the present invention, in the state recognition device that recognizes the measurement range and the state of the measurement target according to the configuration described above, the mutual measurement targets extracted from the general movement flow information of the measurement target. By selecting in advance the measurement range and measurement target for executing the state recognition process based on the action information Ii, it is possible to execute the advanced state recognition process while reducing the processing resources of the entire device.

In the first embodiment, the state recognition process executed by the state recognition unit 7 includes a function of recognizing an abnormal state in the measurement range, a function of recognizing an abnormal behavior of a measurement target, a highly accurate face recognition, and the like, as well as a person. The function of recognizing the attribute of H is not particularly limited, and is not limited to advanced processing in which the processing resources of the entire device are insufficient to execute the entire measurement range.

Next, the state recognition device 1A according to the second embodiment of the present invention will be described with reference to the functional block diagram of FIG. It should be noted that the common points with the first embodiment will be omitted.

The state recognition device 1A shown in FIG. 10 includes a recognition area A that executes advanced state recognition processing and a camera 2 used at that time based on the interaction information Ii extracted from the human flow information Ih acquired by the plurality of cameras 2. This is a device that is selected in advance.

The state recognition device 1A of FIG. 10 differs from the state recognition device 1 of the first embodiment in that the number of units from the human flow information acquisition unit 3 to the state recognition unit 7 is the same as the number of cameras 2 and information. The point is that the common unit 8 and the state recognition command unit 9 are provided.

The information sharing unit 8 is output from the installation position information database 81 that stores the installation position information of a plurality of cameras, the person flow information database 82 that stores the person flow information Ih acquired from each camera image, and the recognition area determination unit 6 in each camera. The recognition area information database 83 for storing the recognition area information and the recognition target information database 84 for storing the recognition target information are provided, and the state recognition command unit 9 performs state recognition processing for each camera from the information of the information sharing unit 8. Command whether or not to execute. The details of the information sharing unit 8 and the state recognition command unit 9 will be described below.
<Information Sharing Department 8>
The information sharing unit 8 shares and holds the information of the plurality of cameras 2 and the analysis results of the camera images. Each information will be described in detail.

The installation position information database 81 holds information on the installation position and angle of view of the camera in the measurement range. The information held here is, for example, information in which a drawing such as a two-dimensional map of the entire measurement range is prepared in advance, and the shooting area of the camera can be grasped from the position where the camera is installed and the orientation and angle of view of the camera. If so, there is no particular limitation.

The person flow information database 82 holds information on the flow line L of each person acquired by the person flow information acquisition unit 3 and the direction of the person flow in the measurement range in each camera image. By sharing this information and connecting the results between the cameras, analysis using the flow line information of the person H in the recognition area candidate selection unit 4, the human flow interaction extraction unit 5, and the recognition area determination unit 6 in the latter stage. Can be executed with high accuracy.

The recognition area information database 83 and the recognition target information database 84 hold position information such as the recognition area A determined by the recognition area determination unit 6 and the person H to be recognized.
<State recognition command unit 9>
The state recognition command unit 9 determines the camera 2 to execute the state recognition process based on the information obtained from the information sharing unit 8. The determination method includes a method of selecting the camera closest to the recognition area A and the recognition target, a method of determining by considering the information obtained from the installation position information database 81 and the human flow information database 82, and the like, and is not particularly limited. .. To give an example of the latter method, is there a shield between the camera 2 and the recognition area A or the recognition target due to a person H or the like from the installation position of the camera 2, the shooting direction, and the latest human flow information Ih _new ? There is a method of determining whether or not the camera is shielded and instructing the camera that is not expected to be shielded to execute the state recognition process.

In the second embodiment of the present invention described above, in a state recognition device that recognizes a measurement range and a state of a measurement target using a plurality of measurement devices (cameras 2), the general flow of movement of the measurement target. By selecting the measurement range, measurement target, and measurement device to execute the state recognition process in advance based on the interaction information Ii between the measurement targets extracted from the information, the processing resources of the entire device are reduced and the advanced state is achieved. Recognition processing can be executed.

Although the second embodiment has described an example in which each camera retains a function of selecting a recognition area A and a recognition target and a function of executing advanced state recognition processing, a function may be assigned to each camera. For example, when two cameras are used, one camera executes the functions of the human flow information acquisition unit 3, the recognition area candidate selection unit 4, the human flow interaction extraction unit 5, and the recognition area determination unit 6 to set the recognition target. By using a dedicated camera for determining and the other camera as a dedicated camera that executes only advanced state recognition processing, the processing resources of the latter camera can be reduced and more advanced state recognition processing can be applied. Is also good.

Further, in the second embodiment, the information sharing unit 8 may also hold the interaction information Ii output from the human flow interaction extraction unit 5 and share it with each camera. For example, when the recognition area determination unit 6 determines the recognition target, when the degree of interaction between the human flow and the recognition target is too small for the first camera, the final recognition target cannot be determined. Recognition that was overlooked by only one camera because the final degree of interaction exceeds the threshold due to the sum of the degree of interaction with the degree of interaction in the second camera by inheriting the degree of interaction between cameras. The state recognition process can be executed on the target by the second and subsequent cameras.

1,1A state recognition device,
2 cameras,
3 person flow information acquisition department,
31 Video acquisition department,
32 Person position measurement unit,
33 Flow line creation department,
34 Abortion Information Generation Department,
4 Recognition area candidate selection department,
41 Personnel Information Storage Department,
42 people flow information history database,
43 Personnel Information Comparison Department,
44 Recognition area candidate output unit,
5 Human-flow interaction extraction unit,
51 Human-flow interaction calculation unit,
51a Positional relationship calculation unit,
51b Flow line change calculation unit,
51c Attention calculation unit,
52 Human-flow interaction output unit,
6 Recognition area determination unit,
7 State recognition unit,
8 Information Sharing Department,
81 Installation location information database,
82 people flow information database,
83 Recognition area information database,
84 Recognized information database,
9 State recognition command unit

Claims (10)

A motion information generator that generates motion information for the measurement target based on the information acquired by the measuring device,
An interaction extraction unit that extracts interaction information between the space and the measurement target or interaction information between the measurement targets based on the motion information.
Based on the interaction information, a recognition target determination unit that determines a recognition target to execute the state recognition process,
A state recognition unit that executes state recognition processing for the recognition target,
A state recognition device characterized by having. In the state recognition device according to claim 1,
Further, it has a recognition target candidate selection unit that selects recognition target candidates from the entire measurement range of the measuring device based on the motion information.
The interaction extraction unit is a state recognition device that utilizes the relationship between the motion information and the recognition target candidate when extracting the interaction information. In the state recognition device according to claim 1 or 2.
The state recognition process executed by the state recognition unit is any one of a process of detecting an abnormal state in the measurement range of the measuring device, a process of detecting an abnormal behavior of the measurement target, and a process of detecting an attribute of the measurement target. A state recognition device characterized by being. In the state recognition device according to any one of claims 1 to 3.
A state recognition characterized by having a plurality of the measuring devices and determining a measuring device to be used when executing the state recognition process based on the shared information shared by the plurality of measuring devices. apparatus. In claim 4,
A state recognition device characterized in that the shared information is any of the installation position information of the measurement device, the movement information of the measurement target, or the information of the recognition target. In the state recognition device according to claim 4 or 5.
The state recognition device is characterized in that the plurality of measuring devices include at least one dedicated measuring device that executes a state recognition process. In the state recognition device according to any one of claims 1 to 6.
A state recognition device characterized in that the motion information is information on the position and movement direction of the measurement target. In the state recognition device according to any one of claims 1 to 7.
The interaction information extracted by the interaction extraction unit is
A state recognition device characterized in that it is information on any one of the distance between the space and the measurement target, the distance between the measurement targets, or the movement speed, flow line, flow line change amount, and line of sight of the measurement target. In the state recognition device according to any one of claims 1 to 8.
The measuring device is a state recognition device characterized by being any one of a monocular camera, a stereo camera, and a distance sensor. Based on the information acquired by the measuring device, the motion information of the measurement target is generated, and
Based on the motion information, the interaction information between the space and the measurement target or the interaction information between the measurement targets is extracted.
Based on the interaction information, the recognition target to execute the state recognition process is determined.
A state recognition method, characterized in that a state recognition process is executed on the recognition target.

Images