JP6947899B1 - Image analysis system, image analysis method, and image analysis program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide useful information for visually searching for a risk holder. An image analysis system analyzes an image of a risk holder identification unit 22 that detects a person range of a risk holder from a person reflected in image data and a person range of the risk holder, and as a result, of the risk holder. A video analysis server 20 having an image analysis unit 24 that obtains a combination of attributes and their accuracy as personal information analysis data, and a predetermined extraction for comparing the accuracy between attributes belonging to an attribute item composed of a plurality of attributes. The wind body information extraction unit 31 that extracts the attributes that satisfy the conditions from the person information analysis data as the wind body information data to be included in the transmission content, and excludes the attributes that do not meet the predetermined extraction conditions from the transmission content as uncertain information, and the wind body information. A transmission content including data An application server 30 including a wind body information transmission unit 34 for transmitting data to an external terminal 40 of a person concerned is provided. [Selection diagram] Fig. 1

Description

The present invention relates to an image analysis system, an image analysis method, and an image analysis program.

As a technique for automatically extracting an object from image data taken by a camera and tracking the movement of the extracted object, Patent Document 1 describes that the person to be tracked is classified or tagged. Further, the information processing device of Patent Document 1 selects explanatory information of a feature amount (feature amount that contributes to the classification) that has a great influence on the classification of a person, and displays the selected explanatory information on the display device.

Japanese Unexamined Patent Publication No. 2018-156597

There are two main methods for tracking the movement of risk holders automatically extracted from image data. The first method is a method of automatically tracking by image analysis across frames of image data. The second method is a method in which the explanation information of the risk holder is notified to a related person (transmitted person) such as a clerk, and the related person visually searches for the risk holder in the store. Since the camera cannot always capture the entire area of the store, the second method is also used as appropriate.

FIG. 18 is an explanatory diagram showing various situations in which risk holders are tracked.
In public spaces where many people come and go, the general public is confused with fever detected by temperature measurement, suspicious persons who perform annoying acts such as complaints and shoplifting, and risk holders such as lost children such as infants. Therefore, it is necessary for related parties to search for risk holders so that the provision of services to the general public in public spaces will not be hindered.
Stakeholders use their smart devices to check the route taken by the risk holder and search for the risk holder. In addition, when a risk holder is discovered, an alarm is notified to the smart device.

However, since the content of the communication to the parties concerned may be seen by the general public, personal information such as the face image of the risk holder was excluded from the content of the communication so as not to cause discomfort to the general public. Better.
Therefore, as a clue to visually identify the risk holder, consider creating communication content that can be easily understood by humans by verbalizing attribute information such as gender, age, and clothing color. As shown in FIG. 19, the content of the communication to the parties concerned is not useful because the risk holders can be narrowed down as the number increases.

FIG. 19 is a table showing an example of selection of certain attributes and uncertain attributes.
This table shows the attribute items that show the characteristics of risk holders such as gender and age, the attributes that show the values that can be taken as the attribute items, and the accuracy (range of 0 to 1) that shows the certainty as the classification result of each attribute. The closer it is to 1, the more certain it is), and the result indicating whether or not to finally notify as the content of transmission is associated with the accuracy. The accuracy 1 and the result 1 correspond as the first analysis result, and the accuracy 2 and the result 2 correspond as the second analysis result.

With the combination of "attribute: accuracy", for example, "male: 0.9" can be determined to be male, and "male: 0.1" can be determined not to be male. However, the judgment of "not a man" does not immediately mean "a woman", and whether or not a woman is a woman is separately judged by whether or not the accuracy of the "woman" is high.
In the first analysis result, when only one accuracy is outstanding in the attribute item, the accuracy can be determined. For example, “male: 0.9” with accuracy 1 has a significant difference (0.9-0.1 = 0.8) over “female: 0.1” with accuracy 1. Therefore, as a result 1, "male: 0.9" can be determined as the gender.

On the other hand, in the second analysis result, when there are a plurality of attribute items having high accuracy, the accuracy cannot be determined as a candidate. For example, “male: 0.6” with accuracy 2 has a small difference from “female: 0.5” with accuracy 2 (0.6-0.5 = 0.1). Therefore, as a result 2, the gender cannot be determined as either male or female.
If such uncertain information that "gender cannot be determined for both men and women" is included in the content of the transmission, it will not lead to an accurate image of the risk holder, and whether or not the person concerned is the risk holder in the field. It becomes difficult to judge.
In addition, since the conventional technique such as Patent Document 1 presupposes the first method of automatically tracking the risk holder, the explanatory information is only used as an auxiliary and is sent including uncertain information. Therefore, it was not possible to properly search for the relevant person.

Therefore, the main object of the present invention is to provide useful information for visually searching for a risk holder.

In order to solve the above problems, the image analysis system of the present invention has the following features.
The present invention includes a risk holder identification unit that detects a person range of a risk holder from a person reflected in image data, and a risk holder identification unit.
An image analysis server having an image analysis unit that performs image analysis of the person range of the risk holder and, as a result, obtains a combination of the attributes of the risk holder and its accuracy as person information analysis data, and
An attribute that satisfies a predetermined extraction condition for comparing the accuracy between attributes belonging to an attribute item composed of a plurality of attributes is extracted from the person information analysis data as wind body information data to be included in the transmission content, while the predetermined A wind body information extraction unit that excludes attributes that do not satisfy the extraction conditions from the transmitted content as uncertain information,
It is provided with an application server having a wind body information transmission unit that transmits the transmission content including the wind body information data to an external terminal of a person concerned .
As the predetermined extraction condition, the wind body information extraction unit has an accuracy of the first threshold value or more for an attribute item composed of attributes that can be distinguished from each other, and the accuracy of another attribute belonging to the same attribute item. the difference between the case where the second threshold value or more, and that you extract attributes of better likelihood of belonging to the same attribute item is higher as the Futei information data.
Other means will be described later.

According to the present invention, it is possible to provide useful information for visually searching for a risk holder.

It is a block diagram of the image analysis system which concerns on this embodiment. It is a risk holder information table regarding this embodiment. It is an attribute classification table about this embodiment. It is a threshold table about this embodiment. It is a table which shows the extraction condition of the attribute for each group defined in FIG. 3 concerning this embodiment. It is a hardware block diagram of the image analysis system which concerns on this embodiment. It is a flowchart which shows the process of the image analysis server which concerns on this embodiment. It is explanatory drawing of the position information of the risk holder concerning this embodiment. It is image data about this embodiment. It is explanatory drawing of the risk holder concerning this embodiment. It is explanatory drawing which shows an example of the extraction process of the person characteristic amount / attribute which concerns on this embodiment. It is a flowchart which shows the process of the application server which concerns on this Embodiment. It is a flowchart which shows the detail of the extraction process of the attribute information about this embodiment. It is explanatory drawing which shows the specific example of the process of FIG. It is a screen view which shows the display example by converting the risk holder data about this embodiment into an easy-to-understand format. As a modification of FIG. 12 regarding the present embodiment, it is a flowchart to which a process of receiving position information from a related person is added. As a modification of FIG. 12 regarding the present embodiment, it is a flowchart to which a process of causing a related person to acquire an image of a risk holder is added. It is explanatory drawing which shows various situations which track the risk holder about this embodiment. It is a table which shows the selection example of the certain attribute and the uncertain attribute about this embodiment.

Hereinafter, the present embodiment will be described with reference to the drawings.

FIG. 1 is a configuration diagram of an image analysis system.
The image analysis system includes a video capturing device 11, a risk determination device 12, a video analysis server 20, an application server 30, and an external terminal 40. The video analysis server 20 and the application server 30 may be housed in the same housing.
The videographing device 11 captures passersby including the risk holder. The risk determination device 12 is a device for determining whether or not a passerby photographed by the video photographing device 11 is a risk holder, and is, for example, a temperature measuring device for a thermal camera that detects a fever by measuring the temperature.

The image analysis server 20 tracks the risk holder based on the image data of the image capturing device 11 and the determination result of the risk determination device 12. The application server 30 extracts the wind body information (gender, age, etc.) of the risk holder tracked by the video analysis server 20.
The external terminal 40 is a smart device possessed by a related party, and transmits the risk holder's style information notified from the application server 30 to the related party as a clue of the risk holder.

The video analysis server 20 includes a video data processing unit 21, a risk holder identification unit 22, a temporary recording unit 23, an image analysis unit 24, and a person tracking unit 25. Details of each processing unit of these video analysis servers 20 will be clarified by the flowchart of FIG. 7.
The application server 30 has a wind body information extraction unit 31, a risk holder information management unit 32, a DB 33, and a wind body information transmission unit 34. Details of each processing unit of these application servers 30 will be clarified by the flowchart of FIG.
The risk holder information table 33A (details are shown in FIG. 2), the attribute classification table 33B (details are shown in FIG. 3), and the threshold table 33C (details are shown in FIG. 4) are stored in the DB 33.

FIG. 2 is a risk holder information table 33A.
The risk holder information table 33A associates the wind body information extracted by the wind body information extraction unit 31 with the position information tracked by the person tracking unit 25 for each risk holder ID.
For example, the style information of the risk holder ID "001" is for men in their 30s and up, and as the location information, they were in store A at 10:30, but moved to store B at 11:00. Route information is shown.

FIG. 3 is an attribute classification table 33B.
The attribute classification table 33B shows the results of classifying 12 attribute items into 3 groups. The characteristics of the attribute items belonging to each group are similar to each other in the explanation column of the attribute classification table 33B.
For example, in Group G1, there are gender, hairstyle, upper body clothing, lower body clothing, and footwear as attribute items for which it is difficult to determine whether or not a person concerned is a risk holder unless they are clearly distinguished. In other words, if the genders of both are included in the transmitted information at the same time without clearly distinguishing the two attributes of male and female, the person concerned who received the transmitted information is the risk holder of the passerby in front of him. I can't tell if it's not.
These groupings are used for selecting the extraction conditions of attributes suitable for each group, as will be described later in FIG.

Group G1 includes attribute items (attribute items composed of attributes that can be distinguished from each other) that are difficult to distinguish unless they are clearly distinguished for one risk holder, as shown below.
The attribute item "gender" is one of the attributes "male, female".
The attribute item "hairstyle" is one of the attributes "none, short hair, long hair".
The attribute item "upper body clothing" is one of the attributes "sleeveless, short sleeve, long sleeve, casual, formal, jacket, suit, sweater, T-shirt, V-neck, with logo, coarse stripe, fine stripe, plaid, etc." Is.
The attribute item "lower body clothing" is one of the attributes "casual, formal, suit, jeans, long skirt, short skirt, hot pants, shorts, capri pants, slacks, logo, fine stripes, plaid".
The attribute item "footwear" is one of the attributes "boots, leather shoes, sandals, shoes, sneakers, stockings".

As shown below, group G2 includes attribute items (attribute items composed of attributes to be discriminated by the range) that can be discriminated only within a certain range for one risk holder. For example, since red and orange are similar, it is difficult to judge, so they are abstracted into one attribute such as warm colors and reds.
The attribute item "age" can take a value such as the attribute "15 years old or younger, 30 years old or younger, 45 years old or younger, 60 years old or younger, 61 years old or older".
The attribute item "hair color" can take a value such as the attribute "red, orange, yellow, green, purple, brown, gray, white, black".
The attribute items "upper body clothing color", "lower body clothing color", and "footwear color" take values such as the attributes "red, orange, yellow, green, blue, purple, peach, brown, gray, white, and black", respectively. sell.

Group G3 includes attribute items (attribute items that can extract multiple attributes) that may be duplicated for one risk holder, as shown below. For example, one risk holder may carry both a backpack and an umbrella.
The attribute item "decoration" is any one or more of the attributes "mask, hair band, hat, headphones, scarf, muffler, shawl, sunglasses, none".
The attribute item "belongings" is one or more of the attributes "stroller, backpack, folder, carry case, messenger bag, plastic bag, shopping cart, suitcase, umbrella, etc., no belongings".

FIG. 4 is a threshold table 33C.
The threshold table 33C stores an accuracy threshold for determining whether or not the attribute is satisfied. The threshold value table 33C may be a table 33C1 for setting the accuracy threshold value for each group of FIG. 3, a table 33C2 for setting the accuracy threshold value for each attribute item, or the two tables may be used in combination. By using the table 33C2 as the threshold value table 33C, the accuracy threshold value which is the extraction condition of FIG. 5 can be individualized.

Hereinafter, when the accuracy of the attribute "male" is "0.6", it is described as "male: 0.6". For example, since the accuracy threshold of group G1 is "0.5", if the accuracy of the gender belonging to group G1 is "0.5" or more such as "male: 0.6", the attribute "male" of that accuracy is extracted. Moreover, since the accuracy threshold of the hairstyle is "0.4", if it is "0.4" or more like "long hair: 0.5", the attribute "long hair" of the accuracy is extracted.
As described above, the risk holder information table 33A is the result calculated by the application server 30, and the attribute classification table 33B and the threshold table 33C are prepared in advance by the administrator as setting data in the application server 30.
Therefore, the external terminal 40 may display each table in the DB 33 (risk holder information table 33A, attribute classification table 33B, threshold table 33C), or have a related person edit it.

FIG. 5 is a table showing the extraction conditions of the attributes for each group defined in FIG. This table associates the applied extraction condition with the ID of the extraction condition for each group. By setting the extraction conditions individually for each group in this way, the extraction conditions can be adjusted to suit the customer environment, and discovery and tracking can be performed in accordance with the customer's request. The table of FIG. 5 is also prepared in advance in the application server 30 by the administrator as setting data.

For group G1 (those that are difficult to distinguish unless clearly distinguished), attributes are applied from the extraction conditions in the order of G11 → G12 → G13. In addition, for example, there is a description (eg, 0.5 or more) in the extraction condition of G11, but this “0.5” is a certainty threshold value (first threshold value) preset in the threshold value table 33C.
For example, in the case of "male: 0.6, female: 0.5", G11 (0.5 or more) is satisfied by both males and females, but G12 is not satisfied because the difference 0.1 between the two accuracy is not 0.3 (second threshold) or more. Therefore, attributes are not extracted from "male: 0.6, female: 0.5".

Examples of extraction of the attribute item "gender" include the following according to the extraction condition "extract those having an accuracy of 0.5 or more and a relative difference of 0.3 or more".
-The attribute extracted from the attribute "male: 0.9, female: 0.4" is "male".
-The attribute extracted from the attribute "male: 0.6, female: 0.4" is "unknown (not extracted)".
-The attribute extracted from the attribute "male: 0.4, female: 0.1" is "unknown (not extracted)".

For group G2 (those that can be identified only within a certain range), attributes are applied from the extraction conditions in the order of G21 → G22 or G21 → G23. For example, in the case of "red: 0.8, orange: 0.7, others: 0.1", red and orange satisfying G21 (first threshold = 0.5 or more) have a difference of 0.1 between the two accuracy of 0.3 (second threshold) or more. No. Therefore, although it is not possible to clearly distinguish between red and orange, it can be abstracted like a red system in G23. This makes it possible to find risk holders with high accuracy.
On the other hand, in the case of "blue: 0.9, white: 0.8, others: 0.1", blue and white satisfying G21 (0.5 or more) cannot be abstracted by G23, so blue and white are extracted individually. ..

Examples of extraction of the attribute item "age" include the following according to the extraction condition "extract those having an accuracy of 0.5 or more, and if those having an accuracy of 0.5 or more are consecutively extracted, collectively extract".
-The attribute extracted from the attribute "0 to 15 years old: 0.9, 16 to 30 years old: 0.2, 31 to 45 years old: 0.1" is "0 to 15 years old".
-The attribute extracted from the attribute "0 to 15 years old: 0.9, 16 to 30 years old: 0.8, 31 to 45 years old: 0.1" is "0 to 30 years old (collectively, extracted as a large classification)".
-The attribute extracted from the attribute "0 to 15 years old: 0.9, 16 to 30 years old: 0.2, 31 to 45 years old: 0.9" is "Unknown (not extracted)". This is because if the highly accurate 0 to 15 years old and 31 to 45 years old are abstracted as 0 to 45 years old, the less accurate 16 to 30 years old will also be included.

As an example of extracting the attribute item "hair color", the extraction condition "extracts those with an accuracy of 0.5 or more, and when there are multiple items with an accuracy of 0.5 or more, when the color system is close, it is extracted collectively, while when the system is far, each According to "Extract", the following can be mentioned.
-The attribute extracted from the attribute "red: 0.9, orange: 0.2, white: 0.3, blue: 0.1" is "red".
-The attribute extracted from the attribute "red: 0.9, orange: 0.9, white: 0.3, blue: 0.1" is "red system (collectively, extracted as a large classification)".
-The extraction attribute from the attribute "red: 0.1, orange: 0.3, white: 0.9, blue: 0.9" is "white, blue (multiple extraction)".

For group G3 (overlapping), attributes are applied from the extraction conditions in the order of G31 → G32. For example, in G31, only "None" cannot be compatible with other attributes, but in G32, possession of "bag" and "umbrella" is compatible.
As an example of extracting the attribute item "decorative item", the extraction condition "If all items with an accuracy of 0.5 (first threshold value) or more are extracted and the attribute [none] is extracted, no attribute of the decorative item is extracted". According to, the following can be mentioned.
-The extracted attribute from the attribute "A: 0.8, B: 0.9, C: 0.1, D: 0.1, none: 0.1" is "A, B".
-The extraction attribute from the attribute "A: 0.8, B: 0.1, C: 0.1, D ...: 0.1, none: 0.9" is "none (not extracted)".

FIG. 6 is a hardware configuration diagram of the image analysis system.
A computer 900 such as a video analysis server 20 or an application server 30 has a CPU 901, a RAM 902, a ROM 903, an HDD 904, a communication I / F 905, an input / output I / F 906, and a media I / F 907.
The communication I / F 905 is connected to an external communication device 915. The input / output I / F 906 is connected to the input / output device 916. The media I / F 907 reads and writes data from the recording medium 917. Further, the CPU 901 controls each processing unit by executing a program (also referred to as an application or an abbreviation thereof) read into the RAM 902. Then, this program can be distributed via a communication line, or can be recorded and distributed on a recording medium 917 such as a CD-ROM.

FIG. 7 is a flowchart showing the processing of the video analysis server 20.
The video capturing device 11 captures video data of a passerby (S101). The video data processing unit 21 controls the video capturing device 11. This control content is, for example, instructing the shooting parameters (angle of view, etc.) of the video shooting device 11 and converting the format of the video data received from the video shooting device 11 into a format that facilitates image analysis. .. The video data processing unit 21 receives the video data of S101 and converts it into image data (S102).

The risk determination device 12 determines the position information of the risk holder reflected in the image data (S103). The photographing process of S101 and the risk determination process of S103 are actually performed in parallel in the same public space.
The risk holder identification unit 22 detects a person range in the image data of the risk holder from the image data of S102 and the position information of the risk holder in S103 (S104), and obtains the detected image data of the risk holder. Temporary recording is performed in the temporary recording unit 23 (S105).

The image analysis unit 24 acquires the temporarily recorded image data (S111), and extracts the feature amount and the attribute of the person from the image data (S112). Then, the image analysis unit 24 outputs the person information analysis data (characteristics and attributes of the person), which is the extraction result of S112, to the application server 30 (S113).

The person tracking unit 25 obtains the movement history data of the risk holder by tracking between the frames of each image data constituting the video data of S101. Therefore, the person tracking unit 25 acquires the person ID and the person feature amount for each passerby (S121), and detects the same person (risk holder) from the person feature amount (S122). Then, the person tracking unit 25 outputs the result of tracking the same person in S122 as movement history data to the application server 30 (S123).

FIG. 8 is an explanatory diagram of the position information 100 of the risk holder determined in S103.
Here, the risk determination device 12 uses the position information (X coordinate = 100 Y coordinate = 80 person range width = 95 person range height = 240) in which the risk holder is shown in the image data, and the risk breakdown. (Fever of 37.5 degrees) is detected as a certain risk holder information 101 and transmitted to the image analysis server 20.

FIG. 9 is the image data 110 converted in S102. The following two people are shown in the image data 111.
-"Person ID: 1" is located at "X coordinate = 386 Y coordinate = 224 width of person range = 95 height of person range = 240" (reference numeral 112).
-"Person ID: 2" is located at "X coordinate = 100 Y coordinate = 80 width of person range = 95 height of person range = 240" (reference numeral 113).

FIG. 10 is an explanatory diagram of the risk holder detected in S104.
The detection result 120 includes image data 121 including a person range of "person ID: 1" and detailed information 122 of "person ID: 1". The detailed information 122 is such that the risk holder information 101 and the position information of the reference numeral 112 are associated with each other by the same position.

FIG. 11 is an explanatory diagram showing an example of the extraction process of the person feature amount / attribute of S112.
The image analysis unit 24 inputs the person detection image 130 into the convolution layer 140 (first layer 141, second layer 142, third layer 143) of the deep learning module of artificial intelligence. Then, the feature of the person detection image 130 is extracted as the accuracy 160 for each attribute through the fully connected layer 150 (first layer 151, second layer 152, third layer 153). The first layer 151 shows the feature amount, each node of the third layer 153 corresponds to each attribute, and it can be said that the closer the accuracy 160 is to 1, the more the attribute is.
The image analysis unit 24 is not limited to the Deep Learning module, and may obtain the accuracy 160 from the person detection image 130 using an arbitrary classifier.

FIG. 12 is a flowchart showing the processing of the application server 30.
The wind body information extraction unit 31 acquires the person information analysis data output in S113 (S201), and extracts the attribute information satisfying the extraction condition of FIG. 5 (S202, details of FIG. 13). Then, the wind body information extraction unit 31 outputs the extraction result of S202 as wind body information data to the risk holder information management unit 32 (S203).
The risk holder information management unit 32 acquires the movement history data of S123 (S211), and integrates the wind body information data of S203 and the movement history data of S211 (S212).

The risk holder information management unit 32 outputs the integration result of S212 as risk holder data to the risk holder information table 33A (S213). The wind body information transmission unit 34 transmits the risk holder data of the risk holder information table 33A to the external terminal 40 of the related party (S214).
As an opportunity to transmit the risk holder data to the external terminal 40, the wind body information transmission unit 34 may autonomously transmit the risk holder data to each external terminal 40 after the calculation of the risk holder data (Push type transmission). Alternatively, when the current inquiry is received from the external terminal 40, the risk holder data may be transmitted from the wind body information transmitting unit 34 as a response to the inquiry (pull type transmission).

FIG. 13 is a flowchart showing details of the attribute information extraction process (S202). The wind body information extraction unit 31 executes the following flowchart with reference to the attribute classification table 33B, the threshold value table 33C, and the table showing the extraction conditions of FIG.
The wind body information extraction unit 31 determines whether or not there is an attribute item including an attribute having an accuracy equal to or higher than a certain value (defined as an accuracy threshold in G11, G21, and G32) (S301). If Yes in S301, proceed to S302, and if No, proceed to S344.

The wind body information extraction unit 31 selects an attribute item corresponding to S301 (S302), and determines whether or not the selected attribute item corresponds to G3 (S331). If Yes in S331, proceed to S332, and if No, proceed to S311.
The wind body information extraction unit 31 determines whether or not there is an attribute with a higher accuracy than the attribute “none” (corresponding to G31) (S332). If Yes in S332, proceed to S344, and if No, proceed to S333. The wind body information extraction unit 31 selects an attribute corresponding to G32 and having a certain degree of accuracy or higher (S333).

The wind body information extraction unit 31 determines whether or not there is a difference of a certain value or more between the highest accuracy in the selected attribute item and the next point (whether or not it corresponds to G12) (S311). If Yes in S311, the process proceeds to S312, and if No, the process proceeds to S321. The wind body information extraction unit 31 selects the attribute corresponding to G13 with the highest accuracy (S312).
The wind body information extraction unit 31 determines whether or not the attribute item selected in S302 corresponds to G2 (S321). If Yes in S321, proceed to S322, and if No, proceed to S344. The wind body information extraction unit 31 abstracts and selects a plurality of attributes corresponding to G23 (S322).

The wind body information extraction unit 31 extracts the attributes selected in S312, S322, and S333 (S341). Alternatively, the wind body information extraction unit 31 does not extract attributes from the attribute items selected in S302 (S344).
The wind body information extraction unit 31 determines whether or not there is an attribute item that has not yet been selected in S302 (S342). If Yes in S342, the process proceeds to S343, and if No, the process of FIG. 13 ends. The wind body information extraction unit 31 newly selects an unselected (another) attribute item (S343), and returns the process to S331.

FIG. 14 is an explanatory diagram showing a specific example of the process of FIG.
Table 201 shows the person information analysis data acquired by the wind body information extraction unit 31 in S201. Table 201 has a mixture of highly accurate attributes such as "female: 0.8" and less accurate attributes such as "blue shoes: 0.2". Therefore, if the table 201 is notified to the external terminal 40 as it is, the attribute with low accuracy becomes noise and hinders the narrowing down of risk holders from passers-by.

Table 202 shows the result of the wind body information extraction unit 31 extracting the attribute information satisfying the extraction condition in S202 from the table 201. As for the extracted attributes (extracted attributes) of the table 202, only certain attributes that satisfy the extraction conditions are selected from the table 201. By narrowing down the reports to the attributes with high accuracy in this way, it is easy to find the target person.
The wind body information data 203 is a collection of the extraction attributes of the table 202 as verbalized character information by the wind body information extraction unit 31, and is the data output to the risk holder information management unit 32 in S203. By converting the raw data (list of numerical values) described in FIG. 11 into character information by the wind body information extraction unit 31, the format becomes readable by the persons concerned, and the search for the risk holder by the persons concerned becomes easy.

The movement history data 204 is data acquired by the risk holder information management unit 32 in S211 and indicates the movement route (position and time at which the risk holder data was discovered) of the risk holder in the wind body information data 203. Since the movement history data 204 includes the movement route of the risk holder to the current position, it is useful for narrowing down the risk holder.
The movement history data 204 may be the result of image analysis (mechanical tracking result) output by the person tracking unit 25, or may be the result of a human search reported by a person concerned (details are shown in the figure). 16), both may be used together.

The risk holder data 205 is data in which the risk holder information management unit 32 integrates the wind body information data 203 and the movement history data 204 in S212, and is transmitted to the external terminal 40 by e-mail or the like in S214. The risk holder data 205 may include a link for viewing the face image of the risk holder (details are shown in FIG. 17).

FIG. 15 is a screen view showing a display example by converting the risk holder data 205 into an easy-to-understand format.
The display column 211 shows a display example in which "female: 0.8" in the table 202 of FIG. 14 is converted into a graph. This graph is obtained by converting the wind body information data of the character information into graph information showing numerical values according to the accuracy. The accuracy of men increases as it approaches the left side, and the accuracy of women increases as it approaches the right side. For example, by displaying a star mark at a position corresponding to the value of (female accuracy) ÷ ((male accuracy) + (female accuracy)), the accuracy of a woman can be determined rather than simply indicating whether it is a man or a woman. Can be intuitively understood by the parties concerned.

In addition, a graph showing "age: teenager" is also displayed in the display column 211. In this graph, the left side is 0 years old and the right side is 100 years old, and by emphasizing the corresponding period of teenagers, the person concerned can intuitively understand "age: teenager".
The display field 212 shows a display example in which the movement history data 204 of FIG. 14 is converted into a plane map format. The person concerned can intuitively understand the movement route of the risk holder who is recognized as a fever by the temperature measurement.

FIG. 16 is a flowchart in which a process of receiving position information from a related party is added as a modification of FIG.
In FIG. 16, S221 and S222 are newly added to the process of FIG. When the risk holder information management unit 32 receives the risk holder's position information (discovery position, discovery time) from the external terminal 40 (S221, Yes), the risk holder information management unit 32 adds the received position information to the movement history data 204. , Update the location information (S222).
As a result, in addition to the mechanical tracking result by the person tracking unit 25, detailed movement history data 204 including the result of the human search fed back from the related person can be provided to the related person, and highly accurate tracking is possible. Become.

FIG. 17 is a flowchart in which a process of having a related party acquire an image of a risk holder is added as a modification of FIG.
In FIG. 17, S211B and S11C were newly added to the process of FIG. 12, and S212 was replaced with S212B.
The risk holder information management unit 32 acquires an image of the risk holder from the video capturing device 11 (S211B), saves the image in the DB 33, and acquires a link of the save destination (S211C).
Then, the risk holder information management unit 32 integrates the wind body information data of S203, the movement history data of S211 and the link of the storage destination of S211C (S212B).

As a result, as shown in the risk holder data 205 of FIG. 14, the transmitted information includes an image link. Therefore, the people concerned protect the privacy by not clicking the link of the image in the place where there are many passersby and there is a concern about peeping on the display screen (such as a cash register). On the other hand, related parties can check the image of the risk holder by clicking the image link in a place with few passersby (waiting room, etc.), and highly accurate tracking is possible.

In the present embodiment described above, the wind body information extraction unit 31 of the application server 30 sets the attribute that satisfies a predetermined extraction condition for comparing the accuracy between the attributes belonging to the attribute item composed of a plurality of attributes as personal information. While extracting from the analysis data as wind body information data to be included in the transmission content, attributes that do not satisfy the predetermined extraction conditions are excluded from the transmission content as uncertain information. Then, the wind body information transmission unit 34 transmits the transmission content including the wind body information data to the external terminal 40 of the person concerned.
As a result, by comparing the accuracy between the attributes belonging to the attribute item, if there are multiple attributes with high accuracy in the same attribute item as shown in accuracy 2 in FIG. 19, the predetermined extraction conditions are satisfied. It can be appropriately excluded as uncertain information that does not meet. Therefore, it is possible to narrow down to useful information (accuracy satisfying a predetermined extraction condition) for visually searching for a risk holder as shown by accuracy 1 in FIG. 19 and provide it to the external terminal 40.

The present invention is not limited to the above-described embodiment, and includes various modifications. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to those having all the described configurations.
Further, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment.
Further, it is possible to add / delete / replace a part of the configuration of each embodiment with another configuration. Further, each of the above configurations, functions, processing units, processing means and the like may be realized by hardware by designing a part or all of them by, for example, an integrated circuit.
Further, each of the above configurations, functions, and the like may be realized by software by the processor interpreting and executing a program that realizes each function.
As an example, by adding text-to-speech software to the wind body information transmission unit 34, a voice transmission method is possible. In addition, by converting the wind body information verbalized by the wind body information extraction unit 31 into 3D-CG using 3D-CG software, information that can be visually confirmed excluding personal information can be transmitted. It is possible.

Information such as programs, tables, and files that realize each function can be stored in memory, hard disks, recording devices such as SSDs (Solid State Drives), IC (Integrated Circuit) cards, SD cards, DVDs (Digital Versatile Discs), etc. Can be placed on the recording medium of. You can also use the cloud.
In addition, the control lines and information lines indicate those that are considered necessary for explanation, and do not necessarily indicate all the control lines and information lines in the product. In practice, it can be considered that almost all configurations are interconnected.
Further, the communication means for connecting each device is not limited to the wireless LAN, and may be changed to a wired LAN or other communication means.

11 Video shooting device 12 Risk judgment device 20 Video analysis server 21 Video data processing unit 22 Risk holder identification unit 23 Temporary recording unit 24 Image analysis department 25 Person tracking department 30 Application server 31 Wind body information extraction department 32 Risk holder information management department 33 DB (database)
33A Risk holder information table 33B Attribute classification table 33C Threshold table 34 Wind body information transmitter 40 External terminal

Claims (12)

The risk holder identification unit that detects the range of risk holders from the person reflected in the image data,
An image analysis server having an image analysis unit that performs image analysis of the person range of the risk holder and, as a result, obtains a combination of the attributes of the risk holder and its accuracy as person information analysis data, and
An attribute that satisfies a predetermined extraction condition for comparing the accuracy between attributes belonging to an attribute item composed of a plurality of attributes is extracted from the person information analysis data as wind body information data to be included in the transmission content, while the predetermined A wind body information extraction unit that excludes attributes that do not satisfy the extraction conditions from the transmitted content as uncertain information,
It is provided with an application server having a wind body information transmission unit that transmits the transmission content including the wind body information data to an external terminal of a person concerned .
As the predetermined extraction condition, the wind body information extraction unit has an accuracy of the first threshold value or more for an attribute item composed of attributes that can be distinguished from each other, and the accuracy of another attribute belonging to the same attribute item. image analysis system for the difference may be smaller than the second threshold value, characterized that you extract attributes of better likelihood of belonging to the same attribute item is higher as the Futei information data with. The risk holder identification unit that detects the range of risk holders from the person reflected in the image data,
An image analysis server having an image analysis unit that performs image analysis of the person range of the risk holder and, as a result, obtains a combination of the attributes of the risk holder and its accuracy as person information analysis data, and
An attribute that satisfies a predetermined extraction condition for comparing the accuracy between attributes belonging to an attribute item composed of a plurality of attributes is extracted from the person information analysis data as wind body information data to be included in the transmission content, while the predetermined A wind body information extraction unit that excludes attributes that do not satisfy the extraction conditions from the transmitted content as uncertain information,
It is provided with an application server having a wind body information transmission unit that transmits the transmission content including the wind body information data to an external terminal of a person concerned .
As the predetermined extraction condition, the wind body information extraction unit has an accuracy of the first threshold value or more and the accuracy of other attributes belonging to the same attribute item with respect to the attribute item composed of the attributes to be determined in the range. If the difference between is smaller than the second threshold value, an image, wherein that you extract attributes of one range abstracting range of a plurality of attributes with the accuracy equal to or larger than the first threshold value as the Futei information data Analysis system. When the wind body information extraction unit cannot abstract the range of a plurality of attributes having a certainty of the first threshold value or more as the predetermined extraction condition, the wind body information extraction unit individually sets the range of the plurality of attributes as the wind body information data. Characterized by extracting as
The image analysis system according to claim 2. The risk holder identification unit that detects the range of risk holders from the person reflected in the image data,
An image analysis server having an image analysis unit that performs image analysis of the person range of the risk holder and, as a result, obtains a combination of the attributes of the risk holder and its accuracy as person information analysis data, and
An attribute that satisfies a predetermined extraction condition for comparing the accuracy between attributes belonging to an attribute item composed of a plurality of attributes is extracted from the person information analysis data as wind body information data to be included in the transmission content, while the predetermined A wind body information extraction unit that excludes attributes that do not satisfy the extraction conditions from the transmitted content as uncertain information,
It is provided with an application server having a wind body information transmission unit that transmits the transmission content including the wind body information data to an external terminal of a person concerned .
As the predetermined extraction condition, the wind body information extraction unit extracts an attribute from the attribute item when the accuracy when there is no attribute to be extracted is equal to or higher than the first threshold value for the attribute item capable of extracting a plurality of attributes. An image analysis system characterized by not extracting. The wind body information extraction unit is characterized in that the first threshold value set in advance for each attribute item is read from the database and used for the predetermined extraction condition.
The image analysis system according to any one of claims 1 to 3. The risk holder identification unit that detects the range of risk holders from the person reflected in the image data,
An image analysis server having an image analysis unit that performs image analysis of the person range of the risk holder and, as a result, obtains a combination of the attributes of the risk holder and its accuracy as person information analysis data, and
An attribute that satisfies a predetermined extraction condition for comparing the accuracy between attributes belonging to an attribute item composed of a plurality of attributes is extracted from the person information analysis data as wind body information data to be included in the transmission content, while the predetermined A wind body information extraction unit that excludes attributes that do not satisfy the extraction conditions from the transmitted content as uncertain information,
It is provided with an application server having a wind body information transmission unit that transmits the transmission content including the wind body information data to an external terminal of a person concerned .
The video analysis server further has a person tracking unit that obtains movement history data of the risk holder by tracking between frames of the image data for the risk holder reflected in the image data.
The application server further transmits the wind body information data and the movement history data in order to transmit the transmission content including the movement history data from the wind body information transmission unit to the external terminal in addition to the wind body information data. It has a risk holder information management department to integrate
The risk retention information management unit, an image analysis system and adding to said Rukoto position information risk holders fed back from the external terminal to the movement history data. The risk holder identification unit that detects the range of risk holders from the person reflected in the image data,
An image analysis server having an image analysis unit that performs image analysis of the person range of the risk holder and, as a result, obtains a combination of the attributes of the risk holder and its accuracy as person information analysis data, and
An attribute that satisfies a predetermined extraction condition for comparing the accuracy between attributes belonging to an attribute item composed of a plurality of attributes is extracted from the person information analysis data as wind body information data to be included in the transmission content, while the predetermined A wind body information extraction unit that excludes attributes that do not satisfy the extraction conditions as uncertain information from the above-mentioned transmission contents,
It is provided with an application server having a wind body information transmission unit that transmits the transmission content including the wind body information data to an external terminal of a person concerned .
The video analysis server further has a person tracking unit that obtains movement history data of the risk holder by tracking between frames of the image data for the risk holder reflected in the image data.
The application server further transmits the wind body information data and the movement history data in order to transmit the transmission content including the movement history data from the wind body information transmission unit to the external terminal in addition to the wind body information data. It has a risk holder information management department to integrate
The risk holder information management unit stores the image data of the risk holder reflected in the image data in the database, receives the link information for acquiring the image data from the database, and sends the link information to the external terminal. image analysis system, characterized that you transfer. 7. The image analysis system according to item 1. The image analysis system includes a video analysis server having a risk holder identification unit and an image analysis unit, and an application server having a wind body information extraction unit and a wind body information transmission unit.
The risk holder identification unit detects the person range of the risk holder from the person reflected in the image data, and then
The image analysis unit performs image analysis of the person range of the risk holder, and as a result, obtains the combination of the attributes of the risk holder and the accuracy thereof as the person information analysis data.
The wind body information extraction unit includes attributes that satisfy a predetermined extraction condition for comparing the accuracy between attributes belonging to an attribute item composed of a plurality of attributes as wind body information data to be included in the transmission contents from the person information analysis data. On the other hand, attributes that do not satisfy the predetermined extraction conditions are excluded from the transmitted content as uncertain information.
As the predetermined extraction condition, the wind body information extraction unit has an accuracy of the first threshold value or more for an attribute item composed of attributes that can be distinguished from each other, and the accuracy of another attribute belonging to the same attribute item. When the difference from is equal to or greater than the second threshold value, the attribute having the higher probability of belonging to the same attribute item is extracted as the wind body information data.
The image analysis method, characterized in that the wind body information transmitting unit transmits the transmission content including the wind body information data to an external terminal of a person concerned. The image analysis system includes a video analysis server having a risk holder identification unit and an image analysis unit, and an application server having a wind body information extraction unit and a wind body information transmission unit.
The risk holder identification unit detects the person range of the risk holder from the person reflected in the image data, and then
The image analysis unit performs image analysis of the person range of the risk holder, and as a result, obtains the combination of the attributes of the risk holder and the accuracy thereof as the person information analysis data.
The wind body information extraction unit includes attributes that satisfy a predetermined extraction condition for comparing the accuracy between attributes belonging to an attribute item composed of a plurality of attributes as wind body information data to be included in the transmission contents from the person information analysis data. On the other hand, attributes that do not satisfy the predetermined extraction conditions are excluded from the transmitted content as uncertain information.
As the predetermined extraction condition, the wind body information extraction unit has an accuracy of the first threshold value or more and the accuracy of other attributes belonging to the same attribute item with respect to the attribute item composed of the attributes to be determined in the range. When the difference from the second threshold value is less than the second threshold value, the attribute of one range that abstracts the range of the plurality of attributes having the accuracy of the first threshold value or higher is extracted as the wind body information data.
The image analysis method, characterized in that the wind body information transmitting unit transmits the transmission content including the wind body information data to an external terminal of a person concerned. The image analysis system includes a video analysis server having a risk holder identification unit and an image analysis unit, and an application server having a wind body information extraction unit and a wind body information transmission unit.
The risk holder identification unit detects the person range of the risk holder from the person reflected in the image data, and then
The image analysis unit performs image analysis of the person range of the risk holder, and as a result, obtains the combination of the attributes of the risk holder and the accuracy thereof as the person information analysis data.
The wind body information extraction unit includes attributes that satisfy a predetermined extraction condition for comparing the accuracy between attributes belonging to an attribute item composed of a plurality of attributes as wind body information data to be included in the transmission contents from the person information analysis data. On the other hand, attributes that do not satisfy the predetermined extraction conditions are excluded from the transmitted content as uncertain information.
As the predetermined extraction condition, the wind body information extraction unit extracts an attribute from the attribute item when the accuracy when there is no attribute to be extracted is equal to or higher than the first threshold value for the attribute item capable of extracting a plurality of attributes. I decided not to extract
The image analysis method, characterized in that the wind body information transmitting unit transmits the transmission content including the wind body information data to an external terminal of a person concerned. An image analysis program for causing the video analysis server and the application server to execute the image analysis method according to any one of claims 9 to 11.

Images