JP7318321B2 - Information processing device, information processing method, person search system, and person search method - Google Patents

Description

The present invention relates to an information processing apparatus and information processing method for extracting a feature amount relating to attributes of a person from an image, a person search system using the information processing apparatus, and a person search method using the information processing method.

2. Description of the Related Art Conventionally, various techniques have been proposed for retrieving a person from an image of the person. For example, in Patent Literature 1, a search query for person search is generated from an input image including a full-length image of a person, an image database is searched using the generated search query, and the search result is displayed. .

Further, Patent Document 2 discloses a technique of searching for a target person from a plurality of images using bib number or uniform number as a key. An image in which the target person appears in the foreground is extracted according to the degree of superimposition of the body region estimated from the face region of the image and the shielding state such as the bib number.

Further, Patent Document 3 discloses a technique for extracting a photographed image suitable for a user from a plurality of photographed images. Specifically, the user is allowed to select specific conditions such as face, height, positional relationship, and motion, and appropriate images are extracted using each as a key. For example, if a "face" is specified, an image that is easy to recognize, such as a frontal face, can be extracted. I am trying to extract the image and display it to the user.

JP 2016-162414 A JP 2018-25966 A JP 2017-28688 A

By the way, for example, in an image captured by a surveillance camera, there is a possibility that an unspecified number of people are captured, or a person overlaps with an object such as a signboard, so that a part of the full-body image of the person may be blocked. In Japanese Patent Laid-Open No. 2004-100001, it is assumed that an image in which a full-length image of a person is not blocked is used for person search. There is a possibility that it cannot be performed with high accuracy, and in actual operation, images used for person search are limited to images showing a full body image of a person.

Further, in Patent Document 2, regarding person search, it is assumed that an image contains an item that can identify a person to be searched, such as a bib number. ), it is difficult to perform person search. Therefore, the images used for person search are substantially limited to images containing the above items.

In addition, surveillance cameras obtain images by photographing an unspecified number of people obliquely from above, so it is difficult to obtain an image in which the person's face is reflected from the front. is also difficult to obtain. For this reason, in Patent Document 3, it is difficult to use an image captured by a surveillance camera for person search, and the image used for person search is substantially limited to an image including a front view.

The present invention has been made to solve the above problems, and its object is to avoid the limitation of images used for person search to specific images, and is suitable for a system that performs highly accurate person search. It is an object of the present invention to provide an information processing apparatus and information processing method, a person search system, and a person search method that can be used for

An information processing apparatus according to one aspect of the present invention includes a control point information acquisition unit that acquires control point information including position information of control points that define a skeleton of the person from an image of the person; A feature amount information acquisition unit that acquires feature amount information including a feature amount related to attributes of the person from the image, and a certainty factor that indicates the reliability of the feature amount based on the control point information and the feature amount information. and a certainty determination unit for determining.

A person search system according to another aspect of the present invention includes the above information processing device, a database that stores the certainty of the feature amount determined by the information processing device, and in response to a search request from the outside, A person search unit that searches for a person having a feature amount that matches a search condition based on the certainty stored in the database, and a search result output unit that outputs a search result obtained by the person search unit.

An information processing method according to still another aspect of the present invention includes a control point information acquiring step of acquiring control point information including position information of control points that define the skeleton of the person from an image of the person. a feature quantity information acquiring step of acquiring feature quantity information including a feature quantity relating to attributes of the person from the image; and confidence indicating reliability of the feature quantity based on the control point information and the feature quantity information. and a confidence determination step of determining the degree of confidence.

A person search method according to still another aspect of the present invention is a person search method using the information processing method described above, wherein the certainty factor of the feature amount determined by the information processing method is stored in a database. a person search step of searching for a person having a feature quantity that matches a search condition based on the certainty stored in the database in response to a search request from the outside; and a search result of the person search step. and a search result output step of outputting.

Based on the control point information and the feature amount information obtained from the photographed image of the person, in order to determine the degree of certainty of the feature amount related to the attributes of the person, for example, a full-body image of the person is obtained by capturing an unspecified number of persons. Also, even if it is difficult to obtain a frontal image of the face or a part of the person may be hidden, even using an image obtained by a surveillance camera, the person can be searched with high accuracy from the above image. necessary information (confidence) can be obtained. Therefore, an information processing apparatus and an information processing method that are suitable for a system that performs highly accurate person retrieval can be avoided from restricting the image used for person retrieval to a specific image such as an image showing a full-length image of a person. Realization is possible. For example, if the above information (certainty) is stored in a database, for a person search query sent from the outside, feature values that match the search conditions are extracted in descending order of certainty, and the corresponding person is retrieved. It becomes possible to perform with high accuracy.

1 is an explanatory diagram showing a schematic configuration of a person search system according to an embodiment of the present invention; FIG. It is a block diagram which shows the structure of the outline of the information processing apparatus contained in the said person search system. It is a block diagram showing a schematic configuration of a person search device included in the person search system. 4 is a flow chart showing a flow of processing by a person search method implemented in the person search system; FIG. 10 is an explanatory diagram showing an example of an image in which an unspecified number of people are reflected; FIG. 4 is an explanatory diagram schematically showing an example of the positional relationship between two persons in an image; FIG. 5 is an explanatory diagram schematically showing another example of the positional relationship; FIG. 9 is an explanatory diagram schematically showing still another example of the positional relationship; FIG. 4 is an explanatory diagram showing the relationship between attributes of a person, feature amounts related to the attributes, and positions of control points related to the attributes; FIG. 4 is an explanatory diagram schematically showing an example of an image of each frame captured by a camera and a feature amount relating to an attribute extracted from the image; FIG. 10 is an explanatory diagram showing an example of processing for determining the certainty of a feature amount; FIG. 9 is an explanatory diagram showing another example of the process of determining the certainty factor; FIG. 4 is an explanatory diagram showing an example of information stored in a database; It is explanatory drawing which shows an example of the image which image|photographed the inside of the store. FIG. 4 is an explanatory diagram showing an example of an image in which a person is facing forward; FIG. 4 is an explanatory diagram showing an example of an image in which a person is facing backward; FIG. 5 is a flow chart showing the details of the confidence determination process of FIG. 4; FIG. FIG. 4 is an explanatory diagram schematically showing an example of an image in which a person and an object overlap; FIG. 11 is an explanatory diagram schematically showing another example of an image in which a person and an object overlap; FIG. 11 is an explanatory diagram schematically showing still another example of an image in which a person and an object overlap; FIG. 11 is an explanatory diagram schematically showing still another example of an image in which a person and an object overlap;

Each embodiment of the present invention will be described below with reference to the drawings. In addition, the present invention is not limited to the following contents.

[Person search system]
FIG. 1 is an explanatory diagram showing a schematic configuration of a person search system 1 of this embodiment. A person search system 1 includes a camera 2 , an information processing device 3 , and a person search device 4 . The camera 2, the information processing device 3, and the person search device 4 are communicably connected to each other via a communication line NW. The communication line NW is configured by appropriately selecting from, for example, a cable, optical fiber, wired LAN (Local Area Network), wireless LAN, Internet line, and the like. Details of the camera 2, the information processing device 3, and the person search device 4 will be described below.

(camera)
The camera 2 is installed, for example, on the ceiling or wall of indoor/outdoor commercial facilities, public facilities, shops, etc., and is composed of a camera (image acquisition unit) that captures a person from above and acquires images of frames that differ in time. It is In this embodiment, as the camera 2, for example, a surveillance camera installed in a store is considered. The camera 2 is, for example, an RGB camera having pixels corresponding to R (red), G (green), and B (blue). A color image of a person can be obtained by photographing with an RGB camera.

The number of cameras 2 installed in a store or the like is not particularly limited, and may be one or two or more. Image data acquired by at least one camera 2 is provided to the information processing device 3 via the communication line NW.

(Information processing device)
FIG. 2 is a block diagram showing a schematic configuration of the information processing device 3. As shown in FIG. The information processing device 3 extracts information necessary for person search (for example, the attribute of the person described later) from the image acquired by the camera 2 and input to the information processing device 3, that is, the image of each frame in which the person is photographed from above. is a terminal device that acquires the degree of confidence of the feature amount related to, and is configured by, for example, a personal computer. The information processing device 3 may be installed in the same store where the camera 2 is installed, or may be installed outside the store and connected to the camera 2 so as to be communicable.

Such an information processing device 3 includes an input unit 31, a storage unit 32, a display unit 33, a communication unit 34, a reading unit 35, a control point information acquisition unit 36, a feature amount information acquisition unit 37, It is configured with a certainty determination unit 38 and a control unit 39 .

The input unit 31 is composed of, for example, a keyboard, a mouse, a touch pad, a touch panel, etc., and receives various instruction inputs from an operator who operates the information processing device 3, for example.

The storage unit 32 stores an operation program for operating each unit of the information processing device 3, image data input from the camera 2, information (control point information, feature amount information, etc.). The storage unit 32 is composed of, for example, a hard disk, but may be configured by appropriately selecting from recording media such as RAM (Random Access Memory), ROM (Read Only Memory), optical disk, magneto-optical disk, and non-volatile memory. good.

The display unit 33 is a device that displays various types of information, and is configured by, for example, a liquid crystal display device. The communication unit 34 is an interface including an input/output port for communicating with the outside. The communication unit 34 includes an antenna, a transmission/reception circuit, a modulation circuit, a demodulation circuit, and the like when performing wireless communication with the outside.

The reading unit 35 is a device for reading information recorded on the recording medium R (for example, an operation program of the control unit 39), and is composed of, for example, a disk drive. If the recording medium R is a portable non-volatile memory, the reading unit 35 also includes a connection port into which the connection unit of the non-volatile memory is inserted.

The control section 39 is composed of a central processing unit (CPU) that controls the operation of each section of the information processing device 3 , and operates according to an operation program stored in the storage section 32 .

The control point information acquisition unit 36 and the feature amount information acquisition unit 37 are configured by GPUs (Graphics Processing Units), for example. A GPU is an arithmetic unit (processor) specialized for real-time image processing. In this way, by configuring the control point information acquisition unit 36 and the like with a GPU, a neural network capable of machine learning such as a convolutional neural network (CNN) is constructed on the GPU, and this neural network is controlled. It can function as the point information acquisition unit 36 and the feature amount information acquisition unit 37 .

The control point information acquisition unit 36 acquires information on control points (control point information) from an image acquired by the camera 2, that is, an image of a person. Here, a control point refers to a point that defines the skeleton of a person in an image. Therefore, the control points include joint points (movable joints connecting bones) of a person, the center point of the skull, points at the tip of the bones of the fingers and toes, and the like. In addition, the positions of the eyes, nose, and ears of the person in the image can also be control points because they define the skeleton (size and shape of the skull) of the person.

The control point information includes position information indicating the position (two-dimensional position) of the control point in the image and the detection likelihood (first detection likelihood) of the control point. The first detection likelihood is a value (index) indicating the probability of the detected control point. By preliminarily learning the CNN that constitutes the control point information acquisition unit 36, it is possible to output the first detection likelihood of each control point of the person included in the image from the input image.

The feature amount information acquisition unit 37 extracts a person rectangle surrounding the person from the image of the person, and acquires feature amount information including the feature amount related to the attribute of the person. Here, at least one of the person's age, age group, sex, clothing, wearable item, and portable item can be considered as the attribute of the person. When the attribute of a person is age, specific ages such as 10 years old, 20 years old, and 30 years old can be considered as feature amounts related to the attribute of a person. Also, when the attribute of a person is an age group, rough age divisions such as teens, twenties, and thirties can be considered as feature amounts. Note that the age group may be classified into children/adults/elderly.

Also, when the attribute of a person is gender, the feature amount can be male/female. When it is difficult to distinguish between genders, male/female/other (indistinguishable) classifications may be used. When the attribute of a person is clothes, the feature amount can be considered as color, short sleeve/long sleeve, shirt, sweatshirt, trousers, skirt, and the like. When the attribute of a person is an object to be worn, the feature amount can be a necklace, a bracelet, glasses, a hat, or the like. When the attribute of a person is a portable item, the feature amount can be a bag (hand type, carry type), an umbrella, a pet, and the like.

In addition, the feature amount information includes the feature amount related to the attributes of the person as well as the detection likelihood (second detection likelihood) of the feature amount. The second detection likelihood is a value (index) indicating the probability of the detected feature quantity. By pre-learning the CNN constituting the feature amount information acquisition unit 37, it is possible to output the detection likelihood of the feature amount related to the attribute of the person included in the image from the input image.

The certainty factor determination unit 38 determines the certainty factor of the feature amount based on the control point information acquired by the control point information acquisition unit 36 and the feature amount information acquired by the feature amount information acquisition unit 37 . The certainty factor is a value (index) indicating the reliability of the feature quantity. Such a certainty determination unit 38 is configured by, for example, the same CPU as or a separate CPU from the control unit 39 . Information on the degree of certainty determined by the degree of certainty determination unit 38 is output to the person search device 4 and stored in the database 42 (see FIG. 3). The details of the process of determining the certainty by the certainty determining unit 38 will be described later.

(person search device)
FIG. 3 is a block diagram showing a schematic configuration of the person search device 4. As shown in FIG. The person search device 4 includes a program storage unit 41 , a database 42 , a person search unit 43 , a search result output unit 44 and a control unit 45 . The control unit 45 is composed of a CPU that controls the operation of each unit of the person search device 4 .

The program storage unit 41 stores an operation program for the control unit 45, and is composed of, for example, a hard disk. The database 42 is a memory that stores the reliability of the feature amount determined by the information processing device 3 for each feature amount related to each attribute, and is configured by the same hard disk as the program storage unit 41, for example. Further, the database 42 stores the confidence factor together with shooting information about the shooting location and shooting date and time of the camera 2 (see FIG. 11).

In response to a search request (receipt of a search query) from the outside (for example, a terminal device on the user side), the person search unit 43 has a feature amount that matches the search condition based on the degree of certainty stored in the database 42. Search for people. Such a person search unit 43 is composed of a CPU that is the same as or different from that of the control unit 45 .

The search result output unit 44 outputs the result searched by the person search unit 43 . In particular, the search result output unit 44 outputs, as a search result, photographing information of the camera 2 that has photographed a person having a feature amount that matches the search condition. Such a search result output section 44 is configured to have a display section 44a and a communication section 44b. The display unit 44a is composed of a display (for example, a liquid crystal display) that displays the search results. The communication unit 44b is a communication interface for transmitting the information of the search result to the outside (for example, a terminal device on the user side).

The person search device 4 may be configured by a personal computer having an input unit and the like. Then, the user may input search conditions by operating the input unit. Further, the database 42 that stores the certainty of the feature amount may be provided independently outside the person search device 4 and connected to the information processing device 3 and the person search device 4 so as to be communicable. Further, the information processing device 3 described above may have the function of each part of the person searching device 4 (the information processing device 3 may be integrated with the person searching device 4).

[Person search method]
Next, a person search method according to this embodiment will be described. FIG. 4 is a flow chart showing the flow of processing by the person search method of this embodiment. For example, when a camera 2 installed in a store takes a picture of a person in the store and acquires images for a plurality of frames (S1), the image data of each frame is input to the information processing device 3, and the information processing device 3 performs an information processing step (S2).

That is, the control point information acquisition unit 36 acquires control point information including the position information of the control points and the first detection likelihood of the control points from the image of the person (S21; control point information acquisition step). In S21, when a plurality of persons are included in the image, the above control point information is obtained for each person. Note that the first detection likelihood of the control point depends on how the person appears in the image (whether the image includes the whole body or the image has a part of the whole body cut off) and how the person overlaps with others (mutual positional relationship). It fluctuates for each control point (for example, see the left diagram of FIG. 9 and the left diagram of FIG. 10).

Subsequently, the feature amount information acquisition unit 37 acquires feature amount information including the feature amount related to the attribute of the person and the second detection likelihood of the feature amount from the image of the person (S22; feature amount information acquisition step ). In S22, the feature quantity information acquisition unit 37 acquires a feature quantity and a second detection likelihood for at least one of a plurality of attributes of a person (see, for example, the middle diagram of FIG. 9 and the middle diagram of FIG. 10). .

Then, the certainty determining unit 38 determines the certainty of the feature amount based on the control point information acquired in S21 and the feature amount information acquired in S22 (S23; certainty determining step). Note that when the feature amount information is acquired for each of the plurality of attributes in S22, the certainty of the feature amount is determined for each of the plurality of attributes in S23 (for example, the right figure in FIG. 9, the (see right figure).

When the certainty factor of the feature amount is determined by the certainty factor determination unit 38, the information of the above-mentioned certainty factor is output to the person search device 4 and stored in the database 42 (S3; storage step). Then, when the person search device 4 receives a search request (search query) from the outside (S4), the person search unit 43 responds to the search request based on the certainty stored in the database 42, and satisfies the search conditions. A person having matching feature values is searched (S5; person search step). For example, the person search unit 43 searches for a person having the feature amount by extracting the feature amount that matches the search condition from the database 42 in descending order of certainty.

After that, the search result output unit 44 outputs the search result in S5 (S6; search result output step). For example, the display unit 44a of the search result output unit 44 displays the search results, and the communication unit 44b outputs information indicating the search results to the terminal device on the user side. Note that the above search result is information on a person having a feature amount that matches the search condition. Substitute information.

When the person search device 4 receives the next search query from the outside (Yes in S7), the process returns to S5 to further perform person search by the person search unit 43, and if the next search query has not been received ( No in S7), the series of processes is terminated.

[Basic concept of confidence determination]
Next, the basic idea of the certainty factor determination by the certainty factor determination unit 38 of the information processing device 3 described above will be described. FIG. 5 shows an example of an image in which an unspecified number of people are reflected. In this image, in the person rectangle R1, the full body image of the target person is shown without being blocked. This case is the most ideal state because there is no influence of other persons or objects (signboards, etc.) in acquiring the feature amount related to the attribute of the person. In this case, since the reliability of the feature amount acquired for the person is high, the person search can be performed with high accuracy based on the feature amount. For example, for a search query that specifies a feature value of "red shirt", a person in the person rectangle R1 wearing a "red shirt" can be found with high accuracy.

However, in reality, due to the existence of a plurality of persons in the image, other persons are likely to appear in front of the target person (indicated by diagonal lines) as in the person rectangle R2. In this case, since there is a high possibility that the feature amount of the other person in front is extracted as the feature amount of the target person, it is difficult to perform a person search with high accuracy based on the feature amount. Since a person is basically a moving object, it is not always superimposed. often varies from frame to frame.

Also, for example, in an image of a clothing store, if a "mannequin" or "clothes on display" in the store are captured, there is a high possibility that these will be erroneously detected as the target person or feature quantity. Furthermore, if the image includes "the person who is the subject of the advertisement," this person may be erroneously detected as the target person. In this way, in an image, not only the presence of a real person but also objects and advertising media that are close to the person can be factors that hinder highly accurate person retrieval.

Therefore, in the present embodiment, the degree of certainty of the attribute feature amount of each person is determined according to the positional relationship of an unspecified number of persons in the image. In particular, for frames in which it is detected that a person is superimposed, the reliability of the feature values of the attribute grasped from the area where the influence of the other person is reflected in the image (the area where the other person overlaps) is lowered. Let Note that the positional relationship among an unspecified number of persons can be detected, for example, by the control point information acquisition unit 36 based on the positional information of the control points of each person.

6A to 6C schematically show the positional relationship of two people in an image as an example. In these drawings, white circles indicate the positions of control points (the same applies to other drawings). As shown in FIG. 6A, two persons are almost overlapping each other, and if the control point position information is obtained only for one person (the person in the front), the certainty determination unit 38 determines that the person in the front is , the reliability of the detected feature amount is set to a high value, and the above feature amount is used for person search.

Further, as shown in FIG. 6B, when two persons partially overlap each other, the certainty factor determination unit 38 sets the certainty factor of the detected feature quantity to a high value for the person in the foreground. quantity is adopted for person search. On the other hand, the certainty factor determination unit 38 sets the certainty factor of the feature amount related to the attribute grasped from the area to a low value for the area of the person in the background that overlaps the person in the foreground. is not used for person search, and for other areas that do not overlap with the person in the foreground, the confidence of the feature amount related to the attribute grasped from that area is set to a high value, and the above feature amount is used for person search adopted for use.

Furthermore, as shown in FIG. 6C, when the two persons do not overlap each other, the certainty determining unit 38 sets the certainty of the feature amount detected for each person to a high value, and for each person, is used for person search. However, for the person in the lower left of the image, the lower half of the body is not shown, and the first detection likelihood is expected to be low. lower the confidence of the feature with respect to the corresponding attribute).

Here, FIG. 7 shows the relationship between the attribute of a person, the feature amount related to the attribute, and the position of the control point associated with the attribute. For example, attributes such as gender, age, age group, upper body clothing, part of the wearer (necklace), part of the carrying (bag (hand type), pet) are the characteristics of the region of the upper body of the person in the image. is detected by extracting , it can be said that the control point positioned in the region of the upper body corresponds (is related to) the attribute such as gender described above. In other words, the region of the upper body of a person in an image has a great influence on determination of attributes such as gender (detection of feature amounts relating to attributes). Therefore, when the upper body region of a person is hidden in the image by being superimposed on another person, it becomes difficult to detect the feature amount (for example, male/female) related to the attribute with high accuracy.

Therefore, in the present embodiment, the certainty determination unit 38 refers to FIG. Set the feature confidence for the corresponding attribute to a low value. On the other hand, for a region of a person in the image that does not overlap (hide) another person, referring to FIG. set to a value. It should be noted that how much the degree of certainty is changed can be determined based on the first detection likelihood of the control point and the second detection likelihood of the feature quantity. If the positional information of the control point is uncertain, the certainty factor of the feature amount of the attribute corresponding to the control point may be set to zero.

[Details of confidence determination]
Next, the details of determination of the certainty by the certainty factor determination unit 38 will be described. FIG. 8 shows an image of each frame taken by camera 2 (ID: 1F South B) installed at a predetermined location in the store, and the feature values related to the attribute of the person extracted (acquired) from the image. is shown as an example. Note that the ID (identification information) of the camera 2 is information indicating the installation location (shooting location) of the camera 2 . Also, n in the figure indicates the number of frames arranged in time series. In FIG. 8, two people are shown in the image, and the position of each person changes over time. , and the person in the foreground is defined as “human=002” to distinguish these persons from each other.

As shown in FIG. 8, in this embodiment, extraction (acquisition) of feature amounts from an image is performed independently for each frame. Although person identification processing (processing for determining whether or not the person in the image is the same person between frames) is not performed here, such processing may be performed. Note that the likelihood of the feature amount detected by the feature amount acquiring unit 37 among the feature amounts related to each attribute (Gender, Age, Tops (upper body clothing), Bottoms (lower body clothing)) shown in FIG. A feature amount whose degree (second detection likelihood) is equal to or less than a predetermined value is indicated by "Unknown".

FIG. 9 shows the control point information acquired by the control point information acquisition unit 36 and the feature amount information acquired by the feature amount information acquisition unit 37 for a person with human=001 in the 0th frame (n=0). , schematically shows a process in which the certainty factor determination unit 38 determines the certainty factor based on . Here, the control point information acquisition unit 36 detects the position of each control point (head, etc.) with a first detection likelihood as shown in the left diagram of FIG. Suppose that a feature amount related to a person's attribute is detected with a second detection likelihood as shown in the middle diagram of .

Since the 0th frame image does not include the head, face, upper body, and arms of the person with human=001, these control points (for example, the head, right wrist, right shoulder, The first detection likelihood of the chest, left shoulder, left wrist) is lower than the threshold Th (for example, first detection likelihood=0). Also, for the person with human=001, it is difficult to accurately recognize the feature amount related to the attributes (e.g. gender, age group) having the correspondence shown in FIG. 7 from the head etc. not included in the image. , the reliability of the detected features is considered to be low.

Therefore, in this case, the certainty determination unit 38 determines the certainty of the feature amount of the attribute associated with the control point whose first detection likelihood is lower than the threshold value Th, of the attribute acquired by the feature amount information acquisition unit 37. It is set to a value lower than the second detection likelihood of the feature quantity. In the right diagram of FIG. 9, the certainty of the feature values related to gender, age group, upper body clothing, wearing (with or without hat, with or without glasses), and carrying item (with or without bag (hand type)) is shown in the middle diagram of FIG. It can be seen that the value is set lower than the second detection likelihood of the feature amount. In addition, regarding attributes such as gender, age group, and belongings (bag presence/absence), in some cases, information on the lower body of the person may be obtained. Do not lower the degree completely to zero, but leave a certain value (see the right figure in FIG. 9).

On the other hand, for the person with human = 001, the first detection likelihood of the control points of the lower body (waist, right knee, left knee, etc.) is high. can do. Therefore, the certainty factor determination unit 38 sets the certainty factor of the feature amount related to lower-body clothing to a relatively high value (see the right diagram of FIG. 9). For example, the certainty factor determination unit 38 determines the certainty factor of the feature amount to be a high value equivalent to the second detection likelihood of the feature amount detected by the feature amount information acquisition unit 37 .

The certainty of the feature determined by the certainty determination unit 38 in this way is registered in the database 42 of the person search device 4 together with the shooting information of the camera 2 (including the shooting date and time and the shooting location) for each feature ( memory).

FIG. 10 shows the control point information acquired by the control point information acquisition unit 36 and the feature amount information acquisition unit 37 for the person with human=001 in the second frame (n=2) of FIG. 4 schematically shows the process of determining the certainty factor by the certainty factor determination unit 38 based on the obtained feature amount information. Here, the control point information acquisition unit 36 detects the position of each control point with the first detection likelihood as shown in the left diagram of FIG. Suppose that a feature amount related to a person's attribute is detected with a second detection likelihood as shown.

In the image of the second frame, a part of the person of human=001 is hidden by the person of human=002. The control point information acquisition unit 36 determines that the person of human=001 and the person of human=002 overlap with each person's head position and torso balance from the position information of the detected control points of each person. can be detected. Therefore, regarding the person with human=001, hidden information by the other person (person with human=002) is reflected in the first detection likelihood of the control point. That is, the first detection likelihood of each control point indicating the chest, right wrist, left wrist, and waist of the person of human=001 is a low value considering the overlap with the person of human=002.

On the other hand, as shown in the middle diagram of FIG. 10, for the person of human=001, the second detection likelihood of the attribute-related feature amount is high for each attribute. However, since the feature amount information acquisition unit 37 extracts the feature amount from the image without considering the overlapping of the persons, the attribute ( For example, there is a high possibility that the feature amount of the person with human=002 is reflected in the feature amount related to the upper body clothing, the lower body clothing, and the carry-on item (bag). As a result, it is considered that the second detection likelihood of the feature quantity is high for the attribute.

In addition, when the person of human=001 and the person of human=002 overlap in the image, the feature amount extracted from the image is the feature amount of the person of human=001 or the feature of the person of human=002. It is generally difficult to distinguish whether it is a quantity or not. For example, if the person with human=001 wears black clothes, it is difficult to distinguish this black clothes from the black head or black hat of the person with human=002. Also, when a person with human=001 wears a T-shirt printed with a person image, it is difficult to distinguish this T-shirt (upper body clothing) from the head of the person with human=002.

Therefore, based on the control point information, when the control point information acquisition unit 36 detects that the person with human=001 overlaps with the person with human=002 in the image, the certainty determination unit 38 determines , the certainty of the feature amount of the attribute (see FIG. 7) related to the control point of the area of the person of human=001 hidden by the person of human=002 is obtained by the attribute information acquisition unit 37. It is set to a value lower than the second detection likelihood of the feature amount. In the right diagram of FIG. 10, the upper and lower bodies of the human=001 person are partly hidden by the human=002 person, so the attributes associated with the hidden upper and lower body control points (for example, upper body clothing, lower body clothing , and (bag)) is set to a value lower than the second detection likelihood of the feature quantity of the attribute acquired by the feature quantity information acquiring unit 37. FIG. The certainty factor of the set feature amount is stored in the database 42 together with the photographing information of the camera 2 as needed.

FIG. 11 shows an example of information stored in the database 42. As shown in FIG. The database 42 stores the photographing information of the camera 2 as well as the certainty factor for each feature amount relating to the attributes of a person (eg, gender, age group, and upper body clothing are shown in FIG. 11). For example, when the shooting date and time is "2019/3/31 15:32", the face and upper body of the person with human=001 are not included in the image, so the confidence of the feature amount related to gender is shown in the middle diagram of FIG. is set to a value lower than the second detection likelihood of the feature amount.

[Regarding determination of certainty in cases other than superimposition with a person]
FIG. 12 shows an example of an image captured by the camera 2 inside the store. In the image captured inside the store, there is a high possibility that the person is hidden by the product shelf 101 and the article 102 inside the store, as in the person rectangle R3. In this case, the certainty determination unit 38 sets the certainty factor of the feature amount related to the attribute corresponding to the position of the control point in the region to a high value only for the region of the person that is not hidden by the product shelf 101 or the like. It is desirable to set a low value for the certainty factor of the feature amount regarding the position of the control point in the hidden area such as 101 and the corresponding attribute. Whether or not the person is hidden by the product shelf 101 or the like can be detected by the control point information acquisition unit 36 based on the first detection likelihood of the control point of the person. Of course, it is also possible to select an image of a scene in which a person is not hidden (for example, a scene in which a person appears in an aisle with no product shelf 101 or the like) and acquire the feature amount of the attribute of the person. . Objects that make a person hide include not only the product shelf 101 and the article 102 described above, but also advertisements, signboards (for example, installed at store entrances), hanging signboards (advertisements hung in the store, etc.), and the like. Conceivable.

Also, FIGS. 13A and 13B respectively show the state in which the person is facing forward and backward in the image. In these figures, L indicates "left" and R indicates "right". If left and right (for example, right shoulder and left shoulder) can be specified from the position of each control point of the person, the orientation (forward/backward) of the person in the image can be estimated. In general, the gender and age as attributes of a person can be accurately recognized using a face image, and since the face image cannot be obtained when the person is facing backward, it is difficult to estimate the person's sex and age. becomes difficult. Therefore, when the control point information acquiring unit 36 detects that the person is facing backward from the image based on the control point information, the certainty determining unit 38 sets the certainty of the feature amount related to gender and age to a low value. You may Note that it is also possible to detect a face area from an image, determine the likelihood of a face, and determine whether or not the person is facing backward based on the determination result.

[Summary of processing for determining certainty]
In this embodiment, considering the various cases described above, the confidence factor is determined as follows. That is, the certainty factor determination step shown in S23 of FIG. 4 can be summarized as follows. FIG. 14 is a flow chart showing the details of the certainty determination process.

First, when the control point information acquisition unit 36 detects that there is a possibility that another person is superimposed in front of the target person based on the control point information (the position of the control point, the first detection likelihood) (Yes in S101), the certainty determination unit 38 determines the certainty of the feature amount of the attribute associated with the overlapping area based on the attribute acquired by the feature amount information acquisition unit 37, as described above. A value lower than the second detection likelihood of the feature amount is set (S102), and the process proceeds to S103. Note that the target person refers to a person for whom the certainty factor of the feature amount is determined.

There is no possibility that another person is superimposed in front of the target person (No in S101), and the control point of the target person detected by the control point information acquisition unit 36 has the first detection likelihood lower than the threshold Th. If the control point is included (Yes in S103), the certainty factor determination unit 38 determines the certainty factor of the feature amount of the attribute associated with the control point with the low first detection likelihood through feature amount information acquisition. A value lower than the second detection likelihood of the feature amount related to the attribute acquired by the unit 37 is set (S104), and the process proceeds to S105.

The control points of the target person do not include control points with the first detection likelihood lower than the threshold Th (No in S103), and the control point information acquisition unit 36 detects that the target person is facing backward. If detected (Yes in S105), the certainty determination unit 38 causes the feature amount information acquisition unit 37 to acquire the certainty of the feature amount of the attributes (e.g., gender, age) related to the face of the target person. is set to a value lower than the second detection likelihood of the feature amount of the above attribute (S106), and the process proceeds to S107.

When the target person is not facing back (No at S105) and the control point information acquisition unit 36 detects that there is an object in front of the target person's control point (Yes at S107), the certainty The determination unit 38 sets the certainty of the feature amount of the attribute associated with the control point superimposed on the object to be lower than the second detection likelihood of the feature amount of the attribute acquired by the feature amount information acquisition unit 37. value (S108), and a series of processes relating to determination of certainty is terminated. Note that even if there is no object in front of the target person's control point (No in S105), the series of processes is terminated.

〔effect〕
As described above, in the present embodiment, the certainty factor determination unit 38 determines the certainty factor of the feature amount related to the attribute of the person based on the control point information and the feature amount information acquired from the image of the person. . As a result, for example, only a part of the target person is shown in the image due to an unspecified number of people, the target person overlaps with other people or other objects in the image, or the target person's face is not visible. Even if the person is not viewed from the front, it is possible to obtain the information (certainty factor) necessary for highly accurate person retrieval from such an image. Therefore, as an image used for person search, it is possible to use an image captured by a surveillance camera in which an unspecified number of people may be captured. It is possible to avoid being restricted to a specific image such as an image in which a face is reflected from the front or the like.

Further, by determining the degree of certainty of the feature amount, it is possible to perform a highly accurate person search based on the degree of certainty. In other words, as in the present embodiment, by storing the certainty of the feature amount in the database 42 (see FIG. 11), the person search unit 43 can match the search conditions with respect to the person search query transmitted from the outside. It is possible to retrieve the feature amount in descending order of certainty, and to search for the corresponding person with high accuracy. Therefore, it is possible to realize the information processing device 3 and the information processing method suitable for the person search system 1 that performs highly accurate person searches.

Further, the control point information acquired by the control point information acquisition unit 36 includes the first detection likelihood of the control point in addition to the position information of the control point, and the feature amount information acquired by the feature amount information acquisition unit 37 is In addition to the feature amount related to the attribute, the second detection likelihood of the feature amount is included. In this case, since the certainty factor determination unit 38 can determine the certainty factor of the feature amount in consideration of the first detection likelihood of the control point and the second detection likelihood of the feature amount, for example, the first detection likelihood Confidence is set to a value equivalent to the second detection likelihood of the feature amount for the attribute associated with the control point with a high degree of probability, and the attribute associated with the control point with the low first detection likelihood is set to the feature amount. A confidence factor can be set to a value lower than the second detection likelihood. By this, by giving a difference in certainty between the feature amount of the attribute associated with the control point with the high first detection likelihood and the feature amount of the attribute associated with the control point with the low first detection likelihood, It becomes possible to perform person search based on certainty with higher accuracy.

Further, when the control point information acquisition unit detects that the target person overlaps with another person, the certainty determination unit 38 determines the control point of the area of the target person hidden by the other person in the image. is set to a value lower than the second detection likelihood of the feature amount of the attribute acquired by the feature amount information acquiring unit 37 (FIGS. 8, 10, and 14). (See S101 and S102). A region of a target person hidden by another person in an image is greatly affected by the attributes of the other person, so an erroneous feature value is likely to be acquired as the feature value related to the attribute of the target person. Therefore, by setting the reliability of the feature amount of the attribute associated with the control point of the area of the target person to a low value, for example, a value with a low degree of certainty is registered in the database 42, and when searching for a person, the feature amount It is possible to prevent a person with a finger from being caught, thereby making it possible to reliably perform a highly accurate search.

In addition, the certainty factor determination unit 38 determines the certainty factor of the feature amount of the attribute associated with the control point whose first detection likelihood is lower than the threshold value Th as compared to the feature amount of the attribute acquired by the feature amount information acquisition unit 37. A value lower than the second detection likelihood is set (see S103 and S104 in FIGS. 8, 9 and 14). For example, for a person whose head is not shown in the image, the first detection likelihood of the control point indicating the head is detected as a value lower than the threshold Th. Therefore, the certainty determining unit 38 registers low certainty values in the database 42, for example, by setting the certainty values of the feature amounts of the attributes (e.g., gender and age) related to the person's head to low values. Therefore, when searching for a person, it is possible to prevent a person having the above feature amount (a person whose feature amount is unreliable in terms of gender, age, etc.) from being caught, and it is possible to reliably perform a highly accurate search. .

Further, when the control point information acquiring unit 36 detects that the person is facing backward in the image, the certainty determining unit 38 determines the certainty of the feature amount of the attribute related to the person's face as follows: A value lower than the second detection likelihood of the feature quantity of the attribute acquired by the feature quantity information acquisition unit 37 is set (see S105 and S106 in FIGS. 13B and 14). For example, for a person facing the back in an image, the feature values (e.g., male/female, child/adult/elderly, glasses) of attributes related to the face of the person (e.g., gender, age group, wearing equipment) are used. difficult to obtain from the image. In this case, the certainty determining unit 38 sets the certainty of the feature amount of the attribute related to the person's face to a low value, for example, registers the low certainty value in the database 42, and performs the person search. Sometimes, it is possible to prevent a person having the above feature amount (a person whose feature amount is unreliable in terms of gender, etc.) from being caught, and it is possible to reliably perform a highly accurate person search.

Further, when the control point information acquisition unit 36 detects that the person overlaps with an object other than the person, the certainty determination unit 38 determines whether the person is hidden by the object in the image. The certainty factor of the feature amount of the attribute to be set to a value lower than the second detection likelihood of the feature amount of the above attribute acquired by the feature amount information acquisition unit 37 (see S107 and S108 in FIGS. 12 and 14). . A region of a person hidden by an object in an image is greatly affected by the object (for example, the color of the product shelf 101 and the article 102 in FIG. 12). easily acquired (for example, the color of the article 102 is easily detected as the color of the clothing). Therefore, by setting the reliability of the feature amount of the attribute associated with the control point of the region of the person to a low value, for example, the value of the low reliability is registered in the database 42, and the feature amount is used when searching for a person. It is possible to prevent the holding person (for example, a wrong person whose clothes are the same color as the article 102) from being caught, and it is possible to reliably perform a highly accurate person search.

In addition, in this embodiment, the attributes of a person are at least one of age, age group, sex, clothing, wear, and belongings of the person. In the configuration in which the feature amount information acquisition unit 37 acquires feature amount information related to an attribute such as a person's age, and the certainty determination unit 38 determines the certainty of the feature amount related to the attribute, the effects of the present embodiment described above are obtained. be able to.

The person search system 1 of this embodiment also includes the information processing device 3 of this embodiment described above, a database 42 , a person search unit 43 , and a search result output unit 44 . According to this person search system 1, the user can grasp the presence or absence of a person who matches the search conditions based on the search results output from the search result output unit 44. FIG.

Also, the person system 1 of the present embodiment includes the camera 2, and the database 42 stores the certainty of the feature amount together with the photographing information regarding the photographing location and the photographing date and time of the camera 2. FIG. Then, the search result output unit 44 outputs, as a search result, photographing information of the camera 2 that photographed the person having the feature amount that matches the search condition (see S6 in FIG. 4). Thereby, the user can grasp the place (the same as the shooting place of the camera 2) and the time at which the person matching the search condition existed based on the output shooting information.

The search result output unit 44 also includes a display unit 44a. In this case, the user can easily grasp the information (existence place and time) of the person who matches the search condition by looking at the search result displayed on the display unit 44a.

〔others〕
15A to 15D schematically show examples of images in which a person and an object overlap. In these drawings, regions of persons or objects are indicated by rectangular frames. Some systems also have a function of detecting objects other than people. In this case, areas where the person overlaps with the object and areas where the object does not overlap are detected, and the degree of certainty of the feature value related to the attribute grasped from each area is determined, and the relationship between the person and the object is detected (estimated). You may For example, a person is in a car (see FIG. 15A), a car is running next to a walking person (see FIG. 15B), there is a plant in front of the person (see FIG. 15C), and the person is an animal. (see FIG. 15D), etc. may be detected. In addition, it is also possible to detect that a person is pushing a cart, that a person is on an escalator, and the like.

In addition, commercial facilities and public facilities often have multiple cameras installed, and among the multiple camera images taken at the same time, depending on the camera, the image may not be ideal (a person may be blocked by an object). In some cases, an image without the object is obtained, and an image in which the person is blocked by an object is obtained. In the latter case, the certainty factor of the feature amount may be determined and stored in the database only for the frames that are not affected by the shielding of the object in each camera image. Further, when a search query is received from the user side and a target person is searched, the person can be searched across multiple camera images (across multiple camera images) based on the certainty factor. Often, a person may be searched for by targeting images of remaining frames obtained by thinning frames in which the person is blocked by an object in time series.

[Program and recording medium]
The information processing apparatus 3 described in this embodiment can be configured by, for example, a computer (PC) in which a predetermined program (application software) is installed. A computer (for example, the control unit 39 as a CPU) reads and executes the above program, so that each unit of the information processing device 3 can be operated to execute each process (each process) described above. Such a program is acquired by external download via a network, for example, and stored in the storage unit 32 . Further, the program is recorded in a computer-readable recording medium R such as a CD-ROM (Compact Disk-Read Only Memory), and the computer reads the program from the recording medium R and stores it in the storage unit 32. may be

<Supplement>
The information processing apparatus, the person search system, the information processing method, and the person search method of this embodiment described above can also be expressed as follows.

1. a control point information acquisition unit that acquires control point information including position information of control points that define the skeleton of the person from an image of the person;
a feature amount information acquisition unit that acquires feature amount information including a feature amount related to attributes of the person from the image;
An information processing apparatus, comprising: a certainty factor determination unit that determines a certainty factor indicating reliability of the feature amount based on the control point information and the feature amount information.

2. The control point information includes, in addition to the position information of the control point, a first detection likelihood indicating the detection likelihood of the control point,
2. The information processing apparatus according to 1, wherein the feature amount information includes a second detection likelihood indicating a detection likelihood of the feature amount in addition to the feature amount.

3. The control point information acquisition unit detects whether or not the plurality of persons overlap in the image based on the control point information,
When the control point information acquisition unit detects that the target person for whom the confidence level is to be determined overlaps with another person, the certainty factor determination unit is configured to A value lower than the second detection likelihood of the feature amount of the attribute acquired by the feature amount information acquisition unit, as a certainty factor of the feature amount of the attribute associated with the control point of the target person's hidden area. 3. The information processing apparatus according to the above 2, characterized in that it is set to .

4. The certainty factor determining unit determines the certainty factor of the feature amount of the attribute associated with the control point for which the first detection likelihood is lower than a threshold value of the feature amount of the attribute acquired by the feature amount information acquiring unit. 4. The information processing apparatus according to 2 or 3 above, wherein a value lower than the second detection likelihood is set.

5. The control point information acquisition unit detects whether or not the person is facing backward in the image based on the control point information,
The certainty determination unit determines the certainty of a feature amount of an attribute related to the face of the person when the control point information acquisition unit detects that the person is facing backward in the image. 5. The information processing apparatus according to any one of 2 to 4, wherein the feature amount of the attribute acquired by the feature amount information acquisition unit is set to a value lower than the second detection likelihood.

6. The control point information acquisition unit detects whether or not the person overlaps with an object other than the person in the image based on the control point information,
When the control point information acquisition unit detects that the person overlaps with the object, the certainty determination unit associates the control point with the control point of the area of the person hidden by the object in the image. 5, wherein the reliability of the feature amount of the attribute to be set to a value lower than the second detection likelihood of the feature amount of the attribute acquired by the feature amount information acquisition unit. The information processing device according to any one of the above.

7. 7. The information processing apparatus according to any one of 1 to 6, wherein the attribute of the person is at least one of the person's age, age group, sex, clothing, wear, and carrying item.

8. 8. The information processing device according to any one of 1 to 7;
a database that stores the certainty factor of the feature value determined by the information processing device;
a person search unit that searches for a person having a feature amount that matches a search condition, based on the certainty stored in the database, in response to a search request from the outside;
A person search system, comprising: a search result output unit that outputs a search result obtained by the person search unit.

9. further comprising a camera that acquires the image and provides data of the image to the information processing device;
The database stores the certainty factor of the feature amount together with shooting information related to the shooting location and shooting date and time of the camera,
9. The person search system according to 8 above, wherein the search result output unit outputs, as the search result, the photographing information of the camera that has photographed the person having a feature amount that matches the search condition.

10. 10. The person search system according to 8 or 9 above, wherein the search result output unit includes a display unit that displays the search result.

11. a control point information acquisition step of acquiring control point information including position information of control points that define the skeleton of the person from an image of the person;
A feature amount information acquiring step of acquiring feature amount information including a feature amount related to attributes of the person from the image;
An information processing method, comprising: determining a certainty factor indicating reliability of the feature amount based on the control point information and the feature amount information.

12. The control point information includes, in addition to the position information of the control point, a first detection likelihood indicating the detection likelihood of the control point,
12. The information processing method according to 11 above, wherein the feature amount information includes, in addition to the feature amount, a second detection likelihood indicating a detection likelihood of the feature amount.

13. In the control point information acquisition step, based on the control point information, it is detected whether or not the plurality of persons overlap in the image;
In the certainty factor determination step, when it is detected in the control point information acquiring step that the target person whose certainty factor is to be determined overlaps with another person, A value lower than the second detection likelihood of the feature amount of the attribute acquired in the feature amount information acquisition step, as a certainty factor of the feature amount of the attribute associated with the control point of the hidden area of the target person. 13. The information processing method according to 12 above, characterized in that the setting is set to .

14. In the certainty factor determination step, the certainty factor of the feature amount of the attribute associated with the control point for which the first detection likelihood is lower than a threshold is determined as the feature amount of the attribute acquired in the feature amount information acquiring step. 14. The information processing method according to 12 or 13 above, wherein the second detection likelihood is set to a lower value.

15. In the control point information obtaining step, based on the control point information, it is detected whether or not the person is facing backward in the image;
In the certainty factor determination step, when the control point information acquisition step detects that the person is facing backward in the image, the certainty factor of the feature amount of the attribute related to the face of the person is determined. 15. The information processing method according to any one of 12 to 14, wherein the second detection likelihood of the feature amount of the attribute acquired in the feature amount information acquiring step is set to a value lower than the second detection likelihood.

16. In the control point information acquiring step, based on the control point information, it is detected whether or not the person overlaps with an object other than the person in the image;
In the certainty factor determination step, when it is detected in the control point information acquisition step that the person overlaps with the object, a control point of an area of the person hidden by the object in the image is associated with a control point. 15, characterized in that the certainty of the feature amount of the attribute to be set to a value lower than the second detection likelihood of the feature amount of the attribute acquired in the feature amount information acquisition step. The information processing method according to any one of the above.

17. 17. The information processing method according to any one of 11 to 16, wherein the attribute of the person is at least one of the person's age, age group, sex, clothing, wear, and carrying item.

18. 18. A person search method using the information processing method according to any one of 11 to 17,
a storage step of storing in a database the certainty factor of the feature value determined by the information processing method;
a person search step of searching for a person having a feature amount that matches a search condition in response to a search request from the outside, based on the certainty stored in the database;
and a search result output step of outputting search results obtained by the person search step.

19. 19. The above-described item 18, wherein in the search result outputting step, as the search result, photographing information regarding a photographing location and a photographing date and time of a camera that photographed the person having the feature quantity that matches the search condition is output. People search method.

20. 20. The person search method according to 18 or 19, wherein, in the search result output step, the search result is displayed on a display unit.

Moreover, the program and recording medium of this embodiment can also be expressed as follows.

21. 18. A program for causing a computer to execute the information processing method according to any one of 11 to 17 above.

22. 22. A computer-readable recording medium recording the program according to 21 above.

Although the embodiments of the present invention have been described above, the scope of the present invention is not limited thereto, and can be implemented by being expanded or modified without departing from the gist of the invention.

INDUSTRIAL APPLICABILITY The present invention can be used for a system that searches for a person using information registered in a database.

1 person search system 2 camera 3 information processing device 36 control point information acquisition unit 37 feature amount information acquisition unit 38 certainty factor determination unit 42 database 43 person search unit 44 search result output unit 44a display unit

Claims (18)

a control point information acquisition unit that acquires control point information including position information of control points that define the skeleton of the person from an image of the person;
a feature amount information acquisition unit that acquires feature amount information including a feature amount related to attributes of the person from the image;
a certainty factor determination unit that determines a certainty factor indicating reliability of the feature amount based on the control point information and the feature amount information ,
The control point information includes, in addition to the position information of the control point, a first detection likelihood indicating the detection likelihood of the control point,
The feature amount information includes, in addition to the feature amount, a second detection likelihood indicating the detection likelihood of the feature amount,
The control point information acquisition unit detects whether or not the plurality of persons overlap in the image based on the control point information,
When the control point information acquisition unit detects that the target person for whom the confidence level is to be determined overlaps with another person, the certainty factor determination unit is configured to A value lower than the second detection likelihood of the feature amount of the attribute acquired by the feature amount information acquisition unit, as a certainty factor of the feature amount of the attribute associated with the control point of the target person's hidden area. An information processing device characterized in that it is set to a control point information acquisition unit that acquires control point information including position information of control points that define the skeleton of the person from an image of the person;
a feature amount information acquisition unit that acquires feature amount information including a feature amount related to attributes of the person from the image;
a certainty factor determination unit that determines a certainty factor indicating reliability of the feature amount based on the control point information and the feature amount information,
The control point information includes, in addition to the position information of the control point, a first detection likelihood indicating the detection likelihood of the control point,
The feature amount information includes, in addition to the feature amount, a second detection likelihood indicating the detection likelihood of the feature amount,
The certainty factor determining unit determines the certainty factor of the feature amount of the attribute associated with the control point for which the first detection likelihood is lower than a threshold value of the feature amount of the attribute acquired by the feature amount information acquiring unit. An information processing apparatus characterized by setting a value lower than a second detection likelihood. a control point information acquisition unit that acquires control point information including position information of control points that define the skeleton of the person from an image of the person;
a feature amount information acquisition unit that acquires feature amount information including a feature amount related to attributes of the person from the image;
a certainty factor determination unit that determines a certainty factor indicating reliability of the feature amount based on the control point information and the feature amount information,
The control point information includes, in addition to the position information of the control point, a first detection likelihood indicating the detection likelihood of the control point,
The feature amount information includes, in addition to the feature amount, a second detection likelihood indicating the detection likelihood of the feature amount,
The control point information acquisition unit detects whether or not the person is facing backward in the image based on the control point information,
The certainty determination unit determines the certainty of a feature amount of an attribute related to the face of the person when the control point information acquisition unit detects that the person is facing backward in the image. and setting the feature amount of the attribute acquired by the feature amount information acquiring unit to a value lower than the second detection likelihood. a control point information acquisition unit that acquires control point information including position information of control points that define the skeleton of the person from an image of the person;
a feature amount information acquisition unit that acquires feature amount information including a feature amount related to attributes of the person from the image;
a certainty factor determination unit that determines a certainty factor indicating reliability of the feature amount based on the control point information and the feature amount information,
The control point information includes, in addition to the position information of the control point, a first detection likelihood indicating the detection likelihood of the control point,
The feature amount information includes, in addition to the feature amount, a second detection likelihood indicating the detection likelihood of the feature amount,
The control point information acquisition unit detects whether or not the person overlaps with an object other than the person in the image based on the control point information,
When the control point information acquisition unit detects that the person overlaps with the object, the certainty determination unit associates the control point with the control point of the area of the person hidden by the object in the image. The information processing apparatus, wherein the certainty factor of the feature quantity of the attribute to be determined is set to a value lower than the second detection likelihood of the feature quantity of the attribute acquired by the feature quantity information acquisition unit. 5. The information processing apparatus according to any one of claims 1 to 4, wherein the attribute of the person is at least one of age, age group, sex, clothing, wear, and carrying item of the person. an information processing apparatus according to any one of claims 1 to 5;
a database that stores the certainty factor of the feature value determined by the information processing device;
a person search unit that searches for a person having a feature amount that matches a search condition, based on the certainty stored in the database, in response to a search request from the outside;
A person search system, comprising: a search result output unit that outputs a search result obtained by the person search unit. further comprising a camera that acquires the image and provides data of the image to the information processing device;
The database stores the certainty factor of the feature amount together with shooting information related to the shooting location and shooting date and time of the camera,
7. The person search system according to claim 6, wherein the search result output unit outputs, as the search result, the photographing information of the camera that has photographed the person having a feature amount that matches the search condition. . 8. The person search system according to claim 6, wherein said search result output unit includes a display unit for displaying said search result. a control point information acquisition step of acquiring control point information including position information of control points that define the skeleton of the person from an image of the person;
A feature amount information acquiring step of acquiring feature amount information including a feature amount related to attributes of the person from the image;
a certainty factor determination step of determining a certainty factor indicating the reliability of the feature amount based on the control point information and the feature amount information;
The control point information includes, in addition to the position information of the control point, a first detection likelihood indicating the detection likelihood of the control point,
The feature amount information includes, in addition to the feature amount, a second detection likelihood indicating the detection likelihood of the feature amount,
In the control point information acquisition step, based on the control point information, it is detected whether or not the plurality of persons overlap in the image;
In the certainty factor determination step, when it is detected in the control point information acquiring step that the target person whose certainty factor is to be determined overlaps with another person, A value lower than the second detection likelihood of the feature amount of the attribute acquired in the feature amount information acquisition step, as a certainty factor of the feature amount of the attribute associated with the control point of the hidden area of the target person. An information processing method characterized by setting a control point information acquisition step of acquiring control point information including position information of control points that define the skeleton of the person from an image of the person;
A feature amount information acquiring step of acquiring feature amount information including a feature amount related to attributes of the person from the image;
a certainty factor determination step of determining a certainty factor indicating the reliability of the feature amount based on the control point information and the feature amount information;
The control point information includes, in addition to the position information of the control point, a first detection likelihood indicating the detection likelihood of the control point,
The feature amount information includes, in addition to the feature amount, a second detection likelihood indicating the detection likelihood of the feature amount,
In the certainty factor determination step, the certainty factor of the feature amount of the attribute associated with the control point for which the first detection likelihood is lower than a threshold is determined as the feature amount of the attribute acquired in the feature amount information acquiring step. An information processing method characterized by setting a value lower than the second detection likelihood. a control point information acquisition step of acquiring control point information including position information of control points that define the skeleton of the person from an image of the person;
A feature amount information acquiring step of acquiring feature amount information including a feature amount related to attributes of the person from the image;
a certainty factor determination step of determining a certainty factor indicating the reliability of the feature amount based on the control point information and the feature amount information;
The control point information includes, in addition to the position information of the control point, a first detection likelihood indicating the detection likelihood of the control point,
The feature amount information includes, in addition to the feature amount, a second detection likelihood indicating the detection likelihood of the feature amount,
In the control point information obtaining step, based on the control point information, it is detected whether or not the person is facing backward in the image;
In the certainty factor determination step, when the control point information acquisition step detects that the person is facing backward in the image, the certainty factor of the feature amount of the attribute related to the face of the person is determined. and setting a value lower than the second detection likelihood of the feature amount of the attribute acquired in the feature amount information acquiring step. a control point information acquisition step of acquiring control point information including position information of control points that define the skeleton of the person from an image of the person;
A feature amount information acquiring step of acquiring feature amount information including a feature amount related to attributes of the person from the image;
a certainty factor determination step of determining a certainty factor indicating the reliability of the feature amount based on the control point information and the feature amount information;
The control point information includes, in addition to the position information of the control point, a first detection likelihood indicating the detection likelihood of the control point,
The feature amount information includes, in addition to the feature amount, a second detection likelihood indicating the detection likelihood of the feature amount,
In the control point information acquiring step, based on the control point information, it is detected whether or not the person overlaps with an object other than the person in the image;
In the certainty factor determination step, when it is detected in the control point information acquisition step that the person overlaps with the object, a control point of an area of the person hidden by the object in the image is associated with a control point. setting a certainty factor of the feature amount of the attribute to be lower than the second detection likelihood of the feature amount of the attribute acquired in the feature amount information acquiring step. 13. The information processing method according to any one of claims 9 to 12, wherein the attribute of the person is at least one of the person's age, age group, sex, clothing, wear, and carrying item. A person search method using the information processing method according to any one of claims 9 to 13,
a storage step of storing in a database the certainty factor of the feature value determined by the information processing method;
a person search step of searching for a person having a feature amount that matches a search condition in response to a search request from the outside, based on the certainty stored in the database;
and a search result output step of outputting search results obtained by the person search step. 15. The method according to claim 14, wherein in said search result outputting step, as said search result, photographing information relating to a photographing location and a photographing date and time of a camera which photographed said person having a feature quantity matching said search condition is output. person search method. 16. The person search method according to claim 14, wherein in said search result output step, said search result is displayed on a display unit. A program for causing a computer to execute the information processing method according to any one of claims 9 to 13. A program for causing a computer to execute the person search method according to any one of claims 14 to 16.

Images