JP2023099581A - Information processing system, information processing device, terminal, processing method, and program - Google Patents

Abstract

To provide an image processing device capable of monitoring a suspicious person even when the suspicious person is not known.SOLUTION: An image processing system comprises: a detection unit which with respect to a person appearing in a video captured from a camera selected based on an input to an input device, detects a gesture of the person; and an estimation unit which estimates a predetermined person candidate based on a result of the detection by the detection unit.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing system, an information processing device, a terminal, a processing method, and a program.

The installation of security cameras is increasing in condominiums, detached houses, urban areas, and the like.
Patent Literature 1 describes a technique for monitoring a suspicious person as a related technique.

Japanese Unexamined Patent Application Publication No. 2010-191620

By the way, the technique described in Patent Document 1 assumes that a suspicious person is known in advance, and cannot monitor a suspicious person if the suspicious person is not known.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide an information processing system, an information processing apparatus, a terminal, a processing method, and a program that can solve the above problems.

In order to achieve the above object, according to one aspect of the present disclosure, an image processing apparatus includes an appearance state determination unit that determines an appearance state of a person in a captured image, and a predetermined person candidate based on the appearance state. a predetermined person candidate estimating unit for estimating.

Further, according to one aspect of the present disclosure, an image processing system includes a camera and an image processing device, the camera captures an image of a predetermined area, and the image processing device captures a person in the captured image. and a predetermined person candidate estimating unit for estimating a predetermined person candidate based on the appearance state.

Further, according to one aspect of the present disclosure, the image processing method determines the appearance state of a person in a captured image, and estimates a predetermined person candidate based on the appearance state.

Further, according to one aspect of the present disclosure, the program comprises: a computer, an appearance state determining means for determining an appearance state of a person in a captured image; a predetermined person candidate estimation unit for estimating a predetermined person candidate based on the appearance state; function as a means.

By the processing of the image processing system of the present invention, it is possible to identify a person candidate based on specified conditions such as a suspicious person.

1 is a diagram showing the minimum configuration of an image processing device according to an embodiment of the present invention; FIG. 1 is a diagram showing the configuration of an image processing system according to a first embodiment of the present invention; FIG. It is a figure which shows the structure of the center system by this embodiment. It is a figure which shows the structure of the input device by this embodiment. It is a figure which shows the structure of the 1st edge device by this embodiment. FIG. 4 is a diagram showing the configuration of a second edge device according to this embodiment; FIG. 4 is a diagram showing the configuration of a third edge device according to this embodiment; It is a figure which shows the 1st data table by this embodiment. It is a figure which shows the 2nd data table by this embodiment. It is a figure which shows the 3rd data table by this embodiment. It is a figure which shows the comprehensive data table by this embodiment. FIG. 4 is a first diagram showing a processing flow according to this embodiment; FIG. 4 is a second diagram showing a processing flow according to this embodiment; It is a figure which shows the structure of the input device by 1st embodiment of this invention.

FIG. 1 is a diagram showing the minimum configuration of an image processing apparatus 100 according to an embodiment of the invention.
The image processing apparatus 100 of the present invention includes at least an appearance state determination unit 101 and a predetermined person candidate estimation unit 102, as shown in FIG.
Appearance state determination unit 101 determines the appearance state of a person in a captured image. The appearance state of a person is, for example, a result indicating the appearance of a person, such as the number of appearances of a person in a video, the appearance of a person for a long time, or the person acting suspiciously such as frequently looking around.
A predetermined person candidate estimation unit 102 estimates a predetermined person candidate based on the appearance state. A predetermined person candidate is, for example, a person such as a shoplifter, a suspicious purchaser, or a stalker.

<First embodiment>
FIG. 2 is a diagram showing the configuration of the image processing system 1 according to the first embodiment of the invention.
First, an image processing system 1 including an image processing apparatus 100 according to the first embodiment of the present invention will be described.
As shown in FIG. 2, the image processing system 1 according to the present embodiment includes a center system 10, an input device 20, a first edge device 30a (information processing device), a second edge device 30b (information processing device), and a third edge device. A device 30c (information processing device), a first camera device 40a, a second camera device 40b, and a third camera device 40c are provided.
In the following description, the first edge device 30a, the second edge device 30b, the third edge device 30c, . . . , the n-th edge device 30n, . Also, the first camera device 40a, the second camera device 40b, the third camera device 40c, . . . , the n-th camera device 40n, .

The input device 20 is, for example, a smart phone, a mobile phone, a personal computer, or the like. The user performs a camera selection information input operation of inputting camera selection information for selecting a camera from which a person feature amount, which is a person feature amount, is to be extracted. A person feature amount is information for specifying a person, such as face, height, gender, clothing, belongings, etc., for example.

The user performs a start instruction input operation for inputting a start instruction to start extracting the person feature amount. The input device 20 transmits camera selection information to the center system 10 based on the start instruction input operation by the user.

The center system 10 receives camera selection information from the input device 20 . The center system 10 identifies the edge device based on the input camera selection information.
The center system 10 sends camera selection information to each of the specified first edge device 30a, second edge device 30b, and third edge device 30c, and a feature quantity extraction start instruction to start extracting a person feature quantity. to send.

When each of the first edge device 30a, the second edge device 30b, and the third edge device 30c receives the camera selection information and the human feature amount extraction start instruction from the center system 10, each of them has the camera indicated by the camera selection information. A camera image is acquired from the camera device 40 . Then, each of the first edge device 30a, the second edge device 30b, and the third edge device 30c extracts the person feature amount from the acquired camera image. Each of the first edge device 30a, the second edge device 30b, and the third edge device 30c generates an index for the extracted person feature amount. Note that the index is information including, for example, an input device ID (IDentifier) that is an identifier of the input device 20 that has transmitted the camera selection information, a person feature amount ID that is an identifier of a person feature amount assigned to each person feature amount, and the like. . Each of the first edge device 30a, the second edge device 30b, and the third edge device 30c transmits to the center system 10 the extracted human feature amount and the generated index.
Also, each of the first edge device 30a, the second edge device 30b, and the third edge device 30c records the extracted human feature quantity and the generated index in the storage unit. Also, each of the first edge device 30a, the second edge device 30b, and the third edge device 30c assigns a generated index to the camera image and records the camera image in the storage unit.

When the center system 10 receives the camera image, the person feature amount, and the index from each of the first edge device 30a, the second edge device 30b, and the third edge device 30c, the center system 10 receives the camera image, the person feature amount, and the index. It is associated and recorded in the storage unit.

Further, the user performs a suspicious person extraction start instruction input operation for inputting a suspicious person extraction start instruction for instructing the start of extraction of a suspicious person. The input device 20 transmits a start of suspicious person extraction to the center system 10 based on a suspicious person extraction start instruction from the user.

The center system 10 inputs a person candidate determination condition for determining a person candidate such as a suspicious person, and records it in the storage unit. It should be noted that the person candidate determination condition is a determination condition for determining an appearance state, which is a result indicating the appearance of a person, such as a person appearing many times in a video or appearing for a long time. When the center system 10 inputs a suspicious person extraction start instruction for instructing the start of extraction of a suspicious person, the center system 10 identifies the suspicious person based on the camera image, the person feature amount, the index, and the appearance state stored in the storage unit. . The center system 10 records the suspicious person identification result of the identified suspicious person in the storage unit.

The user performs an end instruction input operation for inputting an end instruction to end the extraction of the person feature amount. The input device 20 transmits an end instruction to the center system 10 based on the end instruction input operation by the user.

When the center system 10 receives the end instruction from the input device 20, the center system 10 transmits the feature amount extraction end instruction indicating the end of extraction of the person feature amount to the first edge device 30a, the second edge device 30b, and the third edge device 30c. Send to each.

When each of the first edge device 30a, the second edge device 30b, and the third edge device 30c receives the feature quantity extraction end instruction from the center system 10, it ends the extraction of the person feature quantity in the camera video.

As described above, in the image processing system 1 according to the embodiment of the present invention, the camera device 40 in the area selected by the user takes an image. Each of the edge devices 30 extracts a person feature amount from the video captured by the camera device 40 . The center system 10 identifies, as a suspicious person, a person who satisfies the appearance state determined using the person candidate determination condition from among the persons having the person feature quantity extracted by the camera device 40 .

FIG. 3 is a diagram showing the configuration of the center system 10 according to this embodiment.
Next, the configuration of the center system 10 according to this embodiment will be described.
The center system 10 according to this embodiment includes an image processing device 100 as shown in FIG. The image processing apparatus 100 includes an appearance state determination unit 101, a predetermined person candidate estimation unit 102, a communication unit 103, a control unit 104, a camera selection unit 105, a storage unit 106, and a condition input unit 107. .

Appearance state determination unit 101 determines the appearance state of a person in a captured image based on a person candidate determination condition.
The predetermined person candidate estimation unit 102 estimates a predetermined person candidate based on the appearance state of the person in the captured image. For example, the predetermined person candidate estimating unit 102, based on the camera image, the person feature amount, the index, and the appearance state received by the center system 10 from each of the edge devices 30, the camera having the person feature amount that satisfies the appearance state. To identify a person in a video as a suspicious person.
The communication unit 103 transmits and receives information to and from each of the input device 20 and the edge device 30 .

The control unit 104 performs various controls necessary for processing performed by the center system 10 . For example, the control unit 104 controls transmission and reception of information performed by the communication unit 103 . In the following description, even if it is not specifically described that the control unit 104 controls the transmission and reception of information performed by the communication unit 103, the control unit 104 controls the transmission and reception of information performed by the communication unit 103. is controlling

The camera selection unit 105 designates a camera that captures the appearance state of the person in the captured image determined by the appearance state determination unit 101 .
The storage unit 106 stores information necessary for various processes performed by the center system 10 . For example, the storage unit 106 associates and stores camera images, person feature amounts, and indexes received by the center system 10 from each of the edge devices 30 .
The condition input unit 107 acquires a person candidate determination condition and records the acquired person candidate determination condition in the storage unit 106 .

FIG. 4 is a diagram showing the configuration of the input device 20 according to this embodiment.
Next, the configuration of the input device 20 according to this embodiment will be described.
The input device 20 according to this embodiment includes a communication unit 201, an information input unit 202, and a display control unit 203, as shown in FIG.

A communication unit 201 transmits and receives information to and from the center system 10 .
The information input unit 202 detects an input operation by the user and outputs a signal based on the detected input operation. For example, the information input unit 202 transmits the camera selection information to the center system 10 when detecting the camera selection information input operation by the user. Further, for example, the information input unit 202 transmits a start instruction to the center system 10 when detecting a start instruction input operation by the user. Further, for example, the information input unit 202 transmits an end instruction to the center system 10 when detecting an end instruction input operation by the user.

The display control unit 203 controls display on a display screen (not shown). For example, the display control unit 203 controls display of information transmitted by the input device 20 to the center system 10 and information received by the input device 20 from the center system 10 on the display screen.

Next, the configuration of the edge device 30 according to this embodiment will be described.
FIG. 5 is a diagram showing the configuration of the first edge device 30a according to this embodiment.
First, the configuration of the first edge device 30a according to this embodiment will be described.
As shown in FIG. 5, the first edge device 30a according to the present embodiment includes a first communication unit 301a, a first control unit 302a, a first camera control unit 303a, a first storage unit 304a, and a first feature It comprises an amount extractor 305a, a first image receiver 306a, and a first camera interface 307a.

The first communication unit 301a transmits and receives information to and from each of the center system 10 and the first camera device 40a.
The first control unit 302a performs various controls necessary for processing performed by the first edge device 30a. For example, the first control unit 302a controls transmission and reception of information performed by the first communication unit 301a. In the following description, even if it is not specifically described that the first control unit 302a controls transmission and reception of information performed by the first communication unit 301a, the first control unit 302a It controls transmission and reception of information performed by the communication unit 301a.

The first camera control unit 303a transmits a shooting control signal for controlling shooting to the first camera device 40a via the first camera interface 307a.
The first storage unit 304a stores various information necessary for processing performed by the first edge device 30a.

The first feature amount extraction unit 305a extracts the person feature amount captured by the camera included in the camera device 40 specified by the camera selection unit 105 . Specifically, the image received by the first image receiving unit 306a from the first camera device 40a is image-analyzed, and the person feature amount is extracted.

The first image receiving unit 306a receives the image captured by the first camera device 40a.
The first camera interface 307a is an interface provided in the first edge device 30a to connect the first edge device 30a and the first camera device 40a.

FIG. 6 is a diagram showing the configuration of the second edge device 30b according to this embodiment.
Next, the configuration of the second edge device 30b according to this embodiment will be described.
As shown in FIG. 6, the second edge device 30b according to the present embodiment includes a second communication unit 301b, a second control unit 302b, a second camera control unit 303b, a second storage unit 304b, a second feature It comprises an amount extractor 305b, a second image receiver 306b, and a second camera interface 307b.

The second communication unit 301b transmits and receives information to and from each of the center system 10 and the second camera device 40b.
The second control unit 302b performs various controls necessary for processing performed by the second edge device 30b. For example, the second control unit 302b controls transmission and reception of information performed by the second communication unit 301b. In the following description, even if it is not specifically described that the second control unit 302b controls transmission and reception of information performed by the second communication unit 301b, the second control unit 302b controls the second It controls transmission and reception of information performed by the communication unit 301b.

The second camera control unit 303b transmits a shooting control signal for controlling shooting to the second camera device 40b via the second camera interface 307b.
The second storage unit 304b stores various information necessary for processing performed by the second edge device 30b.

The second feature amount extraction unit 305b extracts the person feature amount captured by the camera included in the camera device 40 specified by the camera selection unit 105. FIG. Specifically, the image received by the second image receiving unit 306b from the second camera device 40b is image-analyzed, and the person feature amount is extracted.

The second image receiving unit 306b receives the image captured by the second camera device 40b.
The second camera interface 307b is an interface provided in the second edge device 30b to connect the second edge device 30b and the second camera device 40b.

FIG. 7 is a diagram showing the configuration of the third edge device 30c according to this embodiment.
Next, the configuration of the third edge device 30c according to this embodiment will be described.
As shown in FIG. 7, the third edge device 30c according to the present embodiment includes a third communication unit 301c, a third control unit 302c, a third camera control unit 303c, a third storage unit 304c, and a third feature It comprises an amount extractor 305c, a third video receiver 306c, and a third camera interface 307c.

The third communication unit 301c transmits and receives information to and from each of the center system 10 and the third camera device 40c.
The third control unit 302c performs various controls necessary for processing performed by the third edge device 30c. For example, the third control unit 302c controls transmission and reception of information performed by the third communication unit 301c. In the following description, even if it is not specifically described that the third control unit 302c controls transmission and reception of information performed by the third communication unit 301c, the third control unit 302c It controls transmission and reception of information performed by the communication unit 301c.

The third camera control unit 303c transmits a shooting control signal for controlling shooting to the third camera device 40c via the third camera interface 307c.
The third storage unit 304c stores various information necessary for processing performed by the third edge device 30c.

The third feature amount extraction unit 305c extracts the person feature amount captured by the camera included in the camera device 40 specified by the camera selection unit 105. FIG. For example, the third image receiving unit 306c performs image analysis on the image received from the third camera device 40c, and extracts the face, height, gender, clothes, belongings, and the like.

The third image receiving unit 306c receives the image captured by the third camera device 40c.
The third camera interface 307c is an interface provided in the third edge device 30c to connect the third edge device 30c and the third camera device 40c.

Next, the configuration of the camera device 40 according to this embodiment will be described.
Each camera device 40 includes one or more cameras. In this embodiment, the first camera device 40a is assumed to include a camera 401a, a camera 402a, and a camera 403a. Also, the second camera device 40b is assumed to include a camera 401b, a camera 402b, and a camera 403b. The third camera device 40c is assumed to include a camera 401c and a camera 402c.

Next, the data tables stored by the first edge device 30a, the second edge device 30b, and the third edge device 30c will be described.
FIG. 8 is a diagram showing the first data table TBL1 according to this embodiment.
The first storage unit 304a stores the person feature amount extracted by the first feature amount extraction unit 305a in the first data table TBL1. For example, as shown in FIG. 8, the first storage unit 304a stores the person feature amount extracted by the first feature amount extraction unit 305a from the images captured by the cameras 401a, 402a, and 403a for each camera. is stored in association with the date and time and the index. The date and time of capturing in the first data table TBL1 shown in FIG. It is the date and time when it started. Also, the release date and time in the first data table TBL1 shown in FIG. It is the date and time when it was completed. Also, the index is information including, for example, an input device ID that is an identifier of the input device 20 that has transmitted the camera selection information, a person feature amount ID that is an identifier of a person feature amount assigned to each person feature amount, and the like.

FIG. 9 is a diagram showing the second data table TBL2 according to this embodiment.
The second storage unit 304b stores the person feature amount extracted by the second feature amount extraction unit 305b in the second data table TBL2. For example, as shown in FIG. 9, the second storage unit 304b stores, for each camera, the person feature amount extracted by the second feature amount extraction unit 305b from the images captured by the cameras 401b, 402b, and 403b. is stored in association with the date and time and the index.

FIG. 10 is a diagram showing the third data table TBL3 according to this embodiment.
The third storage unit 304c stores the person feature amount extracted by the third feature amount extraction unit 305c in the third data table TBL3. For example, as shown in FIG. 10, the third storage unit 304c stores the person feature amount extracted by the third feature amount extraction unit 305c from the images captured by the camera 401c and the camera 402c for each camera with date and time. Stored in association with the index.

Next, the integrated data table STBL stored by the center system 10 will be described.
FIG. 11 is a diagram showing the integrated data table STBL according to this embodiment.
The storage unit 106 (person feature amount accumulation unit) stores the person feature amounts extracted by the first feature amount extraction unit 305a, the second feature amount extraction unit 305b, and the third feature amount extraction unit 305c in the general data table STBL. memorize to For example, the storage unit 106 stores a comprehensive data table STBL as shown in FIG. 11, in which the first data table TBL1, the second data table TBL2, and the third data table TBL3 are combined into one data table.

12 and 13 are diagrams showing the processing flow of the image processing system 1 according to this embodiment.
Next, processing of the image processing system 1 according to this embodiment will be described.
Here, the processing when the image processing system 1 monitors a suspicious person at a plurality of shops of the same series will be described. It is assumed that the first edge device 30a and the first camera device 40a are arranged in the first store A. It is assumed that the second edge device 30b and the second camera device 40b are arranged in the second store B. As shown in FIG. It is assumed that the third edge device 30c and the third camera device 40c are arranged at the third store C. Further, in the following description, an example will be described in which the employee of the first store A is the user and the person feature amount is the face.
Although the first edge device 30a, the second edge device 30b, and the third edge device 30c each perform information processing independently, the operation is the same. Therefore, in the processing flow shown in FIGS. It is described as an edge device 30 .

First, the process of extracting a person feature amount performed by the image processing system 1 according to the present embodiment shown in the process flow of FIG. 12 will be described.
The user performs a camera selection information input operation on the input device 20 to select cameras installed in a plurality of stores of the same series from which faces are to be extracted.
The information input unit 202 inputs camera selection information according to a camera selection information input operation by the user (step S1). The camera selection information is camera selection information indicating that all of the cameras installed in the first store A, the cameras installed in the second store B, and the cameras installed in the third store C are to be selected. be. The cameras installed in the first store A are a camera 401a, a camera 402a, and a camera 403a. The cameras installed in the second store B are a camera 401b, a camera 402b, and a camera 403b. The cameras installed at the third store C are a camera 401c and a camera 402c.
Then, the information input unit 202 determines whether or not to start extracting the face (step S2). If the information input unit 202 does not detect a start instruction input operation for inputting a start instruction for starting face extraction, the information input unit 202 returns to the process of step S2.

The user performs a start instruction input operation on the input device 20 to input a start instruction for starting face extraction.
The information input unit 202 detects a start instruction input operation by the user (step S2, YES), and transmits camera selection information to the center system 10 (step S3).

The camera selection unit 105 determines whether or not to receive camera selection information (step S4). When the camera selection information is not received, the camera selection unit 105 returns to the process of step S4.
When camera selection information is received from the input device 20 (step S4, YES), the camera selection unit 105 identifies an edge device based on the received camera selection information (step S5). Specifically, the camera selection unit 105 selects the first edge device 30a corresponding to the camera 401a, the camera 402a, and the camera 403a included in the camera selection information, and the second edge device 30a corresponding to the camera 401b, the camera 402b, and the camera 403b. Identify the second edge device 30b, the camera 401c, and the third edge device 30c corresponding to the camera 402c. For example, the camera selection unit 105 sequentially compares the cameras in the data table indicating the correspondence between cameras and edge devices and the cameras included in the received camera selection information, and identifies the matching cameras. The camera selection unit 105 identifies the edge device corresponding to the matching camera as the selected edge device.

The camera selection unit 105 supplies camera selection information to each of the specified first edge device 30a, second edge device 30b, and third edge device 30c, and a feature amount extraction start instruction for instructing the start of extraction of a person feature amount. is transmitted (step S6).

The first video image receiving unit 306a determines whether or not an instruction to start extracting a person's feature quantity has been received (step S7), and based on the determination result, determines whether to start extracting a person's feature quantity.

When the first image receiving unit 306a determines that the human feature amount extraction start instruction is not received (step S7, NO), the first video image receiving unit 306a determines not to start the human feature amount extraction. In this case, the first image receiving unit 306a receives camera images from each of the camera 401a, the camera 402a, and the camera 403a (step S8), and records the received camera images in the first storage unit 304a (step S9). ).

If the first video image receiving unit 306a determines that it has received a person feature amount extraction start instruction (step S7, YES), it determines to start person feature amount extraction. In this case, the first image receiving unit 306a receives camera images from each of the cameras 401a, 402a, and 403a indicated by the camera selection information (step S10).
The first feature amount extraction unit 305a extracts a face from each camera image received by the first image reception unit 306a (step S11). The first feature amount extraction unit 305a generates an index for the extracted face (step S12). The first feature amount extraction unit 305a transmits the extracted face and the generated index to the center system 10 (step S13).
Also, the first feature amount extraction unit 305a records the generated index together with the face information of the extracted face in the first data table TBL1 of the first storage unit 304a (step S14). Also, the first feature amount extraction unit 305a assigns the generated index to each camera image (step S15), and records each camera image in the first storage unit 304a (step S16).

Similarly to the first image receiving unit 306a, the second image receiving unit 306b determines to start extracting the person feature amount when determining that the human feature amount extraction start instruction has been received (step S7, YES). In this case, the second image receiving unit 306b receives camera images from each of the cameras 401b, 402b, and 403b indicated by the camera selection information (step S10).
The second feature amount extraction unit 305b extracts faces from the respective camera images received by the second image reception unit 306b (step S11). The second feature amount extraction unit 305b generates an index for the extracted face (step S12). The second feature amount extraction unit 305b transmits the extracted face and the generated index to the center system 10 (step S13).
The second feature amount extraction unit 305b also records the generated index together with the face information of the extracted face in the second data table TBL2 of the second storage unit 304b (step S14). The second feature amount extraction unit 305b also assigns the generated index to each camera image (step S15), and records each camera image in the second storage unit 304b (step S16).

Similarly to the first image receiving unit 306a, the third image receiving unit 306c determines to start extracting the person feature amount when determining that the human feature amount extraction start instruction has been received (step S7, YES). In this case, the third image receiving unit 306c receives camera images from each of the camera 401c and the camera 402c indicated by the camera selection information (step S10).
The third feature amount extraction unit 305c extracts faces from each camera image received by the third image reception unit 306c (step S11). The third feature quantity extraction unit 305c generates an index for the extracted face (step S12). The third feature amount extraction unit 305c transmits the extracted face and the generated index to the center system 10 (step S13).
Also, the third feature quantity extraction unit 305c records the face information of the extracted face and the generated index in the third data table TBL3 of the third storage unit 304c (step S14). Also, the third feature amount extraction unit 305c assigns the generated index to each camera image (step S15), and records each camera image in the third storage unit 304c (step S16).

When the appearance state determination unit 101 receives the face information and the index from each of the first edge device 30a, the second edge device 30b, and the third edge device 30c (step S17), it associates the received face information with the index. It is recorded in the comprehensive data table STBL of the storage unit 106 (step S18).
At this time, the center system 10 does not receive the camera images from the first edge device 30a, the second edge device 30b, and the third edge device 30c, and receives only the face information (human feature amount) and the index. The amount of communication data can be reduced by receiving. However, the center system 10 may receive camera images together with face information (human feature amounts) and indexes from each of the first edge device 30a, the second edge device 30b, and the third edge device 30c. Also, the center system 10 may record camera images received from each of the first edge device 30a, the second edge device 30b, and the third edge device 30c in the storage unit.

After that, the user performs an end instruction input operation for inputting an end instruction to end the extraction of the person feature amount.
The information input unit 202 inputs an end instruction based on the end instruction input operation by the user (step S19). The information input unit 202 transmits an end instruction to the center system 10 (step S20).

The appearance state determination unit 101 constantly determines whether or not to end the extraction of the person feature amount (step S21).
If the appearance state determination unit 101 does not receive an end instruction from the input device 20 (step S21, NO), the process returns to step S17.
Further, when the appearance state determination unit 101 receives an end instruction from the input device 20 (step S21, YES), the appearance state determination unit 101 outputs a feature amount extraction end instruction indicating the end of extraction of the person feature amount to the first edge device 30a, the second edge device 30a, and the second edge device 30a. It is transmitted to each of the device 30b and the third edge device 30c (step S22).

The first feature amount extraction unit 305a determines whether or not a feature amount extraction end instruction has been received from the center system 10 (step S23), and based on the determination result, determines whether to end extraction of the face in the camera image. judge.

When the first feature amount extraction unit 305a determines that the feature amount extraction end instruction has not been received (step S23, NO), the first feature amount extraction unit 305a determines not to end extraction of the face in the camera image. In this case, the first feature amount extraction unit 305a returns to the process of step S10.

Further, when the first image receiving unit 306a determines that the feature amount extraction end instruction has been received (step S23, YES), the first image receiving unit 306a determines to end extraction of the face in the camera image. In this case, the first image receiving unit 306a returns to the process of step S8.
The above is the extraction processing of the person feature amount performed by the image processing system 1 according to the present embodiment.

Next, the process of estimating a predetermined person candidate performed by the image processing system 1 according to the present embodiment shown in the process flow of FIG. 13 will be described.
After the process of extracting the person feature amount, the user who has operated the input device 20 or another user who identifies the suspicious person (for example, a police officer) issues a suspicious person extraction start instruction to start extracting the suspicious person. through a device installed directly or remotely to the center system 10 and connected by communication.

The storage unit 106 stores in advance a person candidate determination condition for determining a suspicious person input via the condition input unit 107 . For example, the conditions for judging a candidate to be a suspicious person include: visiting the same store many times on the same day, staying at the same store for a long time on the same day but not purchasing products, visits multiple stores of the same chain many times, spends a long time at multiple stores of the same chain on the same day but does not purchase products, frequently visits multiple stores of the same chain over multiple days but does not purchase products It is a condition for specifying a person who may shoplift. In addition, the criteria for judging a person candidate as a suspicious person are: the person who has visited the same store many times on the same day and has purchased products many times; The number of appearances of a person and the appearance time of a person, etc., determined based on the number of times a person appears and the time at which a person appears, etc. It is a condition that identifies a person who is likely to purchase a large amount of

When a suspicious person extraction start instruction is input (step S31), the predetermined person candidate estimation unit 102 identifies a suspicious person based on the camera image, face information, index, and appearance state stored in the storage unit 106 (step S31). S32). Specifically, predetermined person candidate estimating section 102 rearranges the data for each face information (feature amount) in general data table STBL stored in storage section 106 to determine the number of appearances of each piece of face information and the number of appearances of each piece of face information. The interpolated time and the like are specified, and when the person candidate determination condition is satisfied, the person corresponding to the face information is specified as a suspicious person. If a face is extracted from the camera image of the camera placed at the cash register, the predetermined person candidate estimation unit 102 determines that the person with that face has purchased the product. Further, when the face is not extracted in the camera image of the camera placed at the register position, the predetermined person candidate estimation unit 102 determines that the person with that face did not purchase the product. Further, the predetermined person candidate estimating unit 102 counts the number of appearances of the person with the face according to the number of times the face is extracted in the camera image of the camera arranged at the entrance of the store.
The predetermined person candidate estimating unit 102 records the camera image and the index, which are the suspicious person identification results for the identified suspicious person, in the storage unit 106 (step S33).
The above is the process of estimating the predetermined person candidate performed by the image processing system 1 according to the present embodiment.
When a suspicious person is identified, the predetermined person candidate estimating unit 102 determines the appearance of the suspicious person, such as the face, height, gender, clothing, and belongings of the suspicious person, based on the camera image from which the feature amount of the suspicious person is extracted. You may transmit the data of the camera image|video which can be understood to the input device 20. FIG. In this case, the display control unit 203 causes the display unit to display camera image data that shows the appearance of the suspicious person from the center system 10, so that the user can confirm the suspicious person. Further, the predetermined person candidate estimation unit 102 may provide the Metropolitan Police Department or the like with camera image data showing the appearance of the suspicious person. In this case, it can be used for crime prevention.

The image processing system 1 according to the first embodiment of the present invention has been described above. In the image processing apparatus 100 of the image processing system 1 described above, the appearance state determination unit 101 determines the appearance state of a person in the captured video. The predetermined person candidate estimation unit 102 estimates a predetermined person candidate based on the appearance state determined by the appearance state determination unit 101 .
By doing so, it is possible to identify a person candidate based on specified conditions such as a suspicious person.

<Second embodiment>
Next, an image processing system 1 including an image processing apparatus 100 according to a second embodiment of the invention will be described.
Similar to the image processing system 1 shown in FIG. 2, the image processing system 1 according to this embodiment includes a center system 10, an input device 20, a first edge device 30a, a second edge device 30b, a third edge device It comprises a device 30c, a first camera device 40a, a second camera device 40b, and a third camera device 40c.
The difference between the image processing system 1 according to this embodiment and the image processing system 1 according to the first embodiment is the input device 20 .

FIG. 14 is a diagram showing the configuration of the input device 20 according to this embodiment.
Next, the configuration of the input device 20 according to this embodiment will be described.
As shown in FIG. 14, the input device 20 according to this embodiment includes the communication unit 201, the information input unit 202, and the display control unit 203 shown in FIG. and a selection information generation unit 205 .

The position specifying unit 204 specifies the position of the input device 20 . A communication unit 201 transmits and receives information to and from the center system 10 .
Based on the position of the input device 20 specified by the position specifying unit 204, the camera selection information generating unit 205 generates camera selection information for specifying a camera that captures the appearance state of the person in the captured image. For example, the camera selection information generation unit 205 selects a camera installed within 1 km centering on the position of the input device 20 specified by the position specifying unit 204, a camera installed closest to the position of the input device 20, or the like. Generate camera selection information for selecting a camera for the conditions.

Next, processing of the image processing system 1 according to this embodiment will be described.
Here, processing when the image processing system 1 monitors a stalker will be described. In this embodiment, the user is a person who is being stalked.
The processing flow of the image processing system 1 according to this embodiment is the same as that of the image processing system 1 according to the first embodiment shown in FIGS. is.

Without performing the process of inputting camera selection information in step S1, the information input unit 202 determines whether or not to start face extraction in step S2. If the information input unit 202 does not detect a start instruction input operation for inputting a start instruction for starting face extraction, the information input unit 202 returns to the process of step S2. When the information input unit 202 detects the start instruction input operation, instead of the process of step S3, the position specifying unit 204 causes the input device 20 to be input at predetermined time intervals while the user carrying the input device 20 is moving. 20 are identified. For example, the position specifying unit 204 specifies the position of the input device 20 using a GPS (Global Positioning System) function. The position specifying unit 204 outputs the specified position of the input device 20 to the camera selection information generating unit 205 at predetermined time intervals. The camera selection information generation unit 205 generates camera selection information based on the position of the input device 20 input from the position identification unit 204 . The camera selection information generation unit 205 transmits the generated camera selection information to the center system 10 .

Each time the camera selection unit 105 determines in the process of step S4 that camera selection information has been received, the process of step S5 and subsequent steps may be performed.

Note that the storage unit 106 of the image processing apparatus 100 stores predetermined conditions such as a camera installed within 1 km from the position of the input device 20 and a camera installed closest to the position of the input device 20. In this case, the input device 20 may not include the camera selection information generation unit 205 and may transmit the position of the input device 20 specified by the position specifying unit 204 . Alternatively, the center system 10 may acquire the position of the input device 20 directly from GPS. In these cases, the camera selection unit 105 of the image processing apparatus 100 may specify the edge device based on the acquired position of the input device 20 and the predetermined condition stored in the storage unit 106 .

The image processing system 1 according to the second embodiment of the present invention has been described above. In the image processing apparatus 100 of the image processing system 1 described above, the appearance state determination unit 101 determines the appearance state of the person in the captured image based on the person candidate determination condition. The predetermined person candidate estimation unit 102 estimates a predetermined person candidate based on the appearance state determined by the appearance state determination unit 101 .
By doing so, it is possible to identify a person candidate based on specified conditions such as a suspicious person.

Further, the position specifying unit 204 specifies the position of the input device 20 at predetermined time intervals while the user carrying the input device 20 is moving. The position specifying unit 204 outputs the specified position of the input device 20 to the camera selection information generating unit 205 at predetermined time intervals. The camera selection information generation unit 205 generates camera selection information based on the position of the input device 20 input from the position identification unit 204 . The camera selection information generation unit 205 transmits the generated camera selection information to the center system 10 . Each time the camera selection unit 105 determines that the camera selection information generated by the camera selection information generation unit 205 is received, the processing from step S5 onward is performed.
By doing so, it is possible to cope with the case where it is necessary to change the camera that captures the appearance state of the person in the captured video, such as monitoring a stalker, from time to time.

In the embodiment of the present invention, as long as information is properly transmitted and received, some or all of the functional units included in the image processing device 100 are the first edge device 30a, the second edge device 30b, and the third edge device. It may be provided in any of the devices 30c, . . . nth edge device 30n. Also, some or all of the functional units included in the image processing apparatus 100 are in any of the first edge device 30a, second edge device 30b, third edge device 30c, . . . nth edge device 30n. It may be distributed and provided.

In the embodiment of the present invention, as long as information is properly transmitted and received, each functional unit of the first edge device 30a, the second edge device 30b, the third edge device 30c, . . . A part or all of them may be provided in the center system 10 .

In the embodiment of the present invention, when the input device 20 may be installed at an arbitrary fixed location (for example, as in the first embodiment, the position of the input device 20 is irrelevant to the position of the camera that shoots). case), some or all of the functional units of the input device 20 may be provided in the center system 10 as long as information is properly transmitted and received. In addition, in the embodiment of the present invention, when the input device 20 may be installed at any fixed location, as long as information is properly transmitted and received, some or all of the functional units of the input device 20 are It may be provided in any one of the first edge device 30a, the second edge device 30b, the third edge device 30c, . . . the nth edge device 30n. In addition, in the embodiment of the present invention, when the input device 20 may be installed at any fixed location, as long as information is properly transmitted and received, some or all of the functional units of the input device 20 are It may be distributed to any one of the center system 10, the first edge device 30a, the second edge device 30b, the third edge device 30c, . . . and the nth edge device 30n.

In the embodiment of the present invention, as long as information is properly transmitted and received, the first feature quantity extraction unit 305a, the second feature quantity extraction unit 305b, the third feature quantity extraction unit 305c, . The image processing apparatus 100 may include one or more of the extraction units 305n.

Note that each of the storage unit 106, the first storage unit 304a, the second storage unit 304b, and the third storage unit 304c in the present invention may be provided anywhere as long as appropriate information transmission/reception is performed. Further, each of the storage unit 106, the first storage unit 304a, the second storage unit 304b, and the third storage unit 304c exists in a plurality within a range where appropriate information transmission/reception is performed, and stores data in a distributed manner. may

In the processing flow according to the embodiment of the present invention, the order of processing may be changed as long as appropriate processing is performed.

Although the embodiments of the present invention have been described, each of the image processing device 100, the input device 20, the first edge device 30a, the second edge device 30b, and the third edge device 30c has a computer system inside. have. The process of the above-described processing is stored in a computer-readable storage unit in the form of a program, and the above-described processing is performed by reading and executing this program by a computer. Here, the computer-readable storage means a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Alternatively, the computer program may be distributed to a computer via a communication line, and the computer receiving the distribution may execute the program.

Further, the program may be for realizing part of the functions described above. Further, it may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in the computer system.

While several embodiments of the invention have been described, these embodiments have been presented by way of example and are not intended to limit the scope of the invention. Also, various omissions, replacements, and modifications can be made without departing from the scope of the invention.

Reference Signs List 1 Image processing system 10 Center system 20 Input device 30a First edge device 30b Second edge device 30c Third edge device 40a First camera Device 40b...Second camera device 40c...Third camera device 100...Image processing device 101...Appearance state determination unit 102...Predetermined person candidate estimation units 103, 201...Communication unit 104 ... control section 105 ... camera selection section 106 ... storage section 107 ... condition input section 202 ... information input section 203 ... display control section 204 ... position specifying section 205 ... Camera selection information generation unit 301a First communication unit 301b Second communication unit 301c Third communication unit 302a First control unit 302b Second control unit 302c Third Third control unit 303a First camera control unit 303b Second camera control unit 303c Third camera control unit 304a First storage unit 304b Second storage unit 304c Third storage unit 305a First feature amount extraction unit 305b Second feature amount extraction unit 305c Third feature amount extraction unit 306a First image reception unit 306b Second image Receiver 306c... Third video receiver 307a... First camera interface 307b... Second camera interface 307c... Third camera interface

Claims (15)

A system comprising a terminal and an information processing device capable of communicating with the terminal,
The terminal is
a determination unit that determines that a person appearing in an image captured by a camera is a predetermined person when the person behaves in a predetermined manner;
a transmission unit configured to transmit the video image of the person and the information of the person to the information processing device;
The information processing device is
a receiving unit that receives the information about the person and the video;
a storage unit that stores the information about the person and the video;
a transmission unit that transmits predetermined information to a terminal different from the terminal when the reception is performed;
An information processing system comprising The information on the person is a person feature amount that is information for specifying a person,
The information processing system according to claim 1. The predetermined information is data that shows the appearance of the person determined to be the predetermined person,
The information processing system according to claim 1 or 2. 4. The information processing system according to any one of claims 1 to 3, wherein said predetermined information is data captured by a camera. An information processing device communicable with a terminal,
a receiving unit that receives an image of a person performing a predetermined behavior in an image captured by a camera and information about the person;
a storage unit that stores the information about the person and the video;
a transmission unit that transmits predetermined information to a terminal different from the terminal when the reception is performed;
with
The person is a person who is determined to be a predetermined person when the terminal performs a predetermined behavior in the video,
Information processing equipment. The information on the person is a person feature amount that is information for specifying a person,
The information processing device according to claim 5 . The predetermined information is data that shows the appearance of the person determined to be the predetermined person,
The information processing apparatus according to claim 5 or 6. 8. The information processing apparatus according to any one of claims 5 to 7, wherein said predetermined information is data captured by a camera. Instead of the terminal, a determination unit that determines the person as the predetermined person when the person appearing in the video behaves in a predetermined manner,
The information processing apparatus according to any one of claims 5 to 8. A terminal capable of communicating with an information processing device,
The terminal is
a determination unit that determines that a person appearing in an image captured by a camera is a predetermined person when the person behaves in a predetermined manner;
a transmission unit configured to transmit the video image of the person and the information of the person to the information processing device;
The information processing device is
a receiving unit that receives the information about the person and the video;
a storage unit that stores the information about the person and the video;
a transmission unit that transmits predetermined information to a terminal different from the terminal when the reception is performed;
terminal. A processing method executed by a system comprising a terminal and an information processing device capable of communicating with the terminal,
The terminal is
when a person appearing in an image captured by a camera behaves in a predetermined manner, the person is determined to be a predetermined person;
transmitting the video in which the person appears and information about the person to the information processing device;
The information processing device is
receiving the information about the person and the video;
storing the information of the person and the video;
Sending predetermined information to a terminal different from the terminal when the reception is performed;
Processing method. A processing method executed by an information processing device capable of communicating with a terminal,
Receiving an image of a person performing a predetermined behavior in an image captured by a camera and information about the person;
storing the information of the person and the video;
transmitting predetermined information to a terminal different from the terminal when the reception is performed;
including
The person is a person who is determined to be a predetermined person when the terminal performs a predetermined behavior in the video,
Processing method. In the computer of the information processing device that can communicate with the terminal,
Receiving an image of a person performing a predetermined behavior in an image captured by a camera and information about the person;
storing the information of the person and the video;
transmitting predetermined information to a terminal different from the terminal when the reception is performed;
and
The person is a person who is determined to be a predetermined person when the terminal performs a predetermined behavior in the video,
program. A processing method executed by a terminal capable of communicating with an information processing device,
Determining a person appearing in an image captured by a camera as a predetermined person when the person behaves in a predetermined manner;
transmitting the video in which the person appears and information about the person to the information processing device;
The information processing device is
a receiving unit that receives the information about the person and the video;
a storage unit that stores the information about the person and the video;
a transmission unit that transmits predetermined information to a terminal different from the terminal when the reception is performed;
Processing method. In the computer of the terminal that can communicate with the information processing device,
Determining a person appearing in an image captured by a camera as a predetermined person when the person behaves in a predetermined manner;
transmitting the video in which the person appears and information about the person to the information processing device;
The information processing device is
a receiving unit that receives the information about the person and the video;
a storage unit that stores the information about the person and the video;
a transmission unit that transmits predetermined information to a terminal different from the terminal when the reception is performed;
program.

Images