JP6975666B2 - Person identification system and person identification method - Google Patents

Description

The present invention relates to a technique for identifying a person.

The following prior arts are the background technologies in this technical field. Patent Document 1 (Japanese Unexamined Patent Publication No. 2012-48550) describes a specific person detection system using a face image. Further, in Patent Document 2 (Japanese Unexamined Patent Publication No. 2002-150296), when authenticating a visitor to a facility, a person to be authenticated who is a visitor inserts his / her hand into the authentication box to obtain a goo. A series of movements such as choki-par are performed, this kind of movement is photographed by a camera device, the captured image is subjected to predetermined image processing by the image processing unit, and then the image processing is performed by the judgment unit. The correlation coefficient between the applied image data and the motion image data of the hand of the certifier registered in advance is obtained, and if it is equal to or more than a predetermined threshold value, the certifier corresponds to the motion image data. It is intended to make a judgment, and an authentication device intervening a habit peculiar to the certifier when performing authentication is described.

Japanese Unexamined Patent Publication No. 2012-48550 Japanese Unexamined Patent Publication No. 2002-150296

Real-Time Object Detection, [Searched on January 6, 2018], Internet <URL: https://pjreddie.com/darknet/yolo/> SSD: Single Shot MultiBox Detector, [Search January 6, 2018], Internet <URL: https://github.com/weiliu89/caffe/tree/ssd>

As mentioned above, there are various methods for identifying a person, but a technique for identifying a moving person without using a facial image has not been established. In particular, it is difficult to take a face image of a worker who is working in a factory or the like, and it is difficult for the worker to identify the worker without stopping the work and turning his face to the camera.

Further, as described in Patent Document 2, if a person is specified by the movement of the person, the desired accuracy cannot be obtained, and the authentication accuracy is improved by using the authentication ID (input of a number or presentation of a card) or a face image in combination. Need to improve.

A typical example of the invention disclosed in the present application is as follows. That is, it is a person identification system that identifies a person using a moving image, and includes an arithmetic unit that executes a program and a storage device connected to the arithmetic unit, and the storage device is the attendance of the person to be specified. The data is stored, and the arithmetic unit extracts a plurality of still images from the acquired moving images, extracts the feature amount of the movement of the person from the extracted plurality of still images, and constructs the feature amount of the movement of the person. Using the obtained learning data, a probability indicating who is the person with the extracted feature amount is calculated, and with reference to the attendance data, if the person is absent, the probability is set to 0. It is characterized by setting.

According to one aspect of the present invention, a person in operation can be identified without interrupting the operation. Issues, configurations and effects other than those mentioned above will be clarified by the description of the following examples.

It is a figure which shows the logical configuration example of the person identification system in a learning phase. It is a figure which shows the logical configuration example of the person identification system in a prediction phase. It is a figure which shows the physical configuration example of the person identification system. It is a figure which shows the example of the person identification auxiliary data. It is a flowchart of the process executed by a person identification system. It is a flowchart of the details of the process of step S5 of FIG.

1 and 2 are diagrams showing a logical configuration example of the person identification system according to the embodiment of the present invention, FIG. 1 shows a logical configuration in a learning phase, and FIG. 2 shows a logical configuration in a prediction phase.

In the learning phase, in the person identification system of this embodiment, as shown in FIG. 1, a moving image storage unit 200 for storing a moving image taken by a camera 100, a learning unit 700 for learning the relationship between movement and a person, and a learning unit 700. It has learned data 500 learned by the learning unit 700. The learning unit 700 is composed of a plurality of functional blocks, and has a function 300 for converting a moving image into a time-series still image, a function 310 for cutting out a person from the still image, and a function 320 for setting the name of the cut out person. , With an artificial intelligence learning function 400.

Further, in the person identification system of this embodiment, as shown in FIG. 2, in the prediction phase, a moving image storage unit 200 for storing a moving image taken by the camera 100, a prediction unit 800 for identifying a person, and learned data 500. And the person identification auxiliary data 600 which is used auxiliary for identifying a person. The same functions as those in the learning phase (FIG. 1) described above are designated by the same reference numerals. The prediction unit 800 is composed of a plurality of functional blocks, and has a function 300 for converting a moving image into a time-series still image, a function 310 for cutting out a person from a still image, and learned data with reference to the learned data 500. It includes a function 330 for extracting a person candidate from the person candidate and a function 340 for collating the person candidate and the specific auxiliary data with reference to the person identification auxiliary data 600.

FIG. 3 is a diagram showing a physical configuration example of the person identification system according to the embodiment of the present invention.

The person identification system of this embodiment is composed of a computer having a processor (CPU) 11, a memory 12, an auxiliary storage device 13, a communication interface 14, an input interface 15, and an output interface 18.

The processor 11 executes a program stored in the memory 12. The memory 12 includes a ROM which is a non-volatile storage element and a RAM which is a volatile storage element. The ROM stores an invariant program (for example, BIOS) and the like. The RAM is a high-speed and volatile storage element such as a DRAM (Dynamic Random Access Memory), and temporarily stores a program executed by the processor 11 and data used when the program is executed. Note that all or part of the processing performed by the person identification system by executing the program may be performed by hardware (for example, FPGA).

The auxiliary storage device 13 is a large-capacity, non-volatile storage device such as a magnetic storage device (HDD) or a flash memory (SSD). Specifically, the auxiliary storage device 13 stores the data of the moving image storage unit 200, the learned data 500, and the person identification auxiliary data 600. Further, the auxiliary storage device 13 stores a program executed by the processor 11. That is, the program is read from the auxiliary storage device 13, loaded into the memory 12, and executed by the processor 11.

The communication interface 14 is a network interface device that controls communication with other devices according to a predetermined protocol.

The input interface 15 is an interface to which a keyboard 16, a mouse, etc. are connected and which receives input from an operator. The output interface 18 is an interface to which a display device 19, a printer (not shown), and the like are connected, and the execution result of the program is output in a format that can be visually recognized by the operator. The person identification system may be operated by an input / output interface provided in a terminal connected via a network.

The program executed by the processor 11 is provided to the person identification system via a removable medium (CD-ROM, flash memory, etc.) or a network, and is stored in a non-volatile auxiliary storage device 13 which is a non-temporary storage medium. Therefore, the person identification system may have an interface for reading data from removable media.

The person identification system is a computer system composed of one computer physically or a plurality of computers logically or physically configured, and is a virtual system constructed on a plurality of physical computer resources. It may operate on a computer.

FIG. 4 is a diagram showing an example of the definition of the person identification auxiliary data 600 according to the embodiment of the present invention.

The definition of the person identification auxiliary data 600 shown in FIG. 4 is shown by columns of data type, data content, and supplement of data content, but the actual person identification auxiliary data 600 is determined according to the person to be determined. The data related to the data content 602 as an item is registered.

The person identification auxiliary data 600 includes at least one determination item of attendance data, entry / exit data, personal position data, and access restriction data.

The attendance data is data showing the attendance status on the current day, for example, data that records whether absenteeism or attendance is made for each specific target person, and is data that can be acquired from an attendance management system that manages the attendance of employees. By referring to the attendance data, you can see whether you are currently absent or attending work. When the person candidate is absent, it can be determined that the probability that the person candidate exists in the moving image is low.

The entry / exit data records the room in which the specific target person is located. For example, the data is the data that records the person in the room for each room or the data that records the room in which the specific target person is in the room. Yes, it is the data that can be obtained from the entrance / exit management system. The current location can be estimated by referring to the entry / exit data. When the person candidate is not present in the room, it can be determined that the probability that the person candidate exists in the moving image is low.

The personal position data records the position of a specific target person. For example, the position is recorded for each specific target person, and the position information for recording the location of the person in real time using GPS, a beacon, or the like. It is the data that can be obtained from the system. The current location can be estimated by referring to the person's position data. If the person candidate is not near the camera, it can be determined that the probability that the person candidate exists in the moving image is low.

The restricted access data records areas in the workplace where access of a specific target person is restricted or permitted. For example, an restricted area (area determined by a room or coordinates) is recorded for each specific target person. Data or data of people who are permitted (or prohibited) to enter each area (for example, work place). By referring to the restricted access data, the place where the person is likely to exist can be found. When the entry of the person candidate is restricted, it can be determined that the probability that the person candidate exists in the moving image is low.

The person identification auxiliary data 600 may be any information useful for estimating the position of the person to be specified, and data other than those illustrated may be used.

FIG. 5 is a flowchart of processing executed by the person identification system according to the embodiment of the present invention.

After the person identification system is started, the processes of steps S1 to S7 are repeatedly executed at predetermined time intervals. Note that this process may be executed at any timing (for example, a request from an operator, an event occurrence, etc.).

First, a moving image in which a specific target person is photographed is acquired and stored in the moving image storage unit 200 (S1). For example, acquire a video of a worker working in a factory work place.

After that, the still image conversion function 300 of the prediction unit 800 converts the acquired moving image into a time-series still image (S2). For example, a 30 fps moving image contains 1800 frames per minute. Cut out 40 frames from it at regular intervals. This method has the following advantages. First, since the still image is acquired evenly from the whole, there is a high possibility that a scene useful for identifying a person is included. Moreover, since it is acquired at equal intervals, the speed of movement can be known.

Next, the person cutting function 310 of the prediction unit 800 cuts out a person from a still image (S3). In step S3, a known technique for identifying the type of the object to be captured in the image and the position in the image (for example, Selective Search, SSD (Single Shot MultiBox Detector)) or the like can be adopted.

After that, the processes of steps S4 to S7 are repeatedly executed for each person cut out. In step S4, the person candidate extraction function 330 of the prediction unit 800 extracts the person candidate from the learned data 500.

After that, the person identification auxiliary data matching function 340 of the prediction unit 800 collates the person candidate with the person identification auxiliary data 600 so that the probability of the person is low if the person should not exist in the moving image. Estimate the probability of a person candidate (S5). This process will be described later with reference to FIG.

The person identification function 350 of the prediction unit 800 outputs the attribute (name, etc.) of the person with the highest probability (S6), and records the determination result (S7). It should be noted that the attributes of a predetermined number of people may be output from the top with the highest probability.

FIG. 6 is a detailed flowchart of the process of step S5 of FIG. In the process shown in FIG. 6, a plurality of determination items are determined in order, and when the person should not exist in the moving image in one determination item, the probability of the person is set to 0 and the process ends. Hereinafter, the processing procedure will be specifically described.

First, the attendance data is determined with reference to the person identification auxiliary data 600 (S51). Specifically, when the person candidate is absent, the probability of the person is set to 0 (S55).

On the other hand, when the person candidate is at work, the entry / exit data is determined (S52). Specifically, when the person candidate does not exist in the room where the moving image was shot, the probability of the person is set to 0 (S55).

On the other hand, when the person candidate is present in the room where the moving image is shot, the person position data is determined (S53). Specifically, when the person candidate is not near the camera that shot the moving image, the probability of the person is set to 0 (S55).

On the other hand, when the person candidate is in the vicinity of the camera that shot the moving image, the access restriction data is determined (S54). Specifically, when the person candidate is detected in the area where access is restricted, the probability of the person is set to 0 (S55). On the other hand, when the person candidate is detected outside the restricted area, the person candidate may exist in the moving image, so the process of step S5 is terminated and the process proceeds to step S6.

Unlike the illustrated example, when the person should not be present in the moving image, it may be further determined by the following determination item by multiplying by a coefficient that lowers the probability of the person. In this way, the probability of the person can be determined by using a plurality of determination items in combination.

As described above, according to the person identification system of the embodiment of the present invention, a plurality of still images are extracted from the acquired moving images, and the feature amount of the movement of the person is extracted from the extracted plurality of still images. , Since the probability of indicating who the person with the extracted feature amount is is calculated using the learning data constructed by the feature amount of the movement of the person, the person in motion can be selected without interrupting the movement. Can be identified. In particular, it is difficult to take a face image of a worker who is working in a factory or the like, but the worker can be identified without stopping the work and turning his face to the camera.

In addition, since a still image is extracted by extracting a plurality of frames at equal intervals longer than the frame rate of the acquired video, it is possible that the still image is evenly acquired from the whole and a scene useful for identifying a person is included. It is possible to identify a person with high accuracy. Moreover, since it is acquired at equal intervals, the speed of movement can be known.

In addition, the person identification auxiliary data including at least one determination item of attendance data, entry / exit data, position data, and entry restriction data is stored, and the person specified from the moving image is identified by referring to the person identification auxiliary data. If it should not appear in the moving image, the probability of the person is calculated so that the probability of the person is low, so it is highly accurate without using the authentication ID (input of number or presentation of card). You can identify the person. Therefore, the person who is working can be identified with high accuracy without interrupting the work.

It should be noted that the present invention is not limited to the above-mentioned examples, but includes various modifications and equivalent configurations within the scope of the attached claims. For example, the above-mentioned examples have been described in detail in order to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to those having all the described configurations. Further, a part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Further, the configuration of another embodiment may be added to the configuration of one embodiment. In addition, other configurations may be added / deleted / replaced with respect to a part of the configurations of each embodiment.

Further, each configuration, function, processing unit, processing means, etc. described above may be realized by hardware by designing a part or all of them by, for example, an integrated circuit, and the processor realizes each function. It may be realized by software by interpreting and executing the program to be executed.

Information such as programs, tables, and files that realize each function can be stored in a memory, a hard disk, a storage device such as an SSD (Solid State Drive), or a recording medium such as an IC card, an SD card, or a DVD.

In addition, the control lines and information lines show what is considered necessary for explanation, and do not necessarily show all the control lines and information lines necessary for mounting. In practice, it can be considered that almost all configurations are interconnected.

11 Processor 12 Memory 13 Auxiliary storage device 14 Communication interface 15 Input interface 18 Output interface 100 Camera 200 Video storage unit 500 Learned data 600 Person identification auxiliary data 700 Learning unit 800 Prediction unit

Claims (6)

It is a person identification system that identifies a person using a moving image.
It includes an arithmetic unit that executes a program and a storage device connected to the arithmetic unit.
The storage device stores the attendance data of the person to be identified,
The arithmetic unit is
Extract multiple still images from the acquired video and
From the plurality of extracted still images, the feature amount of the movement of the person is extracted.
Using the learning data constructed by the feature amount of the movement of the person, the probability indicating who the person with the extracted feature amount is is calculated .
A person identification system, characterized in that the probability is set to 0 when the person is absent with reference to the attendance data. The person identification system according to claim 1.
The arithmetic unit is a person identification system characterized in that a still image is extracted by extracting a plurality of frames at equal intervals longer than the frame rate of the acquired moving image. The person identification system according to claim 1.
The storage device stores person identification auxiliary data including at least one of entry / exit data, location data, and access restriction data.
The arithmetic unit calculates the probability of the person so that the probability of the person is low when the person specified from the moving image should not appear in the moving image with reference to the person identification auxiliary data. A person identification system characterized by doing. It is a person identification method executed by a person identification system that identifies a person using a moving image.
The person identification system has an arithmetic unit for executing a program and a storage device connected to the arithmetic unit.
The storage device stores the attendance data of the person to be identified,
The method is
The procedure for the arithmetic unit to extract a plurality of still images from the acquired moving image, and
A procedure in which the arithmetic unit extracts a feature amount of a person's movement from the extracted plurality of still images, and a procedure.
The arithmetic unit uses the learning data constructed by the feature quantity of an operation of the human, saw including a step of calculating a probability that a person of the extracted feature indicates who is,
In the procedure for calculating the probability, the arithmetic unit refers to the attendance data and sets the probability to 0 when the person is absent . The method for identifying a person according to claim 4.
A method for identifying a person, characterized in that, in the procedure for extracting a still image, the arithmetic unit extracts a plurality of frames at equal intervals longer than the frame rate of the acquired moving image. The method for identifying a person according to claim 4.
The storage device stores person identification auxiliary data including at least one of entry / exit data, location data, and access restriction data.
In the method, if the arithmetic unit refers to the person identification auxiliary data and the person specified from the moving image should not appear in the moving image, the probability of the person is reduced. A person identification method comprising a procedure for calculating the probability of.

Images