WO2024057385A1 - 情報処理システム、情報処理方法、及び、記録媒体 - Google Patents

情報処理システム、情報処理方法、及び、記録媒体 Download PDF

Info

Publication number
WO2024057385A1
WO2024057385A1 PCT/JP2022/034160 JP2022034160W WO2024057385A1 WO 2024057385 A1 WO2024057385 A1 WO 2024057385A1 JP 2022034160 W JP2022034160 W JP 2022034160W WO 2024057385 A1 WO2024057385 A1 WO 2024057385A1
Authority
WO
WIPO (PCT)
Prior art keywords
face
information
information processing
face information
unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/034160
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
数夫 柘植
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to PCT/JP2022/034160 priority Critical patent/WO2024057385A1/ja
Priority to US18/855,049 priority patent/US20250252776A1/en
Priority to JP2024546542A priority patent/JP7810279B2/ja
Publication of WO2024057385A1 publication Critical patent/WO2024057385A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation

Definitions

  • This disclosure relates to the technical fields of information processing systems, information processing methods, and recording media.
  • Patent Document 1 includes a plurality of image input units into which images are input, a detection unit detects an object area from the image input from the image input unit, and a feature amount is extracted from the image of the object area detected by the detection unit.
  • This document describes a technique for controlling and monitoring detection processing and feature extraction processing performed on images input by a plurality of image input units based on the detection result of an object region by a detection unit.
  • Patent Document 2 discloses that object information of one or more objects is acquired for each frame image of a moving image, the processing load of an image processing device is monitored, and object information to be matched is selected based on the processing load.
  • Patent Document 3 describes a method of first face image matching using a first image taken of a first area on a route where there is a flow of people from a first area to a second area located upstream from a regulating section. a processing unit that performs second face image matching using the first candidate face images narrowed down based on the results and a second image taken of the second area; and a communication unit that outputs the results of the second face image matching.
  • a verification device for use at a gate is described, which is equipped with a regulating section for regulating the flow of people.
  • Patent Document 4 discloses that a subject to be subject to subject matching is detected from an image, a method to be used in subject matching for the subject is specified from among a plurality of methods, and predetermined conditions are set according to the specified method.
  • An external device that can determine whether a subject satisfies a predetermined condition, select an image of a subject determined to satisfy a predetermined condition, and perform subject matching on the selected subject image using multiple methods. The technology for transmitting is described.
  • An object of this disclosure is to provide an information processing system, an information processing method, and a recording medium that aim to improve the techniques described in prior art documents.
  • One aspect of the information processing system includes an acquisition unit that sequentially acquires face information of face areas detected from sequentially captured images, a storage unit that sequentially stores the acquired face information, and a storage unit that satisfies a predetermined condition. an extraction unit that performs an extraction operation for extracting a feature quantity from each face information to be extracted, which is at least a part of the face information stored in the storage unit; and authentication means for facial recognition.
  • One aspect of the information processing method is to sequentially acquire face information of face areas detected from sequentially captured images, sequentially store the acquired face information in a storage means, and when a predetermined condition is satisfied. , performs an extraction operation of extracting feature amounts from each of the face information to be extracted, which is at least a part of the face information stored in the storage means, and performs face authentication using the feature amounts.
  • the computer sequentially acquires facial information of face areas detected from sequentially captured images, sequentially stores the acquired facial information in a storage means, and satisfies a predetermined condition. In this case, performing an extraction operation of extracting a feature quantity from each face information to be extracted, which is at least a part of the face information stored in the storage means, and performing face authentication using the feature quantity.
  • a computer program for executing the information processing method is recorded.
  • FIG. 1 is a block diagram showing the configuration of an information processing system in the first embodiment.
  • FIG. 2 is a block diagram showing the configuration of an information processing system in the second embodiment.
  • FIG. 3 is a flowchart showing the flow of information processing operations of the information processing system in the second embodiment.
  • FIG. 4 is a conceptual diagram of the information processing operation of the information processing system in the second embodiment.
  • FIG. 5 is a conceptual diagram of an information processing system in the third embodiment.
  • FIG. 6 is a block diagram showing the configuration of an information processing system in the third embodiment.
  • FIG. 7 is a flowchart showing the flow of information processing operations of the information processing system in the third embodiment.
  • FIG. 8 is a conceptual diagram of the information processing operation of the information processing system in the third embodiment.
  • FIG. 9 is a block diagram showing the configuration of an information processing system in the fourth embodiment.
  • FIG. 10 is a flowchart showing the flow of information processing operations of the information processing system in the fourth embodiment.
  • FIG. 1 is a block diagram showing the configuration of an information processing system S1 in the first embodiment.
  • the information processing system S1 includes an acquisition section 11, a storage section 12, an extraction section 13, and an authentication section 14.
  • the acquisition unit 11 sequentially acquires face information of face areas detected from sequentially captured images.
  • the storage unit 12 sequentially stores the acquired face information.
  • the extraction unit 13 performs an extraction operation of extracting a feature amount from each piece of face information to be extracted, which is at least a part of the face information stored in the storage unit 12.
  • the authentication unit 14 performs face authentication using feature amounts.
  • the information processing system S1 in the first embodiment extracts a feature amount from each piece of face information to be extracted, which is at least part of the face information stored in the storage unit 12, when a predetermined condition is satisfied. That is, by temporarily storing face information in the storage unit 12 before the extraction operation, a large amount of face information can be processed at once. Thereby, the number of feature values extracted per time can be increased, and throughput can be increased.
  • the information processing system S2 in the second embodiment may be applied to a situation where it is necessary to authenticate a plurality of subjects P one after another.
  • FIG. 2 is a block diagram of the information processing system S2 in the second embodiment.
  • the information processing system S2 in the second embodiment may include an information processing device 2 and an imaging device C. [2-1: Configuration of information processing device 2]
  • the information processing device 2 includes a calculation device 21 and a storage device 22. Furthermore, the information processing device 2 may include a communication device 23, an input device 24, and an output device 25. However, the information processing device 2 does not need to include at least one of the communication device 23, the input device 24, and the output device 25.
  • the arithmetic device 21, the storage device 22, the communication device 23, the input device 24, and the output device 25 may be connected via a data bus 26.
  • the arithmetic device 21 includes at least a GPU (Graphics Processing Unit).
  • the arithmetic device 21 may further include, for example, at least one of a CPU (Central Processing Unit) and an FPGA (Field Programmable Gate Array).
  • Arithmetic device 21 reads a computer program.
  • the arithmetic device 21 may read a computer program stored in the storage device 22.
  • the arithmetic device 21 reads a computer program stored in a computer-readable and non-temporary recording medium using a recording medium reading device (not shown) provided in the information processing device 2 (for example, an input device 24 described later). You can also read it using .
  • the arithmetic device 21 may acquire a computer program from a device (not shown) located outside the information processing device 2 via the communication device 23 (or other communication device) (that is, it may not be downloaded). (or may be loaded). The arithmetic device 21 executes the loaded computer program. As a result, within the arithmetic device 21, a logical functional block for executing the operations that the information processing device 2 should perform is implemented. That is, the arithmetic device 21 can function as a controller for realizing a logical functional block for executing operations (in other words, processing) that the information processing device 2 should perform.
  • FIG. 2 shows an example of logical functional blocks implemented within the arithmetic unit 21 to execute information processing operations.
  • the arithmetic device 21 includes an acquisition unit 211, which is a specific example of the "acquisition means" described in the appendix described later, a face information processing unit 212, and An extraction unit 213 that is a specific example of an “extraction unit” and an authentication unit 214 that is a specific example of an “authentication unit” described in the appendix to be described later are realized.
  • the acquisition unit 211 may include an image reception unit 2111 and a face area detection unit 2112.
  • the face information processing section 212 may include a storage control section 2121. Details of the operations of the image receiving section 2111, face area detecting section 2112, storage controlling section 2121, extracting section 213, and authentication section 214 will be described later in detail with reference to FIGS. 3 and 4.
  • the storage device 22 can store desired data.
  • the storage device 22 may temporarily store a computer program executed by the arithmetic device 21.
  • the storage device 22 may temporarily store data that is temporarily used by the arithmetic device 21 when the arithmetic device 21 is executing a computer program.
  • the storage device 22 may store data that the information processing device 2 stores for a long period of time.
  • the storage device 22 may include at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device. good. That is, the storage device 22 may include a non-temporary recording medium.
  • the storage device 22 may include a face information storage section 221, which is a specific example of a "storage means" described in the appendix described later, a feature amount storage section 222, and a registered information database DB.
  • the storage device 22 does not need to include at least one of the face information storage section 221, the feature amount storage section 222, and the registered information database DB.
  • at least one of the face information storage section 221, the feature amount storage section 222, and the registered information database DB may be implemented in another device.
  • the communication device 23 is capable of communicating with devices external to the information processing device 2 via a communication network (not shown).
  • the information processing device 2 may transmit and receive signals to and from the imaging device C via the communication device 23.
  • the input device 24 is a device that accepts information input to the information processing device 2 from outside the information processing device 2.
  • the input device 24 may include an operating device (for example, at least one of a keyboard, a mouse, and a touch panel) that can be operated by the operator of the information processing device 2.
  • the input device 24 may include a reading device capable of reading information recorded as data on a recording medium that can be externally attached to the information processing device 2.
  • the output device 25 is a device that outputs information to the outside of the information processing device 2.
  • the output device 25 may output the information as an image.
  • the output device 25 may include a display device (so-called display) capable of displaying an image indicating information desired to be output.
  • the output device 25 may output the information as audio.
  • the output device 25 may include an audio device (so-called speaker) that can output audio.
  • the output device 25 may output information on paper. That is, the output device 25 may include a printing device (so-called printer) that can print desired information on paper.
  • FIG. 3 is a flowchart showing the flow of information processing operations performed by the information processing system S2 in the second embodiment.
  • FIG. 3(a) shows the flow of the storing operation of the face information FI
  • FIG. 3(b) shows the flow of the extraction operation and the authentication operation.
  • the storing operation of the face information FI shown in FIG. 3(a), the extracting operation shown in FIG. 3(b), and the face authentication operation may be performed in parallel.
  • FIG. 4 is a conceptual diagram of information processing operations performed by the information processing system S2 in the second embodiment.
  • the image receiving unit 2111 acquires the captured image I (step S20).
  • the imaging device C may sequentially capture the captured images I.
  • the image receiving unit 2111 acquires captured images I sequentially captured by the imaging device C via the communication device 23.
  • the operation from "start” to "end” shown in FIG. 3(a) is an operation related to one captured image I. You can.
  • the operation from "start" to "end” shown in FIG. 3(a) may be an operation for each frame.
  • the face area detection unit 2112 detects the face area F from the captured image I (step S21).
  • the face area detection unit 2112 receives the captured image I from the image receiving unit 2111, and detects the face area F included in the captured image I. For example, when the image receiving unit 2111 acquires the captured image I illustrated in FIG. Good too.
  • the storage control unit 2121 sequentially acquires the face information FI of the detected face area F and sequentially stores it in the face information storage unit 221 (step S22).
  • the face information storage unit 221 may be a storage unit that temporarily stores each face information FI before batch processing by the extraction unit 213, which will be described later.
  • the face information FI may include a face image of a face area F cut out from the captured image I.
  • the image receiving unit 2111 stores the captured image I as illustrated in FIG. 4(a).
  • the storage control unit 2121 may sequentially store face information FI3, face information FI4, face information FI5, and face information FI6 in the face information storage unit 221, as illustrated in FIG. 4(c). good. If the face information storage unit 221 stores the face information FI of the target person P corresponding to the detected face information FI, the storage control unit 2121 stores the target person P stored in the face information storage unit 221. The face information FI may be deleted. For example, if the face information FI1 and the face information FI6 are face information of the same person, the storage control unit 2121 may delete the face information FI1 from the face information storage unit 221. That is, the storage control unit 2121 may discard old face information FI of the same person in order to store the latest face information FI. The storage control unit 2121 may use a technique such as a face tracking function to determine whether the facial information FI is of the same person.
  • the face area detection unit 2112 determines whether all face areas F have been detected (step S23). If all face areas F have been detected (step S23: Yes), the operation regarding one captured image I ends. If all face areas F have not been detected (step S23: No), the process returns to step S21. That is, when the captured image I includes a plurality of face areas F, the face area detection unit 2112 may detect all the face areas F.
  • step S24 it is determined whether the number of face information FI stored in the face information storage unit 221 has reached a predetermined number.
  • the face information processing unit 212 determines whether a predetermined number of face information FIs are stored in the face information storage unit 221. If the number of face information FI stored in the face information storage unit 221 has reached the predetermined number (step S24: Yes), the process moves to step S26.
  • step S24 determines that a predetermined amount of time has passed since the previous extraction operation by the extraction unit 213. It is determined whether or not it has been done (step S25). If the predetermined time has not elapsed since the previous extraction operation by the extraction unit 213 (step S25: No), the process returns to step S24.
  • step S26 If a predetermined time has elapsed since the previous extraction operation by the extraction unit 213 (step S25: Yes), the process moves to step S26. That is, in at least one of the following cases: when the number of face information FIs stored in the face information storage unit 221 reaches a predetermined number, and when a predetermined time has passed since the previous extraction operation by the extraction unit 213, step The process moves to S26. In other words, the information processing system S2 waits for a predetermined period of time to elapse before proceeding to step S26 if the predetermined number of face information FIs have not been accumulated.
  • step S24: Yes, or step S25: Yes) the face information processing section 212 instructs the extraction section 213 to perform an extraction operation.
  • the face information processing unit 212 may collectively input to the extraction unit 213 at least some of the face information FI stored in the face information storage unit 221 to be extracted.
  • the face information processing unit 212 may input a predetermined number of face information FI at once.
  • the face information processing unit 212 may instruct the extraction unit 213 to perform batch processing using GPU resources.
  • the extraction unit 213 performs an extraction operation of extracting feature amounts from each of the face information FIs to be extracted, which are at least some of the face information FIs stored in the face information storage unit 221 (step S26).
  • the extraction unit 213 may perform the extraction operation using GPU resources.
  • the extraction unit 213 may receive a plurality of pieces of face information FI from the face information storage unit 221 via the face information processing unit 212, and may collectively extract facial feature amounts FF from the face information FI using GPU resources.
  • the facial feature amount FF extracted by the extraction section 213 may be stored in the feature amount storage section 222.
  • the number of face information FIs that the extraction unit 213 receives from the face information storage unit 221 via the face information processing unit 212 may be a predetermined number.
  • the face information processing unit 212 stores face information FI1, face information FI2, face information FI3, face information FI4, and face information FI5, as illustrated in FIG. 4(d).
  • the amount FF4 and the facial feature amount FF5 may be stored.
  • the facial feature amount FF1 may be the facial feature amount FF extracted from the facial image included in the facial information FI1
  • the facial feature amount FF2 may be the facial feature amount FF extracted from the facial image included in the facial information FI2.
  • the facial feature amount FF3 may be a facial feature amount FF extracted from a face image included in the face information FI3, and the facial feature amount FF4 may be a facial feature amount FF extracted from a face image included in the face information FI4.
  • the facial feature amount FF may be the facial feature amount FF extracted from the image
  • the facial feature amount FF5 may be the facial feature amount FF extracted from the facial image included in the facial information FI5.
  • the storage control unit 2121 may perform a deletion operation of deleting each face information FI stored in the face information storage unit 221 to be extracted.
  • the authentication unit 214 performs face authentication using the facial feature amount FF (step S27).
  • the authentication unit 214 may sequentially acquire facial feature amounts FF from the feature amount storage unit 222.
  • the authentication section 214 may perform face authentication by comparing the facial feature amount FF stored in the feature amount storage section 222 with information registered in the registered information database DB. For example, when the information processing system S2 in the second embodiment is applied to control the opening and closing of gates, the gates may be sequentially controlled to open depending on the success of authentication.
  • the authentication unit 214 determines whether face authentication has been performed for all extracted facial feature amounts FF (step S28). If face authentication is performed for all extracted facial features FF (step S28: Yes), or if a predetermined condition is satisfied (step S24 or step S25: Yes), the operation ends. If face authentication has not been performed for all of the extracted facial features FF (step S28: No), the process returns to step S27. That is, if there are a plurality of facial feature amounts FF extracted in step S26, the authentication unit 214 sequentially performs face authentication on all the facial feature amounts FF.
  • the extraction unit 213 performs the extraction operation using GPU resources
  • at least one of the acquisition unit 211, the face information processing unit 212, and the authentication unit 214 may also perform the operation using GPU resources.
  • At least one of the acquisition unit 211, the face information processing unit 212, and the authentication unit 214 may perform operations using at least one of GPU resources, CPU resources, and FPGA resources.
  • the information processing system S2 in the second embodiment operates when the number of face information FIs stored in the face information storage unit 221 reaches a predetermined number, and when a predetermined time has elapsed since the previous extraction operation by the extraction unit 213.
  • the feature amount is extracted from each of the face information FIs to be extracted, which are at least some of the face information FIs stored in the face information storage unit 221.
  • the extraction unit 213 uses GPU resources for the extraction operation, it is possible to process many face information FIs at once. That is, since the face information FI is temporarily stored in the face information storage unit 221 before the extraction operation by the extraction unit 213, it is possible to process a large number of face information FI at once.
  • the number of items processed in one extraction operation can be increased, the number of items processed per time can be increased, and high throughput can be achieved. Furthermore, since the extraction processing time per item is shortened, the waiting time until the subject P is authenticated can be reduced, and a short response can be realized. In the device that performs authentication, the overall cost is reduced when facial feature FF is extracted using GPU resources, compared to when facial feature FF is extracted using CPU resources. can be reduced. [3: Third embodiment]
  • the information processing system S3 in the third embodiment is used not only in situations where it is necessary to authenticate a plurality of subjects P one after another, but also in situations where it is not known which subject P should be authenticated until the last moment, and in which the order of authentication is changed. May be applied to the scene.
  • the third embodiment will be described below, taking as an example a case where the authentication result by the information processing system S3 is applied to the opening/closing control of the gate device G through which the subject P can pass.
  • FIG. 5 is a conceptual diagram of the information processing system S3 in the third embodiment. As shown in FIG. 5, the information processing system S3 in the third embodiment may include an information processing device 3 and an imaging device C.
  • the gate device G is a device that can control the passage of subjects P1, P2, P3, P4, P5, and P6 (referred to as subjects P if not distinguished). .
  • the gate device G may be opened or closed depending on the authentication result by the information processing system S3.
  • the gate device G may include a plurality of flapper gates FG.
  • the subject P may be able to pass through at least one flapper gate FG among the plurality of flapper gates FG.
  • the imaging device C may be provided for each flapper gate FG.
  • the imaging device C1 provided near the flapper gate FG1 may be capable of imaging the vicinity of the flapper gate FG1.
  • the imaging device C2 provided near the flapper gate FG2 may be capable of imaging the vicinity of the flapper gate FG2.
  • the imaging device C3 provided near the flapper gate FG3 may be capable of imaging the vicinity of the flapper gate FG3.
  • the distance between the imaging device C and the subject P and the distance between the flapper gate FG and the subject P may correspond.
  • the second area A2 is the destination of the target person P, and may be an area where only the authenticated target person P can enter.
  • the flapper gate FG is controlled to be open for the authenticated subject P, and the subject P can move from the first area A1 to the second area A2 via the flapper gate FG.
  • the flapper gate FG is a member that can control passage of the subject P. Although a plate-shaped member is illustrated as the flapper gate FG, a rod-shaped member may also be used.
  • the state of the flapper gate FG may be controlled by the information processing device 3 based on the information processing result of the subject P by the information processing device 3. Specifically, when the information processing device 3 successfully authenticates the target person P (that is, it is determined that the target person P matches the registered person), the state of the flapper gate FG is set such that the target person P is The information processing device 3 may control the flapper gate FG so that the flapper gate FG can be opened.
  • the registered person may be, for example, a person who is permitted to enter the second area A2.
  • the state of the flapper gate FG is such that the target person P is in the flapper gate state.
  • the information processing device 3 may control the FG so that it is in a closed state where it cannot pass through.
  • FIG. 6 is a block diagram showing the configuration of the information processing system S3 in the third embodiment.
  • the information processing system S3 in the third embodiment may include an information processing device 3 and an imaging device C, similar to the information processing system S2 in the second embodiment.
  • the information processing device 3 in the third embodiment includes an arithmetic device 21 and a storage device 22, similar to the information processing device 2 in the second embodiment. Furthermore, the information processing device 3 in the third embodiment may include a communication device 23, an input device 24, and an output device 25 similarly to the information processing device 2 in the second embodiment. However, the information processing device 3 does not need to include at least one of the communication device 23, the input device 24, and the output device 25.
  • the acquisition unit 211 included in the calculation device 21 further includes a distance calculation unit 3113
  • the face information processing unit 212 included in the calculation device 21 includes a priority determination unit 3122 and a selection unit 3123.
  • the information processing device 2 is different from the information processing device 2 in the second embodiment in that the information processing device 2 further includes the following.
  • Other features of the information processing device 3 may be the same as other features of the information processing device 2 in the second embodiment. Therefore, in the following, parts that are different from each of the embodiments already described will be described in detail, and descriptions of other overlapping parts will be omitted as appropriate.
  • the information processing operation performed by the information processing system S3 is different from the information processing operation performed by the information processing system S2 shown in FIG. 3 in operations in step S21, step S22, and step S26.
  • the face area detection unit 2112 detects a face area F from the captured image I (step S211).
  • the face area detection unit 2112 may detect face areas F included in the captured image I in order of size starting from the face area F having the largest size.
  • the face area detection unit 2112 acquires time information indicating the time regarding the detection of the face area F (step S212).
  • the time is, for example, time or date and time
  • the time information is, for example, time information or date and time information.
  • the face area detection unit 2112 may detect when the face area F is detected from the captured image I. Alternatively, the face area detection unit 2112 may detect when the captured image I is acquired. Alternatively, the face area detection unit 2112 may detect when the captured image I is captured.
  • the distance calculation unit 3113 calculates distance information (step S213).
  • the distance information is information indicating the distance from the location where images are sequentially captured to the location of the face area F. That is, the distance calculation unit 3113 calculates the distance between the imaging device C and the subject P corresponding to the face area F.
  • the distance calculation unit 3113 calculates the distance between the imaging device C and the subject P based on the size of the face area F, the distance between the eyes included in the face area F, the size comparison result with the comparison object in the captured image I, etc. may be calculated. Further, the distance calculation unit 3113 may calculate the distance between the imaging device C and the subject P based on the measurement result by the distance measurement sensor.
  • the face information FI may include at least a face image of the face area F, time information, and distance information.
  • the time information and distance information may also be called meta information.
  • the storage control unit 2121 stores the face information FI in the face information storage unit 221 (step S221). That is, the storage control unit 2121 causes the face information storage unit 221 to store time information and distance information in association with the face image.
  • the storage control unit 2121 may cause the face information storage unit 221 to associate face information FI of the same subject P detected from the captured images I captured by the plurality of imaging devices C and store them in the face information storage unit 221.
  • the priority determination unit 3122 determines the priority of the face information FI stored in the face information storage unit 221 according to the time information and distance information (step S222). The priority determining unit 3122 determines whether the target person P corresponding to the face information FI newly stored in the face information storage unit 221 is the target person P corresponding to the already stored face information FI according to the time information and the distance information. It may be determined whether the image capturing device C is closer to the imaging device C than the image capturing device C. If the priority determining unit 3122 determines that the subject P corresponding to the new face information FI is closer to the imaging device C than the subject P corresponding to the accumulated face information FI, the priority determination unit 3122 selects the new face information. The priority of FI may be increased.
  • the face information storage unit 221 stores face information FI1, face information FI2, face information FI3, and face information FI4, and the image receiving unit 2111 stores face information FI1, face information FI2, face information FI3, and face information FI4, and the image receiving unit 2111 stores face information )
  • face information FI1, face information FI2, face information FI3, and face information FI4 are detected in the order of face information FI1, face information FI2, face information FI3, and face information FI4.
  • the size of the face area F corresponding to the face information FI1, face information FI2, face information FI3, and face information FI4 is the same as the size of the face area F5, and the face area F6 is larger than the face area F5. It is assumed that the area F7 is larger than the face area F6, and the face area F8 is larger than the face area F7. In this case, the priority determination unit 3122 determines the priority of the face information FI8 corresponding to the face area F8, the face information FI7 corresponding to the face area F7, and the face information FI6 corresponding to the face area F6, if detected earlier.
  • the priority may be set lower than the priority of the face information FI1 and higher than the priority of the face information FI2, the face information FI3, and the face information FI4.
  • the face information FI having a higher priority may be located higher in the face information storage unit 221 and may be located further to the left. That is, FIG. 8(c) exemplifies a case where the priorities are in the order of face information FI1, face information FI8, face information FI7, face information FI6, face information FI2, face information FI3, face information FI4, and face information FI5. You can leave it there.
  • the selection unit 3123 selects face information FI (step S261).
  • the selection unit 3123 selects face information FI to be included in the extraction target from the face information FI stored in the face information storage unit 221 based on the priority determined in step S222. For example, in the case illustrated in FIG. 8C, the selection unit 3123 extracts face information FI1, face information FI8, face information FI7, face information FI6, and face information FI2 stored in the face information storage unit 221. You may select it as a target.
  • the extraction unit 213 detects a facial feature amount FF from each piece of facial information FI selected by the selection unit 3123 according to an instruction from the facial information processing unit 212 (step S262).
  • the feature amount storage unit 222 stores the facial feature amount FF1 extracted from each of the face information FI1, the face information FI8, the face information FI7, the face information FI6, and the face information FI2.
  • facial feature amount FF8 facial feature amount FF7, facial feature amount FF6, and facial feature amount FF2 may be stored.
  • step S26 similar to the information processing operation in the second embodiment, the authentication unit 214 performs face authentication processing using the facial feature amount FF. If the authentication is successful, the information processing device 3 may control the opening of the gate corresponding to the facial feature amount FF for which the authentication was successful. [3-4: Technical effects of information processing system S3]
  • the information processing system S3 in the third embodiment increases the priority of the face information FI and performs the processing first.
  • the target of the extraction operation to be performed That is, the priority of the facial information FI corresponding to the subject P who should be authenticated first is increased so that the facial feature amount FF is extracted first.
  • the perceived time from the arrival of the subject P at the location to be authenticated to the completion of authentication can be shortened. If applied to the gate device G, the perceived time from the time the subject P arrives at the gate until it is unlocked can be shortened.
  • the information processing system S4 in the fourth embodiment needs to authenticate a plurality of subjects P one after another, and it is difficult to determine which subject P should be authenticated first. It may be applied to situations where the authentication is not known until the last minute, or where the order of authentication is changed. Also in the fourth embodiment, similarly to the information processing system S3 in the third embodiment, the authentication result by the information processing system S4 may be applied to the opening/closing control of the gate device G through which the subject P can pass. [4-1: Configuration of information processing system S4]
  • FIG. 9 is a block diagram showing the configuration of the information processing system S4 in the fourth embodiment.
  • the information processing system S4 in the fourth embodiment includes an information processing device 4 and an imaging device, similar to the information processing system S2 in the second embodiment and the information processing system S3 in the third embodiment. It may also contain C.
  • the information processing device 4 in the fourth embodiment includes an arithmetic device 21 and a storage device 22, similar to the information processing device 2 in the second embodiment and the information processing device 3 in the third embodiment. It is equipped with Furthermore, the information processing device 4 in the fourth embodiment, like the information processing device 2 in the second embodiment and the information processing device 3 in the third embodiment, includes a communication device 23, an input device 24, and an output device 25. It may also have the following. However, the information processing device 4 does not need to include at least one of the communication device 23, the input device 24, and the output device 25.
  • the information processing device 4 in the fourth embodiment is similar to the second embodiment in that the face information processing unit 212 included in the arithmetic device 21 further includes a first change unit 4124, a second change unit 4125, and a third change unit 4126. This is different from the information processing device 2 in the third embodiment and the information processing device 3 in the third embodiment.
  • the first changing unit 4124 changes the predetermined number according to the rate of change in the number of face information FI stored in the face information storage unit 221.
  • the second changing unit 4125 changes the predetermined time according to the time from when the extraction unit 213 performed the previous extraction operation until the number of face information FIs stored in the face information storage unit 221 reaches a predetermined number.
  • the third changing unit 4126 compares the time from the start to the completion of the extraction operation with a predetermined time, and changes the imaging interval corresponding to the imaging interval of sequential imaging.
  • the predetermined number and the predetermined time are changed depending on the situation.
  • a settable lower limit and a settable upper limit are respectively determined for the predetermined number and the predetermined time.
  • FIG. 10 is a flowchart showing the flow of information processing operations performed by the information processing system S4 in the fourth embodiment.
  • the information processing operation shown in FIG. 10 may be an operation that is periodically performed at predetermined intervals.
  • the face information processing unit 212 monitors the rate of change in the number of face information FIs stored in the face information storage unit 221. Further, the face information processing unit 212 monitors the time from when the extraction unit 213 performs the previous extraction operation until the number of face information FI stored in the face information storage unit 221 reaches a predetermined number.
  • the face information processing unit 212 determines whether the time it takes for the number of face information FIs stored in the face information storage unit 221 to reach a predetermined number tends to be short or long. It is determined whether the length is short or long (step S40). For example, the face information processing unit 212 may perform the determination by comparing it with the average of the accumulation tendency of the face information FI for the past 5 minutes.
  • the first changing unit 4124 reduces the predetermined number. Further, the second changing unit 4125 shortens the predetermined time (step S41). By reducing the predetermined number and shortening the predetermined time, it is possible to prevent the waiting time until the data is accumulated from being wasted.
  • the predetermined time is set so that the time difference between the time to accumulate up to the predetermined number and the time until the elapse of the predetermined time falls within a certain range ( ⁇ ).
  • the face information processing unit 212 determines whether at least one of the predetermined number and the predetermined time has reached a settable lower limit (step S42). If at least one of the predetermined number and the predetermined time has reached the lower limit (step S42: Yes), it is predicted that the load on the entire device is lower than expected. Therefore, the third changing unit 4126 shortens the imaging interval (step S43).
  • the image receiving unit 2111 may gradually adjust the imaging interval of the imaging device C so that the imaging interval changes by the third changing unit 4126.
  • the third changing unit 4126 compares the time from the start to the completion of the extraction operation with a predetermined time, and sequentially changes the imaging interval of the imaging. That is, the third changing unit 4126 may compare the time corresponding to the predetermined number with the predetermined time, and may change the imaging interval of sequential imaging.
  • the third changing unit 4126 may change the imaging interval of the imaging device C according to the congestion information corresponding to the imaging device C.
  • the face area detection unit 2112 may calculate congestion information.
  • the congestion information is information indicating the degree of congestion in the vicinity of the imaging device C when the face information FI is detected.
  • the face area detection unit 2112 may calculate from the number of detected face areas F included in the captured image I.
  • the third changing unit 4126 may change the imaging interval of the crowded imaging device C so that it becomes longer with priority. If the area is crowded, it is predicted that there will be little movement of the subject P, and it can be expected that increasing the imaging interval and lowering the frame rate will have little effect.
  • the load is suppressed by lengthening the imaging interval, and delays are also suppressed by prioritizing the processing of the face information FI based on the distance between the subject P and the imaging device C.
  • step S42: No If at least one of the predetermined number and the predetermined time has not reached the lower limit (step S42: No), the information processing operation shown in FIG. 10 ends.
  • step S40 the first changing unit 4124 increases the predetermined number.
  • step S45 the second changing unit 4125 lengthens the predetermined time. If the time it takes to reach the predetermined number tends to be short, it can be determined that the process is congested, so the process moves to step S45, and increases the predetermined number and predetermined time to increase the number of batch processes to improve efficiency. Plan.
  • the face information processing unit 212 determines whether at least one of the predetermined number and the predetermined time has reached the upper limit (step S46). If the upper limit has been reached, it is predicted that the load on the entire device is higher than expected. If at least one of the predetermined number and the predetermined time has reached the upper limit (step S46: Yes), the third changing unit 4126 lengthens the imaging interval (step S47). The third changing unit 4126 may instruct the imaging device C to gradually lengthen the imaging interval. If at least one of the predetermined number and the predetermined time has not reached the upper limit (step S46: No), the process moves to step S44.
  • step S40 if there is neither a short tendency nor a long tendency (step S40: M), the face information processing unit 212 determines whether the load of at least one of the GPU resource and the CPU resource is high (step S44). For example, the trend of the usage rate of at least one of the GPU resource and the CPU resource may be compared with an average value for the past 5 minutes, etc.
  • the third changing unit 4126 changes the imaging interval to become longer (step S47).
  • the image receiving unit 2111 may gradually adjust the imaging interval of the imaging device C so that the imaging interval changes by the third changing unit 4126.
  • step S44 No If the load on at least one of the GPU resource and the CPU resource is not high (step S44: No), the process moves to step S43.
  • the face information processing unit 212 periodically monitors the trend of the number of face information FIs stored in the face information storage unit 221, and stores a predetermined number of FIs in the face information storage unit 221 before batch processing by the extraction unit 213. Adjust the predetermined time according to the load situation. This controls the throughput and response to be optimal for the environment. [4-3: Technical effects of information processing system S4]
  • the first changing unit 4124 dynamically changes the predetermined number according to the load situation, so that it can be controlled to optimize the throughput and response.
  • the extraction operation using GPU resources can be started immediately.
  • the first changing unit 4124 increases the predetermined number, it becomes possible to more effectively utilize the batch extraction operation using GPU resources.
  • the second changing unit 4125 dynamically changes the predetermined time according to the load situation, it is possible to control the throughput and response to be optimal.
  • the third changing unit 4126 dynamically changes the imaging interval according to the load situation, it is possible to perform control so that the throughput and response are optimized.
  • the face information processing unit 212 stores the face information FI corresponding to the subject P who is close to the imaging device C in the face information storage unit 221 later.
  • an excellent effect can be achieved when performing a process of changing the priority so that the process can be performed first. This is particularly effective when the number of face information FIs stored in the face information storage unit 221 exceeds a predetermined number of GPU resource batch processing targets.
  • acquisition means for sequentially acquiring face information of face areas detected from sequentially captured images; storage means for sequentially storing the acquired facial information; Extracting means for performing an extraction operation of extracting a feature amount from each face information to be extracted, which is at least a part of the face information stored in the storage means, when a predetermined condition is satisfied;
  • An information processing system comprising: authentication means that performs face authentication using the feature amount.
  • the extraction means uses GPU resources for the extraction operation, The information processing system according to claim 1, wherein the face information includes a face image of the face area.
  • the face information includes a face image of the face area, time information indicating a time related to detection of the face area, and distance information indicating a distance from the sequential imaging location to the location of the face area,
  • the priority of the face information stored in the storage means is determined according to the time information and the distance information, and the extraction target is determined from the face information stored in the storage means based on the priority.
  • [Additional note 8] Sequentially acquires face information of face areas detected from sequentially captured images, sequentially storing the acquired facial information in a storage means; When a predetermined condition is satisfied, performing an extraction operation of extracting feature amounts from each piece of face information to be extracted, which is at least a part of the face information stored in the storage means; An information processing method that performs face recognition using the feature amount.
  • S1, S2, S3, S4 Information processing system 2 3, 4 Information processing device 11, 211 Acquisition unit 2111 Image receiving unit 2112 Face area detection unit 12 Storage unit 212 Face information processing unit 2121 Storage control unit 221 Face information storage unit 222 Feature storage unit 13, 213 Extraction unit 14, 214 Authentication unit 3113 Distance calculation unit 3122 Priority determination unit 3123 Selection unit 4124 First change unit 4125 Second change unit 4126 Third change unit C Imaging device I Captured image F Face area FI Face information FF Face feature amount

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Collating Specific Patterns (AREA)
PCT/JP2022/034160 2022-09-13 2022-09-13 情報処理システム、情報処理方法、及び、記録媒体 Ceased WO2024057385A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
PCT/JP2022/034160 WO2024057385A1 (ja) 2022-09-13 2022-09-13 情報処理システム、情報処理方法、及び、記録媒体
US18/855,049 US20250252776A1 (en) 2022-09-13 2022-09-13 Information processing system, information processing method, and non-transitory recording medium
JP2024546542A JP7810279B2 (ja) 2022-09-13 2022-09-13 情報処理システム、情報処理方法、及び、コンピュータプログラム

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/034160 WO2024057385A1 (ja) 2022-09-13 2022-09-13 情報処理システム、情報処理方法、及び、記録媒体

Publications (1)

Publication Number Publication Date
WO2024057385A1 true WO2024057385A1 (ja) 2024-03-21

Family

ID=90274426

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/034160 Ceased WO2024057385A1 (ja) 2022-09-13 2022-09-13 情報処理システム、情報処理方法、及び、記録媒体

Country Status (3)

Country Link
US (1) US20250252776A1 (https=)
JP (1) JP7810279B2 (https=)
WO (1) WO2024057385A1 (https=)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017142614A (ja) * 2016-02-09 2017-08-17 レノボ・シンガポール・プライベート・リミテッド 情報処理装置
JP2021149589A (ja) * 2020-03-19 2021-09-27 株式会社キーエンス 光学読取装置
WO2022130616A1 (ja) * 2020-12-18 2022-06-23 富士通株式会社 認証方法、情報処理装置、及び認証プログラム

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017142614A (ja) * 2016-02-09 2017-08-17 レノボ・シンガポール・プライベート・リミテッド 情報処理装置
JP2021149589A (ja) * 2020-03-19 2021-09-27 株式会社キーエンス 光学読取装置
WO2022130616A1 (ja) * 2020-12-18 2022-06-23 富士通株式会社 認証方法、情報処理装置、及び認証プログラム

Also Published As

Publication number Publication date
JP7810279B2 (ja) 2026-02-03
US20250252776A1 (en) 2025-08-07
JPWO2024057385A1 (https=) 2024-03-21

Similar Documents

Publication Publication Date Title
US11729194B2 (en) Automatic model monitoring for data streams
CN110365768B (zh) 分布式系统的数据同步方法、装置、介质、电子设备
EP3067829A1 (en) Person authentication method
JP7353825B2 (ja) 画像処理装置及びその方法、画像入力装置、画像処理システム、プログラム
CN120011091B (zh) 一种云平台中基于深度学习的数据流实时处理方法
RU2488234C1 (ru) Передающее устройство и способ передачи
KR20170123950A (ko) 얼굴 인식을 이용한 출입 관리 방법 및 시스템
CN108564028A (zh) 一种基于嵌入式系统的多线程人脸识别系统
US20250045895A1 (en) Information processing apparatus, information processing method, and non-transitory recording medium
WO2024057385A1 (ja) 情報処理システム、情報処理方法、及び、記録媒体
JP7315002B2 (ja) 物体追跡装置、物体追跡方法、及び、プログラム
US20150066840A1 (en) Analysis system
CN114078141A (zh) 用于控制对象跟踪的设备及其方法
JP7556338B2 (ja) 入場管理装置、入場管理方法、及びコンピュータプログラム
EP4118566B1 (en) Cascaded feature detectors for improved stream processing efficiency
JP2007066114A (ja) 分類器を用いた分類方法及びその装置
Abernethy et al. An efficient bandit algorithm for
JP4442113B2 (ja) 推定支援システム
EP4053788A1 (en) Authentication apparatus, authentication method, and recording medium
KR20210058614A (ko) 적응적인 임계치를 이용한 얼굴 검출 방법 및 장치
WO2013150786A1 (ja) 解析システム
KR102833559B1 (ko) 적응적 Ensemble 조절이 가능한 동영상 합성 판별 시스템 및 방법
US20240086510A1 (en) Face authentication system, face authentication method, and program
KR102615623B1 (ko) 얼굴 인증을 통한 출입 관리 방법 및 시스템
KR102837797B1 (ko) 하이브리드 객체탐지 및 객체트랙킹 방법, 및 시스템

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22958719

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18855049

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 2024546542

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

WWP Wipo information: published in national office

Ref document number: 18855049

Country of ref document: US

122 Ep: pct application non-entry in european phase

Ref document number: 22958719

Country of ref document: EP

Kind code of ref document: A1