WO2024241536A1 - 制御方法、制御プログラム、および情報処理装置 - Google Patents

制御方法、制御プログラム、および情報処理装置 Download PDF

Info

Publication number
WO2024241536A1
WO2024241536A1 PCT/JP2023/019350 JP2023019350W WO2024241536A1 WO 2024241536 A1 WO2024241536 A1 WO 2024241536A1 JP 2023019350 W JP2023019350 W JP 2023019350W WO 2024241536 A1 WO2024241536 A1 WO 2024241536A1
Authority
WO
WIPO (PCT)
Prior art keywords
person
information
image
detection area
authenticated
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/019350
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
リナ セプティアナ
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujitsu Ltd
Original Assignee
Fujitsu Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujitsu Ltd filed Critical Fujitsu Ltd
Priority to PCT/JP2023/019350 priority Critical patent/WO2024241536A1/ja
Priority to EP23938499.3A priority patent/EP4723030A1/en
Priority to CN202380098514.2A priority patent/CN121175716A/zh
Priority to JP2025521729A priority patent/JPWO2024241536A1/ja
Publication of WO2024241536A1 publication Critical patent/WO2024241536A1/ja
Priority to US19/392,006 priority patent/US20260073032A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
    • G06F21/31—User authentication
    • G06F21/32—User authentication using biometric data, e.g. fingerprints, iris scans or voiceprints
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20036—Morphological image processing
    • G06T2207/20044—Skeletonization; Medial axis transform
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • This matter relates to a control method, a control program, and an information processing device.
  • the present invention aims to provide a control method, a control program, and an information processing device that can improve the accuracy of detecting a subject.
  • control method executes a process in which, when a computer detects multiple people from an image that includes a subject who has been successfully authenticated using identification information, the computer controls a subject detection area for detecting the subject in the image.
  • the accuracy of detecting the target can be improved.
  • FIG. 1 is a diagram for explaining an overview of a continuous authentication technology.
  • 13A and 13B are diagrams illustrating check-in details.
  • FIG. 1 is a diagram illustrating an example in which a plurality of people are captured in an image captured by a camera.
  • FIG. 1A is a block diagram illustrating an example of an overall configuration of a biometric authentication system according to a first embodiment
  • FIG. 1B is a functional block diagram illustrating each function of an information processing device.
  • (a) is a diagram showing an overhead view of the entrance area
  • (b) is a diagram showing an example of a registered biometric data table stored in the registered data storage unit
  • (c) is a diagram showing an example of an image acquired by the acquisition unit from the matching camera.
  • FIG. 13A to 13C are diagrams for explaining the direction in which a subject is facing
  • 13D is a diagram illustrating an example of the size of a person area in an image
  • 1A is a diagram illustrating an example where multiple users check in in sequence
  • FIG. 1B is a diagram illustrating an example where multiple person areas are detected
  • FIG. 1C is a diagram illustrating an example of control of the detection target area.
  • 11 is a diagram illustrating a registration information table stored in a registration information storage unit.
  • FIG. 1 is a flowchart showing an example of an operation of an information processing device.
  • 1 is a flowchart showing an example of an operation of an information processing device.
  • 1A is a diagram illustrating an example in which the angle of view is wide
  • FIG. 1B is a diagram illustrating an example in which the angle of view is narrow.
  • 13A to 13C are diagrams illustrating an example in which a plurality of people are captured in the detection target area.
  • 13A and 13B are diagrams illustrating an example in which a plurality of people overlap in a detection target area.
  • 13A and 13B are diagrams illustrating examples of angles of a matching camera.
  • 13A and 13B are diagrams illustrating an example of the distance between a biosensor and a user.
  • 13A to 13C are diagrams illustrating an example of a change in a person area.
  • FIG. 2 is a block diagram illustrating a hardware configuration of an information processing device.
  • Biometric authentication is a technology that uses biometric characteristics such as fingerprints, faces, and veins to verify a person's identity. When identity verification is required, biometric data acquired by a biometric sensor is compared (matched) with pre-registered biometric data, and identity verification is performed by determining whether the degree of similarity is equal to or exceeds a threshold for determining identity. Biometric authentication is used in a variety of fields, including bank ATMs and entrance/exit management, and has recently begun to be used for cashless payments at supermarkets, convenience stores, and other facilities.
  • biometric authentication methods are "point” authentication methods that are performed at specific authentication spots, such as in front of an authentication machine.
  • point authentication
  • the authentication state is interrupted when the user leaves the authentication spot, and if the user wishes to receive a service again or is in a location where authentication is required multiple times, authentication must be performed each time. For this reason, there is a demand for continuous authentication technology that does not require multiple authentication actions, but rather allows the authenticated state to continue with a single authentication and allows the user to enjoy services.
  • FIG. 1 is a diagram for explaining the overview of continuous authentication technology.
  • Continuous authentication mainly involves the following three types of authentication:
  • the first step is authentication at check-in.
  • the user performs highly accurate authentication using palm vein authentication, facial recognition, etc. to identify the user's ID, and when authentication is successful, a photo of the user's appearance is taken with a camera to obtain visual characteristic information, which is then linked to the user's ID as registered characteristic information.
  • Next is line authentication. Using matching feature information obtained continuously over time using one or more cameras and registered feature information, an authentication state is maintained in which the person photographed by the camera has been authenticated as a successfully authenticated user.
  • Next is re-authentication. For example, if the user enters a blind spot of the camera due to another person or a pillar, causing the authentication state to be interrupted, re-authentication is performed.
  • FIGS 2(a) and 2(b) are diagrams illustrating the details of check-in.
  • highly accurate biometric data such as a face image captured by a face camera and a vein image captured by a vein sensor are obtained from the biometric sensor 201 and authentication is performed. This identifies the ID, name, etc. of the person checking in.
  • the camera 202 is used to acquire an image of the subject.
  • a person area Bbox: Bounding box
  • the person in the image is analyzed from the person area to extract appearance feature information for identifying the person.
  • feature information includes appearance information such as the color of clothing, physique, facial features, and behavioral features.
  • This feature information is linked to the subject's ID as registered feature information. This allows the registered appearance feature information to be used for subsequent continuous authentication.
  • behavioral features are, for example, skeletal information including the person's joints.
  • check-in is a process of registering that a user has entered a store by performing a specified process when the user enters the store.
  • information regarding the entry into the store may be read from an image attached to a specified position in the store (e.g., the entrance, etc.).
  • a specified position in the store e.g., the entrance, etc.
  • products for sale are placed for users who have checked in.
  • FIG. 3 is a diagram illustrating an example of a case where multiple people appear in an image captured by the camera 202 in FIG. 2. As illustrated in FIG. 3, three people appear in the image. In this case, three person regions are detected, and there is a risk that the appearance characteristic information of people other than the target person will be linked.
  • control method a control program, and an information processing device that can improve the accuracy of detecting a subject are described.
  • Fig. 4(a) is a block diagram illustrating an example of the overall configuration of a biometric authentication system 200 according to the first embodiment.
  • the biometric authentication system 200 includes an information processing device 100, a biometric sensor 110, a matching camera 120, a tracking camera 130, and the like. Each of these devices is connected via a telecommunications line.
  • the biometric sensor 110 is provided at the gate of the continuous authentication space, near the linking camera 120.
  • the biometric sensor 110 is a vein sensor that uses near-infrared rays, etc.
  • a fingerprint is used as the biometric modality
  • the biometric sensor 110 is a capacitive fingerprint sensor, etc.
  • the biometric sensor 110 is a face camera, etc.
  • the linking camera 120 is a camera installed at the gate of the continuous authentication space or the like, and is installed in a position that makes it easy to obtain information about the appearance characteristics of a person.
  • the tracking camera 130 is a camera for tracking a person in the continuous authentication space, and is installed on the ceiling or the like so that it is easy to track the person. There may be one tracking camera 130, or multiple tracking cameras.
  • FIG. 4(b) is a functional block diagram showing each function of the information processing device 100.
  • the information processing device 100 functions as an acquisition unit 11, a person detection unit 12, a number of people determination unit 13, a detection area control unit 14, a direction determination unit 15, a feature extraction unit 16, a registration information storage unit 17, a ReID processing unit 18, an authentication unit 19, a registration data storage unit 20, and the like.
  • the registration data storage unit 20 stores the registered biometric data of each user. This registered biometric data is registered in advance by each user along with their ID, name, and the like.
  • FIG. 5(a) is a diagram illustrating an example of a case where one user checks in.
  • FIG. 5(a) is a diagram showing the entrance area as viewed from above.
  • a biometric sensor 110 and a linking camera 120 are installed at the gate of the entrance area. The user enters the entrance area, approaches the gate, and stops at a position where the biometric sensor 110 can acquire the user's biometric data. This causes the user to stop at a position within the shooting range of the linking camera 120.
  • the biometric sensor 110 acquires the subject's biometric data for matching and sends it to the authentication unit 19.
  • the authentication unit 19 compares the biometric data for matching with each registered biometric data stored in the registered data storage unit 20.
  • FIG. 5(b) is a diagram illustrating a registered biometric data table stored in the registered data storage unit 20.
  • the registered biometric data table stores registered biometric data linked to each user's ID and name.
  • the authentication unit 19 calculates the similarity between the biometric data for matching and each registered biometric data.
  • the authentication unit 19 identifies the subject as a user of registered biometric data whose similarity exceeds a threshold value. This makes it possible to identify the subject's ID.
  • FIG. 5(c) is a diagram illustrating an example of an image acquired by the acquisition unit 11 from the matching camera 120.
  • the person detection unit 12 detects a person area in the image. Methods for detecting a person area include a method of detecting a person area by background difference, and a method of learning person characteristics in advance and detecting person characteristics from an input image. In FIG. 5(c), the area surrounded by a dotted line is the person area.
  • the number of people determination unit 13 determines the number of people appearing in the image. In the example of FIG. 5(c), the number of people determination unit 13 determines the number of people to be one.
  • the detection area control unit 14 does not narrow the detection target area for detecting a person area from the image.
  • the direction determination unit 15 determines whether or not the subject is facing a predetermined direction. For example, the direction determination unit 15 determines whether or not the subject is facing the matching camera 120.
  • FIGS. 6(a) to 6(c) are diagrams for explaining the direction in which the subject is facing. For example, as illustrated in FIG. 6(b), if the subject is carrying luggage on his/her back, the subject is facing away from the linking camera 120 if the luggage is captured in the image. Therefore, in this case, it is determined that the subject is not facing in a specified direction. Note that whether luggage is captured in the image can be determined by prior learning of the image, etc.
  • the subject is determined not to be facing the specified direction. Whether the subject is facing sideways with respect to the linking camera 120 can be determined by prior learning of the image, etc.
  • the subject faces the direction of the linking camera 120, it is determined that the subject faces a predetermined direction. Whether the subject faces the direction of the linking camera 120 can be determined by prior learning of the image, etc.
  • the direction determination unit 15 may determine whether the size of the person area in the image is equal to or larger than a threshold. For example, as illustrated in FIG. 6(d), if the size of the person area in the image is equal to or larger than a threshold, the size of the person area in the image is determined to be equal to or larger than the threshold. If the direction determination unit 15 does not determine that the size of the person area in the image is equal to or larger than the threshold, the direction determination unit 15 may treat the situation in the same way as if it was not determined that the subject is facing a specified direction.
  • FIG. 7(a) is a diagram illustrating a case where multiple users check in in sequence.
  • FIG. 7(a) is a diagram showing the entrance area as viewed from above.
  • a user enters the entrance area, approaches the gate, and stops at a position where the biometric sensor 110 can acquire the user's biometric data. This causes the user to stop in the shooting range of the linking camera 120.
  • people other than the target person will also be captured in the shooting range of the linking camera 120.
  • FIG. 7(b) multiple person areas surrounded by dotted lines can be detected.
  • the detection area control unit 14 narrows the detection target area for detecting a person area from the image horizontally, as shown in FIG. 7(c).
  • the area between the two hatched areas corresponds to the narrowed detection target area.
  • the vertical height of the detection target area may be the same as that of the original image.
  • the range in the image in which the user is located when facing the matching camera 120 is predetermined. Therefore, the detection target area is narrowed to the range in which the user faces the matching camera 120.
  • the person areas surrounded by dotted lines in the detection target area can be combined into one.
  • the direction determination unit 15 determines whether the person in the person area is facing a predetermined direction.
  • the feature extraction unit 16 extracts appearance feature information from the detected person area as registered feature information. Because the person area is accurately detected, the registered feature information can also be accurately extracted.
  • the registration information storage unit 17 stores the registration feature information extracted by the feature extraction unit 16 in association with the ID identified by the authentication unit 19.
  • FIG. 8 is a diagram illustrating an example of a registration information table stored in the registration information storage unit 17. As illustrated in FIG. 8, the registration information table stores the registration feature information in association with the user's ID, name, etc.
  • the acquisition unit 11 acquires an image from the tracking camera 130.
  • the person detection unit 12 detects a person area from the image.
  • the feature extraction unit 16 extracts appearance feature information for matching from the detected person area.
  • the ReID processing unit 18 matches the feature information for matching extracted by the feature extraction unit 16 with the registered feature information stored in the registered information storage unit 17. For example, the ReID processing unit 18 calculates the similarity between the feature information for matching extracted by the feature extraction unit 16 and each registered feature information.
  • the ReID processing unit 18 identifies the person detected from the tracking camera 130 as the user of the registered feature information whose similarity exceeds a threshold value.
  • the detection area control unit 14 may narrow the detection target area.
  • FIGS. 9 and 10 are flowcharts showing an example of the operation of the information processing device 100.
  • the flow of operation of the information processing device 100 will be described with reference to FIG. 9 and FIG. 10.
  • the flowcharts of FIG. 9 and FIG. 10 are executed at a predetermined cycle.
  • the flowcharts of FIG. 9 and FIG. 10 are executed independently for each of the linking cameras 120 and the tracking cameras 130.
  • the acquisition unit 11 acquires an image from either the linking camera 120 or the tracking camera 130 (step S1).
  • the person detection unit 12 detects a person area from the image acquired in step S1 (step S2).
  • the person detection unit 12 determines whether or not a person has been detected (step S3). If the determination in step S3 is "No,” the process is executed again from step S1.
  • step S3 If step S3 returns "Yes," the number-of-people determination unit 13 determines whether the number of people detected is two or more (step S4).
  • step S4 If step S4 is judged as "Yes”, the detection area control unit 14 narrows the target detection area (step S5), as shown in FIG. 7(c). If step S4 is judged as "No”, step S5 is not executed. Therefore, the target detection area is not narrowed.
  • step S6 the direction judgment unit 15 judges whether or not the person in the person area included in the target detection area is facing a predetermined direction (step S6).
  • the direction judgment unit 15 judges whether or not the person is facing the direction of the camera, as described in Figures 6(a) to 6(c). In this case, the direction judgment unit 15 may also judge whether or not the size of the person in the target detection area is equal to or larger than a threshold, as described in Figure 6(d).
  • step S6 If step S6 is judged as "No”, the process is executed again from step S1. Therefore, if the person in the target detection area is not facing a predetermined direction, feature information is not extracted. If step S6 is judged as "Yes”, the biometric sensor 110 detects biometric data for matching from the user checking in (step S7).
  • the authentication unit 19 judges whether or not the biometric sensor 110 has detected biometric data for matching (step S8). If the camera from which the image is acquired in step S1 is the tracking camera 130, the result in step S8 is "No.” If the camera from which the image is acquired in step S1 is the linking camera 120, the result in step S8 is "Yes.”
  • step S8 returns "Yes"
  • the authentication unit 19 performs authentication processing (step S9). Specifically, the authentication unit 19 compares the matching biometric data with each piece of registered biometric data stored in the registered data storage unit 20, and identifies the subject as a user of the registered biometric data whose similarity exceeds a threshold value.
  • the feature extraction unit 16 extracts appearance feature information from the person area (step S10).
  • the registration information storage unit 17 associates the characteristic information with the ID identified in step S9 and stores it as registered characteristic information (step S11). After step S11 is executed, execution of the flowchart ends.
  • step S8 If the result of step S8 is "No", the feature extraction unit 16 extracts and stores appearance matching feature information from the person area (step S12).
  • the ReID processing unit 18 compares the matching feature information extracted in step S12 with each piece of registered feature information stored in the registered information storage unit 17, and calculates the similarity (step S13).
  • the ReID processing unit 18 identifies the person in the person area as the person whose registered characteristic information has the highest similarity among the similarities calculated in step S13 (step S14).
  • the ReID processing unit 18 determines whether a person whose ID can be identified has been absent for a period of time equal to or longer than the threshold time (step S15).
  • step S15 If the result of step S15 is "No", the process is executed again from step S12. If the result of step S15 is "Yes", the ReID processing unit 18 erases the ID that had been specified up to that point (step S16).
  • the target detection area for detecting the target in the image is controlled. For example, the target detection area for the image is narrowed. This increases the accuracy of detecting the person area even if multiple people are captured in the image. As a result, the accuracy of detecting the target is improved.
  • person characteristic information from this person and linking it to an identifier such as the ID of the person to be authenticated, it is possible to achieve highly accurate continuous authentication. By extracting person characteristic information when it is determined that the person to be authenticated is facing a specified direction, the accuracy of subsequent continuous authentication can be improved.
  • the extent to which the detection target area is narrowed may be determined according to the angle of view and focal length of the linking camera 120. For example, when a person with a standard build is standing at a position of the focal length of the linking camera 120, it is preferable to narrow the detection target area to an extent that the entire body of one person is captured.
  • FIG. 11(a) is a diagram illustrating an example when the angle of view is wide.
  • FIG. 11(b) is a diagram illustrating an example when the angle of view is narrow.
  • the detection target area is set to be wide.
  • the feature extraction unit 16 may extract feature information from the tallest person region.
  • the angle of the linking camera 120 is adjusted so that one user is captured as much as possible. For example, as shown in FIG. 14(b), if the capturing range of the linking camera 120 is a wide space in which multiple users can be positioned, there is a high possibility that multiple users will be captured in the image captured by the linking camera 120.
  • the angle of the linking camera 120 is determined so that an image of a space that is wider in the horizontal direction can be captured of a space that is narrower in the horizontal direction.
  • the number of people appearing in the image acquired by the linking camera 120 can be reduced. This makes it possible to reduce the number of people appearing in the target detection area, thereby enabling accurate extraction of information about the user's appearance characteristics.
  • the narrow space is narrow enough to accommodate only one person.
  • FIG. 15(a) in the case of face authentication using a face camera, the distance between the user and the biometric sensor 110 used as the face camera is long, which may result in a large variation in the user's position. Therefore, there is a risk of variation in the accuracy when feature information is extracted using the matching camera 120.
  • FIG. 15(b) in the case of vein authentication or fingerprint authentication, the distance between the user and the biometric sensor 110 is short, which reduces the variation in the user's position. Therefore, it is possible to reduce the variation in accuracy when feature information is extracted using the matching camera 120. Therefore, it is preferable to use vein authentication or fingerprint authentication.
  • the person area may be detected using time-series images acquired by the linking camera 120.
  • a person can be tracked by tracking.
  • the person area of the target person can be accurately detected.
  • the arrows indicate a predetermined time.
  • people can be tracked by using time-series images.
  • the movement direction of each person can be detected.
  • the movement direction of each person can be detected.
  • the movement direction of each person can be detected.
  • the leading person Since the direction in which the leading person approaches the linking camera 120 is known in advance, it is possible to determine whether or not the person in which the person is the leading person. Or, for example, the leading person stops moving and becomes stationary near the biosensor 110. Therefore, it is possible to determine whether or not the person in the person area with the smallest amount of movement per unit time is the leading person. Feature information may be extracted from the person area determined to be the leading person.
  • the biometric authentication system 200 can analyze the behavior of a person who has checked in by using an image captured by a camera.
  • the facility is a railroad facility, an airport, a store, a residence, a hotel, a castle, an amusement park, etc.
  • the gate located at the facility is located at the entrance of a store, a residence, a hotel, a castle, an amusement park, a railroad facility, an airport boarding gate, etc.
  • the check-in target is a railway facility or an airport.
  • the gate is located at the boarding gate of the railway facility or airport.
  • the information processing device 100 determines that authentication using the person's biometric information has been successful.
  • the check-in target is a store.
  • the gate is located at the entrance of the store.
  • the information processing device 100 determines that authentication using the biometric information of the person at the check-in has been successful.
  • the information processing device 100 acquires biometric information of a person passing through a gate located at a specified position within the store. Specifically, the information processing device 100 acquires a vein image, etc. acquired by a vein sensor mounted on a gate located at the entrance of the store from the biometric sensor 110, and performs authentication. At this time, the information processing device 100 identifies the user's ID, name, etc. from the biometric information.
  • the biometric sensor 110 and the linking camera 120 are mounted on a gate placed at a predetermined position in the facility, and detect the biometric information of a person passing through the gate. At this time, the information processing device 100 can also obtain the biometric information of the person using the linking camera 120.
  • the information processing device 100 when authentication based on the person's biometric information is successful, the information processing device 100 generates characteristic information of the person by analyzing an image including the person passing through the gate. Specifically, the information processing device 100 performs authentication by identifying the ID and name of the user of the person passing through the gate. Then, when authentication is successful, the information processing device 100 generates characteristic information of the person by analyzing an image including the person passing through the gate.
  • the characteristic information of the person is skeletal information including the person's joints.
  • the information processing device 100 determines whether the person to be authenticated is facing the camera for acquiring the image in the direction of travel of the gate passage, and generates characteristic information from the target detection area if it is determined that the person to be authenticated is facing the direction of travel of the gate passage. Then, the information processing device 100 stores the ID and name of the user who checks in and the generated characteristic information in a corresponding manner in the storage unit.
  • the information processing device 100 uses the feature information stored in the storage unit to track the person moving within the store while identifying the user's ID and name. For example, the information processing device 100 performs gait authentication using skeletal information including the person's joints using an existing skeletal estimation algorithm. The information processing device 100 tracks the person by comparing whether the person who passed through the gate is the same as the person in the image captured by the tracking camera 130, and identifies the person's trajectory from when they entered the store to when they left.
  • the existing skeletal estimation algorithm is, for example, a skeletal estimation algorithm that uses deep learning, such as HumanPoseEstimation, such as DeepPose and OpenPose.
  • the information processing device 100 identifies the product that the person has acquired from among multiple products placed in the store. Specifically, the information processing device 100 uses skeletal information including the person's joints to determine whether or not the person is holding a product placed in the store, thereby identifying the product that the person has acquired from among multiple products.
  • the information processing device 100 registers in the memory unit the person's identification information and the products acquired by the person in association with each other.
  • the information processing device 100 also generates information that associates the person's trajectory from the time the person enters the store to the time the person leaves the store with the user's ID, name, etc.
  • the information processing device 100 can identify the items purchased by the person in the store by generating information that associates the ID and name of the user who checks in, the person's trajectory in the store, and the products acquired by the person. This makes it possible to analyze the purchasing behavior of the person as they move around the store after they check in.
  • the information processing device 100 includes a CPU 101, a RAM 102, a storage device 103, a communication device 104, and the like. These devices are connected by a bus or the like.
  • the CPU (Central Processing Unit) 101 is a central processing unit.
  • the RAM (Random Access Memory) 102 is a volatile memory that temporarily stores programs executed by the CPU 101 and data processed by the CPU 101.
  • the storage device 103 is a non-volatile storage device. For example, a ROM (Read Only Memory), a solid state drive (SSD) such as a flash memory, or a hard disk driven by a hard disk drive can be used as the storage device 103.
  • ROM Read Only Memory
  • SSD solid state drive
  • each part of the information processing device 100 are realized by the CPU 101 executing the control program stored in the storage device 103.
  • the functions of each part of the information processing device 100 may be configured by a dedicated circuit or the like.
  • the communication device 104 is an interface to an electric communication line.
  • the user's ID is identified using the user's biometric information acquired from the biometric sensor 110, but this is not limited to the above.
  • the user's ID may be identified based on whether or not a password entered by the user using an input device matches a password that has been registered in advance. Even in this case, when the user enters information using an input device, visual characteristic information can be extracted from the image acquired by the matching camera 120.
  • the detection area control unit 14 when the detection area control unit 14 detects multiple people from an image including a person to be authenticated who has been successfully authenticated using identification information, the detection area control unit 14 is an example of a detection area control unit that controls a target detection area for detecting the target in the image.
  • the feature extraction unit 16 is an example of a feature extraction unit that extracts person feature information from the controlled target detection area.
  • the registration information storage unit 17 is an example of a registration information storage unit that associates the feature information with an identifier of the person to be authenticated as registered feature information and stores it.
  • the direction determination unit 15 is an example of a direction determination unit that determines whether the person to be authenticated is facing a predetermined direction relative to the camera that captures the image.
  • Registration data storage unit 100 Information processing device 110 Biometric sensor 120 Linking camera 130 Tracking camera 200 Biometric authentication system

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Collating Specific Patterns (AREA)
PCT/JP2023/019350 2023-05-24 2023-05-24 制御方法、制御プログラム、および情報処理装置 Ceased WO2024241536A1 (ja)

Priority Applications (5)

Application Number Priority Date Filing Date Title
PCT/JP2023/019350 WO2024241536A1 (ja) 2023-05-24 2023-05-24 制御方法、制御プログラム、および情報処理装置
EP23938499.3A EP4723030A1 (en) 2023-05-24 2023-05-24 Control method, control program, and information processing device
CN202380098514.2A CN121175716A (zh) 2023-05-24 2023-05-24 控制方法、控制程序以及信息处理装置
JP2025521729A JPWO2024241536A1 (https=) 2023-05-24 2023-05-24
US19/392,006 US20260073032A1 (en) 2023-05-24 2025-11-17 Control method, non-transitory computer-readable recording medium, and information processing device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2023/019350 WO2024241536A1 (ja) 2023-05-24 2023-05-24 制御方法、制御プログラム、および情報処理装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US19/392,006 Continuation US20260073032A1 (en) 2023-05-24 2025-11-17 Control method, non-transitory computer-readable recording medium, and information processing device

Publications (1)

Publication Number Publication Date
WO2024241536A1 true WO2024241536A1 (ja) 2024-11-28

Family

ID=93589742

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/019350 Ceased WO2024241536A1 (ja) 2023-05-24 2023-05-24 制御方法、制御プログラム、および情報処理装置

Country Status (5)

Country Link
US (1) US20260073032A1 (https=)
EP (1) EP4723030A1 (https=)
JP (1) JPWO2024241536A1 (https=)
CN (1) CN121175716A (https=)
WO (1) WO2024241536A1 (https=)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007249466A (ja) * 2006-03-15 2007-09-27 Konica Minolta Holdings Inc 認証システム、コントローラおよび制御方法
JP2008209969A (ja) * 2007-02-23 2008-09-11 Aisin Seiki Co Ltd 顔特徴点検出装置、顔特徴点検出方法及びプログラム
JP2015226294A (ja) * 2014-05-30 2015-12-14 シャープ株式会社 セキュリティ機能付き装置
WO2020070821A1 (ja) * 2018-10-03 2020-04-09 富士通株式会社 生体認証装置、生体認証方法、及び生体認証プログラム
WO2020115890A1 (ja) * 2018-12-07 2020-06-11 日本電気株式会社 情報処理システム、情報処理装置、情報処理方法、およびプログラム
JP2021531539A (ja) 2018-05-25 2021-11-18 コーニンクレッカ フィリップス エヌ ヴェKoninklijke Philips N.V. 個人識別システムおよび方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007249466A (ja) * 2006-03-15 2007-09-27 Konica Minolta Holdings Inc 認証システム、コントローラおよび制御方法
JP2008209969A (ja) * 2007-02-23 2008-09-11 Aisin Seiki Co Ltd 顔特徴点検出装置、顔特徴点検出方法及びプログラム
JP2015226294A (ja) * 2014-05-30 2015-12-14 シャープ株式会社 セキュリティ機能付き装置
JP2021531539A (ja) 2018-05-25 2021-11-18 コーニンクレッカ フィリップス エヌ ヴェKoninklijke Philips N.V. 個人識別システムおよび方法
WO2020070821A1 (ja) * 2018-10-03 2020-04-09 富士通株式会社 生体認証装置、生体認証方法、及び生体認証プログラム
WO2020115890A1 (ja) * 2018-12-07 2020-06-11 日本電気株式会社 情報処理システム、情報処理装置、情報処理方法、およびプログラム

Also Published As

Publication number Publication date
EP4723030A1 (en) 2026-04-08
US20260073032A1 (en) 2026-03-12
CN121175716A (zh) 2025-12-19
JPWO2024241536A1 (https=) 2024-11-28

Similar Documents

Publication Publication Date Title
KR102455633B1 (ko) 라이브니스 검사 방법 및 장치
US7885433B2 (en) Biometrics authentication method and biometrics authentication system
US20230342451A1 (en) Information processing device
KR101291899B1 (ko) 속눈썹 분석에 의한 신원확인방법 및 그를 이용한 신원확인용 정보 취득 장치
JP4675660B2 (ja) 複数同時バイオメトリクス認証装置
WO2020065954A1 (ja) 認証装置、認証方法および記憶媒体
JP7468779B2 (ja) 情報処理装置、情報処理方法、及び記憶媒体
JP2001256496A (ja) 顔画像認識装置及び顔画像認識方法
JP2020064531A5 (https=)
JP2022155061A (ja) 画像処理装置、画像処理方法およびプログラム
EP4167112A1 (en) Authentication method, information processing device, and authentication program
JP2005242432A (ja) 顔認証システム及びその処理方法及びそのプログラム
EP4723030A1 (en) Control method, control program, and information processing device
US20260064820A1 (en) Determination method, control method, non-transitory computer-readable recording medium, and information processing device
JP2010003010A (ja) 顔認証装置および顔認証方法
JP7338801B2 (ja) 情報処理装置、情報処理方法、および記憶媒体
JP2007249298A (ja) 顔認証装置および顔認証方法
KR20140060081A (ko) 원거리 얼굴인식 기반 스피드 게이트 통제 방법 및 장치
EP4693231A1 (en) Determination method, determination program, and information processing device
CN114930323A (zh) 认证方法、信息处理装置以及认证程序
US20230013232A1 (en) Control method, storage medium, and information processing device
JP2019159372A (ja) 認証装置、認証システム、認証方法、コンピュータプログラム、及び記憶媒体
JP7279774B2 (ja) 情報処理装置
WO2024236765A1 (ja) 判定方法、判定プログラム、および情報処理装置
JPWO2024241536A5 (https=)

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23938499

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2025521729

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2023938499

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2023938499

Country of ref document: EP

Effective date: 20260102

ENP Entry into the national phase

Ref document number: 2023938499

Country of ref document: EP

Effective date: 20260102

ENP Entry into the national phase

Ref document number: 2023938499

Country of ref document: EP

Effective date: 20260102

ENP Entry into the national phase

Ref document number: 2023938499

Country of ref document: EP

Effective date: 20260102

ENP Entry into the national phase

Ref document number: 2023938499

Country of ref document: EP

Effective date: 20260102

ENP Entry into the national phase

Ref document number: 2023938499

Country of ref document: EP

Effective date: 20260102

WWP Wipo information: published in national office

Ref document number: 2023938499

Country of ref document: EP