Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
  • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
  • G06V40/172—Classification, e.g. identification
  • G06V40/173—Classification, e.g. identification face re-identification, e.g. recognising unknown faces across different face tracks
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/50—Context or environment of the image
  • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects

Definitions

• This disclosure relates to the technical fields of information processing systems, information processing methods, and recording media.
• Patent Document 1 As an example of this type of system, technology has been proposed to realize a gateless bicycle parking lot management system by using wireless tags in a bicycle parking lot with separate entrance and exit passages (see Patent Document 1).
• the objective of this disclosure is to provide an information processing system, an information processing method, and a recording medium that aim to improve upon the technology described in prior art documents.
• One aspect of the information processing method involves acquiring at least one of a first image including at least a portion of a first lane through which a first authenticated subject passes as the subject travels from a first area to a second area, and a second image including at least a portion of a second lane through which a second authenticated subject passes as the subject travels from the second area to the first area, and performing at least one of authentication of the first authenticated subject using the first image and authentication of the second authenticated subject using the second image.
• a computer program is recorded to cause a computer to execute an information processing method for acquiring at least one of a first image including at least a portion of a first lane through which a first authenticated subject passes from a first area to a second area, and a second image including at least a portion of a second lane through which a second authenticated subject passes from the second area to the first area, and performing at least one of authenticating the first authenticated subject using the first image and authenticating the second authenticated subject using the second image.
• FIG. 1 is a diagram showing a configuration of an information processing system according to a first embodiment.
• FIG. 2 is a diagram illustrating an example of an authentication area.
• 4 is a flowchart showing an operation of the information processing device according to the first embodiment.
• FIG. 1 is a diagram showing an example of a slope road.
• FIG. 13 is a diagram showing another example of a slope road.
• FIG. 13 is a diagram showing another example of a slope road.
• FIG. 11 is a diagram showing the configuration of a ticket gate system according to a second embodiment.
• FIG. 11 is a block diagram showing a configuration of an information processing device according to a second embodiment.
• FIG. 11 is a diagram for explaining a target tracking process.
• FIG. 11 is a diagram illustrating an example of an ID correspondence table.
• FIG. 13 is a diagram showing another example of the authentication area.
• FIG. 13 is a diagram showing another example of the authentication area.
• the information processing system 1 includes an information processing device 10 and cameras CAM1 and CAM2. Cameras CAM1 and CAM2 will be described with reference to FIG. 2.
• a space e.g., a passage
• connecting the first area and the second area is provided between the first area and the second area.
• the first area may be, for example, either inside or outside a railway station.
• the second area may be the other of inside or outside the railway station.
• the first area may be, for example, either inside or outside an amusement park.
• the second area may be the other of inside or outside the amusement park.
• the first area may be, for example, an area set to a first security level.
• the second area may be an area set to a second security level that is different from the first security level.
• a first lane is set through which objects travelling from the first area to the second area pass
• a second lane is set through which objects travelling from the second area to the first area pass.
• a member such as a fence that physically prevents objects from passing through may be provided between the first and second lanes.
• object is not limited to humans, but includes animals such as dogs and snakes, and moving objects such as robots.
• Camera CAM1 captures an image of at least a portion of the first lane. As a result, camera CAM1 generates a first image that includes at least a portion of the first lane. Camera CAM2 captures an image of at least a portion of the second lane. As a result, camera CAM2 generates a second image that includes at least a portion of the second lane. Note that, in addition to camera CAM1, information processing system 1 may be provided with another camera that captures an image of at least a portion of the first lane. In addition to camera CAM2, information processing system 1 may be provided with another camera that captures an image of at least a portion of the second lane. In other words, information processing system 1 may be provided with three or more cameras.
• the first lane and the second lane do not have to be provided in the same space (e.g., an aisle).
• the first lane may be provided in a first space connecting the first area and the second area
• the second lane may be provided in a second space different from the first space connecting the first area and the second area.
• a third lane may be provided in addition to the first lane and the second lane. That is, three or more lanes may be provided.
• the third lane may be provided in a space connecting the first area and a third area different from the second area, or in a space connecting the second area and a fourth area different from the first area.
• the information processing device 10 has an image acquisition unit 11, an authentication unit 12, and an information database 13 (hereinafter referred to as "information DB 13" where appropriate).
• Information DB 13 stores, for example, information related to registered individuals who are registered as users of the information processing system 1.
• the information related to registered individuals includes information (e.g., features) used in biometric authentication, which will be described later.
• the image acquisition unit 11 of the information processing device 10 acquires a first image and a second image.
• the authentication unit 12 uses the first image to perform biometric authentication of a subject traveling along the first lane from the first area to the second area (in other words, the subject to be authenticated).
• the authentication unit 12 uses the second image to perform biometric authentication of a subject traveling along the second lane from the second area to the first area (in other words, the subject to be authenticated).
• the biometric authentication may be at least one of face authentication and iris authentication.
• the authentication unit 12 may detect the face of a subject included in the first image (in other words, captured in the first image).
• the authentication unit 12 may calculate facial feature amounts indicating the characteristics of the detected face.
• the authentication unit 12 may calculate a matching score from the calculated facial feature amounts and the facial feature amounts registered in the information DB 13 (corresponding to the above-mentioned "information used in biometric authentication").
• the authentication unit 12 may determine (identify) whether the subject included in the first image is a registered person by comparing the matching score with a predetermined threshold value.
• the authentication unit 12 may detect the face of a subject included in the second image (in other words, captured in the second image).
• the authentication unit 12 may calculate facial feature amounts indicating the characteristics of the detected face.
• the authentication unit 12 may calculate a matching score from the calculated facial feature amounts and the facial feature amounts registered in the information DB 13.
• the authentication unit 12 may determine (identify) whether or not the subject included in the second image is a registered person by comparing the matching score with a predetermined threshold value.
• the authentication unit 12 may detect the eyes of a target included in the first image (in other words, reflected in the first image).
• the authentication unit 12 may calculate an iris feature amount indicating the characteristics of the iris of the detected eye.
• the authentication unit 12 may calculate a matching score from the calculated iris feature amount and the iris feature amount registered in the information DB 13 (corresponding to the above-mentioned "information used in biometric authentication").
• the authentication unit 12 may determine (identify) whether or not the target included in the first image is a registered person by comparing the matching score with a predetermined threshold.
• the authentication unit 12 may determine (identify) whether or not the target is a registered person by using both a facial feature amount indicating the facial characteristics of a target included in the first image and an iris feature amount indicating the iris characteristics of the eye of the target. In other words, the authentication unit 12 may perform multimodal authentication.
• the authentication unit 12 may detect the eyes of a target included in the second image (in other words, reflected in the second image).
• the authentication unit 12 may calculate an iris feature amount indicating the characteristics of the iris of the detected eye.
• the authentication unit 12 may calculate a matching score from the calculated iris feature amount and the iris feature amount registered in the information DB 13.
• the authentication unit 12 may determine (identify) whether or not the target included in the second image is a registered person by comparing the matching score with a predetermined threshold.
• the authentication unit 12 may determine (identify) whether or not the target is a registered person by using both a facial feature amount indicating the facial characteristics of a target included in the second image and an iris feature amount indicating the iris characteristics of the eye of the target. In other words, the authentication unit 12 may perform multimodal authentication.
• the authentication unit 12 may perform biometric authentication on a subject (in other words, a subject to be authenticated) traveling along the first lane from the first area to the second area while the subject is passing through the first lane. Similarly, the authentication unit 12 may perform biometric authentication on a subject (in other words, a subject to be authenticated) traveling along the second lane from the second area to the first area while the subject is passing through the second lane. That is, the authentication unit 12 may perform walk-through biometric authentication. For this reason, the first lane and the second lane may be referred to as authentication areas.
• the length of the first lane along the traveling direction of a subject passing through the first lane may be set so that the time required for the subject to pass through the first lane is sufficiently longer than the time required for biometric authentication.
• the length of the second lane along the traveling direction of a subject passing through the second lane may be set so that the time required for the subject to pass through the second lane is sufficiently longer than the time required for biometric authentication.
• the image acquisition unit 11 acquires a first image and a second image (step S101).
• the authentication unit 12 performs biometric authentication using the first image and biometric authentication using the second image (step S102).
• the information processing system 1 may include one of the cameras CAM1 and CAM2, and the information processing device 10.
• the information processing system 1 may only perform biometric authentication of a subject passing through one of the first lane and the second lane.
• the image acquisition unit 11 may only acquire one of the first image and the second image.
• the authentication unit 12 may only perform biometric authentication using the first image and biometric authentication using the second image. In this case, the biometric authentication of a subject passing through the other of the first lane and the second lane may be performed by another information processing system (not shown).
• the image acquisition unit 11 may acquire at least one of a first image and a second image.
• the authentication unit 12 may perform at least one of biometric authentication using the first image and biometric authentication using the second image.
• At least one of a first image including at least a portion of a first lane through which a first authenticated subject passes from a first area to a second area, and a second image including at least a portion of a second lane through which a second authenticated subject passes from a second area to the first area, can be obtained, and an information processing method is performed to perform at least one of authentication of the first authenticated subject using the first image and authentication of the second authenticated subject using the second image.
• the authentication device 1 described above may be realized by a computer reading a computer program recorded on a recording medium.
• the recording medium can be said to have recorded thereon a computer program for causing a computer to execute an information processing method for acquiring at least one of a first image including at least a portion of a first lane through which a first authenticated subject passes from a first area to a second area, and a second image including at least a portion of a second lane through which a second authenticated subject passes from the second area to the first area, and for authenticating at least one of the first authenticated subject using the first image and the second authenticated subject using the second image.
• the first lane and the second lane are not set in the space (e.g., a passage) connecting the first area and the second area, an object moving from the first area to the second area and an object moving from the second area to the first area will pass through the space in a disorderly manner.
• the imaging range of the camera e.g., at least one of the cameras CAM1 and CAM2
• the multiple images captured by the camera may include images that are not suitable for use in biometric authentication.
• an object that has not been biometrically authenticated by the information processing system 1 may enter the first area or the second area.
• the time required for biometric authentication of one object may be relatively long.
• camera CAM1 can be positioned in advance so that it can capture an image of the front of an object passing through the first lane.
• camera CAM2 can be positioned in advance so that it can capture an image of the front of an object passing through the second lane.
• the authentication unit 12 can relatively efficiently detect the face or eyes of the target (in other words, the target to be authenticated) from each of the image captured by camera CAM1 (i.e., the first image) and the image captured by camera CAM2 (i.e., the second image). As a result, the time required for biometric authentication of one target can be shortened. In addition, it is possible to prevent targets who have not been biometrically authenticated by the information processing system 1 from entering the first area or the second area.
• At least one of the first lane and the second lane shown in Fig. 2 may include a sloped road that slopes from one side of the first area and the second area to the other side of the first area and the second area.
• the sloped road is not limited to a slope, but is a concept that includes a structure that enables the movement of an object between two points with a difference in height, such as a staircase, an escalator, and a conveyor belt.
• the first lane may include a sloped road that descends from the first area side toward the second area side.
• the camera CAM1 may capture an image of the sloped road as at least a part of the first lane.
• the camera CAM1 may capture an image of the sloped road to generate an image (e.g., the first image) that includes an object passing through the sloped road.
• the camera CAM1 may capture an image of the sloped road to generate an image (e.g., the first image) that includes an object before entering the sloped road.
• the camera CAM1 may capture an image of the sloped road to generate an image (e.g., the first image) that includes an object after passing through the sloped road.
• the second lane may include a sloped road that descends from the second area side toward the first area side.
• the camera CAM2 may capture an image of the sloped road as at least a part of the second lane.
• the camera CAM2 may capture an image of the sloped road to generate an image (e.g., the second image) that includes an object passing through the sloped road.
• the camera CAM2 may capture an image of the sloped road to generate an image (e.g., the second image) that includes an object before entering the sloped road.
• the camera CAM2 may capture an image of the sloped road to generate an image (e.g., the second image) that includes an object after passing through the sloped road.
• the sloped road may include an escalator E1 as shown in FIG. 4.
• the camera C1 may be positioned so as to be able to capture an image of the escalator E1.
• the escalator E1 can regulate the direction of travel of an object by its operation. This makes it possible to prevent the occurrence of an object traveling in the wrong direction in a lane (e.g., at least one of the first lane and the second lane) (in other words, a person traveling in the wrong direction).
• the camera C1 is an example of the camera CAM1.
• the camera C1 is an example of the camera CAM2.
• the arrow in FIG. 4 indicates the direction of travel of the escalator E1.
• the first lane may include a slope that rises from the first area side toward the second area side.
• Camera CAM1 may capture a predetermined range including the end of the slope as at least a portion of the first lane.
• the second lane may include a slope that rises from the second area side toward the first area side.
• Camera CAM2 may capture a predetermined range including the end of the slope as at least a portion of the second lane.
• a subject climbing a sloping road often turns his face toward the road ahead near the end of the sloping road to check his path. For this reason, if a camera (e.g., at least one of cameras CAM1 and CAM2) is positioned so that it can capture an image of a specified range that includes the end of the sloping road, it can be expected that the camera will capture an appropriate image of the face of the subject (in other words, the subject to be authenticated).
• a camera e.g., at least one of cameras CAM1 and CAM2
• the sloped road may include the escalator E2 shown in FIG. 5.
• the camera C2 may be positioned so as to be able to capture an image of a predetermined range including the end of the escalator E2.
• the escalator E2 can regulate the travel direction of an object by its operation. This makes it possible to prevent the occurrence of an object traveling in the wrong direction in a lane (e.g., at least one of the first lane and the second lane) (in other words, a person traveling in the wrong direction).
• the camera C2 is an example of the camera CAM1.
• the camera C2 is an example of the camera CAM2. Note that the arrow in FIG. 5 indicates the direction of travel of the escalator E2.
• the first lane may include a first slope that descends from the first area side toward the second area side, and a second slope that ascends from the first area side toward the second area side.
• the first lane may include an escalator E1 as the first slope path, and an escalator E2 as the second slope path.
• camera C1 which is an example of camera CAM1
• Camera C2 which is another example of camera CAM1
• the second lane may include a third slope that descends from the second area side toward the first area side, and a fourth slope that ascends from the second area side toward the first area side.
• the second lane may include an escalator E1 as the third slope path, and an escalator E2 as the fourth slope path.
• camera C1 which is an example of camera CAM2
• Camera C2 which is another example of camera CAM2
• a display for digital signage may be placed near at least one of the cameras C1 and C2.
• the face of a subject passing through the first lane or the second lane can be directed toward the camera C1 or C2.
• Second Embodiment A second embodiment of the information processing system, the information processing method, and the recording medium will be described with reference to Figures 7 to 10.
• the second embodiment of the information processing system, the information processing method, and the recording medium will be described using a ticket gate system 2.
• the ticket gate system 2 corresponds to an example of an information processing system. Note that, with regard to the second embodiment, descriptions that overlap with the description of the first embodiment described above will be omitted as appropriate.
• the ticket gate system 2 includes an information processing device 20, a station server 41, and cameras CAM1 and CAM2 installed at a railway station A, and an information processing device 20a, a station server 42, and cameras CAM1a and CAM2a installed at a railway station B, a center server 50, and a face information database 60 (hereinafter, appropriately referred to as "face information DB60").
• the cameras CAM1a and CAM2a at station B correspond to the cameras CAM1 and CAM2 at station A, respectively.
• the information processing devices 20 and 20a correspond to the information processing device 10 according to the first embodiment described above.
• other information processing devices may be installed at station A. That is, multiple information processing devices may be installed at station A.
• other information processing devices may be installed at station B. That is, multiple information processing devices may be installed at station B.
• the information processing device 20 and the station server 41 may be connected via a narrow area network such as a Local Area Network (LAN).
• a narrow area network such as a Local Area Network (LAN).
• the information processing device 20a and the station server 42 may be connected via a narrow area network.
• the station servers 41 and 42, the center server 50, and the face information DB 60 are connected via a wide area network NW such as the Internet.
• the ticket gate area of Station A has a first lane and a second lane as shown in FIG. 2.
• the first area shown in FIG. 2 may be either inside or outside the premises of Station A.
• the second area shown in FIG. 2 may be the other of inside or outside the premises of Station A.
• the first area is outside the premises of Station A, and the second area is inside the premises of Station A.
• an object entering the premises of Station A from outside the premises of Station A will pass through the first lane.
• An object exiting the premises of Station A from inside the premises of Station A will pass through the second lane.
• Camera CAM1 captures an image of at least a portion of the first lane. As a result, camera CAM1 generates a first image that includes at least a portion of the first lane. Camera CAM2 captures an image of at least a portion of the second lane. As a result, camera CAM2 generates a second image that includes at least a portion of the second lane.
• the information processing device 20 may perform facial recognition based on the image captured by the camera CAM1 (i.e., the first image) to identify the target entering station A.
• the information processing device 20 links identification information indicating a registered person corresponding to the target identified by facial recognition with information indicating that the target has entered station A, and transmits this to the station server 41.
• the information processing device 20 may perform facial recognition based on the image captured by the camera CAM2 (i.e., the second image) to identify the target exiting station A.
• the information processing device 20 links identification information indicating a registered person corresponding to the target identified by facial recognition with information indicating that the target has exited station A, and transmits this to the station server 41.
• a registrant is an individual who is registered in face information DB 60 as a user of the ticket gate system 2.
• Information relating to each of a plurality of registrants is registered in face information DB 60.
• information relating to a registrant may include, for example, identification information for identifying the registrant and face information used in face authentication (for example, facial feature amounts indicating the facial features of the registrant).
• face information DB 60 information relating to a registrant may also include terminal information (for example, a telephone number, email address, etc.) relating to a terminal device (for example, a smartphone) possessed by the registrant.
• the station server 41 transmits the information transmitted from the information processing device 20 (i.e., the identification information indicating the registered person corresponding to the target identified by face recognition, and the information indicating that the target has entered Station A or the information indicating that the target has exited Station A) to the center server 50 at a predetermined communication cycle.
• the information processing device 20 i.e., the identification information indicating the registered person corresponding to the target identified by face recognition, and the information indicating that the target has entered Station A or the information indicating that the target has exited Station A
• the ticket gate area of Station B has a first lane and a second lane as shown in FIG. 2.
• the first area shown in FIG. 2 may be either inside or outside the premises of Station B.
• the second area shown in FIG. 2 may be the other of inside or outside the premises of Station B.
• the first area is outside the premises of Station B, and the second area is inside the premises of Station B.
• an object entering the premises of Station B from outside the premises of Station B will pass through the first lane.
• An object exiting the premises of Station B from inside the premises of Station B will pass through the second lane.
• Camera CAM1a captures an image of at least a portion of the first lane. As a result, camera CAM1a generates a third image that includes at least a portion of the first lane. Camera CAM2a captures an image of at least a portion of the second lane. As a result, camera CAM2a generates a fourth image that includes at least a portion of the second lane.
• the information processing device 20a may perform facial recognition based on the image captured by the camera CAM1a (i.e., the third image) to identify the target entering station B.
• the information processing device 20a links identification information indicating a registered person corresponding to the target identified by facial recognition with information indicating that the target has entered station B, and transmits this to the station server 42.
• the information processing device 20a may perform facial recognition based on the image captured by the camera CAM2a (i.e., the fourth image) to identify the target exiting station B.
• the information processing device 20a links identification information indicating a registered person corresponding to the target identified by facial recognition with information indicating that the target has exited station B, and transmits this to the station server 42.
• the station server 42 transmits the information transmitted from the information processing device 20a (i.e., the identification information indicating the registered person corresponding to the target identified by face recognition, and the information indicating that the target has entered Station B or the information indicating that the target has exited Station B) to the center server 50 at a predetermined communication cycle.
• the information processing device 20a i.e., the identification information indicating the registered person corresponding to the target identified by face recognition, and the information indicating that the target has entered Station B or the information indicating that the target has exited Station B
• the center server 50 may calculate the railway fare for each registered person based on information (e.g., identification information indicating a registered person identified by facial recognition, and information indicating entry and exit of a station) transmitted from each of the multiple station servers including the station servers 41 and 42.
• the center server 50 may pay the railway fare for each registered person using payment information related to each registered person.
• the information processing device 20 includes a calculation device 21, a storage device 22, and a communication device 23.
• the authentication device 20 may further include an input device 24 and an output device 25.
• the authentication device 20 may include a face information database 26 (hereinafter, appropriately referred to as "face information DB 26"). It is to be noted that the authentication device 20 does not need to include at least one of the input device 24 and the output device 25.
• face information DB 26 may be connected via a data bus 27.
• Face information DB26 may store the same information as that registered in face information DB60. Face information DB26 may be updated at a predetermined interval so that the information registered in face information DB26 is the same as the information registered in face information DB60.
• the computing device 21 may include, for example, at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), a TPU (Tensor Processing Unit), and a quantum processor.
• a CPU Central Processing Unit
• GPU Graphics Processing Unit
• FPGA Field Programmable Gate Array
• TPU Torsor Processing Unit
• quantum processor a quantum processor
• the storage device 22 may include, for example, at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, an optical magnetic disk device, an SSD (Solid State Drive), and an optical disk array.
• the storage device 22 may include a non-transient recording medium.
• the storage device 22 is capable of storing desired data.
• the storage device 22 may temporarily store a computer program executed by the arithmetic device 21.
• the storage device 22 may temporarily store data that is temporarily used by the arithmetic device 21 when the arithmetic device 21 is executing a computer program.
• the communication device 23 may be capable of communicating with devices external to the authentication device 20. Note that the communication device 23 may perform wired communication or wireless communication.
• the input device 24 is a device capable of accepting information input to the authentication device 20 from the outside.
• the input device 24 may include an operating device (e.g., a keyboard, a mouse, a touch panel, etc.) that can be operated by an operator of the authentication device 20.
• the input device 24 may include a recording medium reading device that can read information recorded on a recording medium that is detachable from the authentication device 20, such as a USB (Universal Serial Bus) memory.
• the communication device 23 may function as an input device.
• the output device 25 is a device capable of outputting information to the outside of the authentication device 20.
• the output device 25 may output visual information such as characters and images, auditory information such as voice, or tactile information such as vibration as the above information.
• the output device 25 may include at least one of a display, a speaker, a printer, and a vibration motor, for example.
• the output device 25 may be capable of outputting information to a recording medium that is detachable from the authentication device 20, such as a USB memory. Note that when the authentication device 20 outputs information via the communication device 23, the communication device 23 may function as an output device.
• the computing device 21 may have an image acquisition unit 211, a tracking unit 212, a face authentication unit 213, a determination unit 214, and an alarm unit 215 as functional blocks realized logically or as processing circuits realized physically.
• At least one of the image acquisition unit 211, the tracking unit 212, the face authentication unit 213, the determination unit 214, and the notification unit 215 may be realized in a form that combines logical functional blocks and physical processing circuits (i.e., hardware).
• the image acquisition unit 211, the tracking unit 212, the face authentication unit 213, the determination unit 214, and the notification unit 215 are functional blocks
• at least some of the image acquisition unit 211, the tracking unit 212, the face authentication unit 213, the determination unit 214, and the notification unit 215 may be realized by the calculation device 21 executing a predetermined computer program.
• the calculation device 21 may obtain (in other words, read) the above-mentioned specific computer program from the storage device 22.
• the calculation device 21 may read the above-mentioned specific computer program stored in a computer-readable and non-transient recording medium using a recording medium reading device (not shown) provided in the authentication device 20.
• the calculation device 21 may obtain (in other words, download or read) the above-mentioned specific computer program from a device (not shown) external to the authentication device 20 via the communication device 23.
• the recording medium for recording the above-mentioned specific computer program executed by the calculation device 21 may be at least one of an optical disk, a magnetic medium, a magneto-optical disk, a semiconductor memory, and any other medium capable of storing a program.
• image acquisition unit 211 "face authentication unit 213” and “face information DB 26” are components corresponding to the “image acquisition unit 11", “authentication unit 12” and “information DB 13" in the first embodiment described above, respectively.
• the information processing device 20a may have the same configuration as the information processing device 20. Therefore, a description of the information processing device 20a will be omitted. Below, the operation of the information processing device 20 will be described, but the information processing device 20a may also perform the same operation as the information processing device 20.
• the information processing device 20 performs face authentication of an object passing through the first lane set at Station A based on the image captured by the camera CAM1 (i.e., the first image).
• the information processing device 20 performs face authentication of an object passing through the second lane set at Station A based on the image captured by the camera CAM2 (i.e., the second image).
• the information processing device 20 may perform facial authentication of a target passing through the first lane while the target passes through the first lane from outside the station A (corresponding to the first area in FIG. 2) to inside the station A (corresponding to the second area in FIG. 2).
• the information processing device 20 may perform facial authentication of a target passing through the second lane while the target passes through the second lane from inside the station A to outside the station A.
• the information processing device 20 may be a device capable of performing walk-through facial authentication. Note that if no device capable of physically obstructing the passage of users is installed in the ticket gate area of station A, the facial authentication method performed by the information processing device 20 may be referred to as gateless facial authentication.
• the image acquisition unit 211 acquires an image (i.e., a first image) captured by camera CAM1 that includes at least a portion of the first lane set at station A.
• the image acquisition unit 211 acquires an image (i.e., a second image) captured by camera CAM2 that includes at least a portion of the second lane set at station A.
• cameras CAM1 and CAM2 may be connected to the information processing device 20 via a cable such as a USB cable. Cameras CAM1 and CAM2 may communicate with the information processing device 20 via wireless communication such as Bluetooth (registered trademark).
• the image acquisition unit 211 may directly acquire images from cameras CAM1 and CAM2. Cameras CAM1 and CAM2 may transmit the captured images to the station server 41. In this case, the image acquisition unit 211 may acquire the images captured by cameras CAM1 and CAM2 from the station server 41.
• the tracking unit 212 performs tracking processing of one or more objects passing through at least one of the first lane and the second lane, using the image (e.g., at least one of the first image and the second image) acquired by the image acquisition unit 211.
• the face authentication unit 213 performs face authentication processing of one or more people passing through at least one of the first lane and the second lane, using the image acquired by the image acquisition unit 211.
• tracking process performed by the tracking unit 212 a case where the tracking unit 212 performs tracking process on a target P1 passing through the first lane (see FIG. 2) will be described with reference to FIG. 9. Also, as a specific example of the face recognition process performed by the face recognition unit 213, a case where the face recognition unit 213 performs face recognition process on a target P1 will be described.
• Images IMG1, IMG2, and IMG3 shown in FIG. 9 are images that include object P1 (in other words, images in which object P1 is captured).
• Image IMG1 is an image generated by camera CAM1 capturing an image of the first lane at time t1.
• Image IMG2 is an image generated by camera CAM1 capturing an image of the first lane at time t2, which is later than time t1.
• Image IMG3 is an image generated by camera CAM1 capturing an image of the first lane at time t3, which is later than time t2.
• Each of images IMG1, IMG2, and IMG3 may be an image corresponding to one frame of a video.
• image IMG2 does not have to be an image corresponding to the frame immediately following the frame corresponding to image IMG1.
• image IMG2 may be an image corresponding to a frame that is two or more frames later than the frame corresponding to image IMG1.
• image IMG3 does not have to be an image corresponding to the frame immediately following the frame corresponding to image IMG2.
• image IMG3 may be an image corresponding to a frame that is two or more frames later than the frame corresponding to image IMG2.
• the tracking unit 212 detects the head of the target P1 contained in the image IMG1 from the image IMG1. Note that the method for detecting a person's head from an image is an application of existing technology, so a detailed explanation is omitted. Based on the detected head of the target P1, the tracking unit 212 sets the area including the head of the target P1 as the tracking area TA1.
• the tracking unit 212 sets the tracking area TA1, it sets a tracking ID, which is identification information for identifying the target P1 related to the tracking area TA1, to the target P1.
• the tracking unit 212 calculates the position of the target P1 (for example, the position of the feet of the target P1) based on the tracking area TA1. Note that since existing technology can be applied to the method of calculating the position of the subject contained in the image from the image, a detailed explanation is omitted.
• the tracking unit 212 links the tracking ID and the position of the target P1 to each other and registers them in the ID correspondence table 221 stored in the storage device 22.
• the tracking unit 212 determines whether or not the face of the target P1 is reflected in the tracking area TA1. In other words, the tracking unit 212 performs face detection on the tracking area TA1. When it is determined that the face of the target P1 is reflected in the tracking area TA1, the tracking unit 212 generates a face image including the face area of the target P1. The tracking unit 212 links the generated face image with the tracking ID related to the target P1 and transmits it to the face authentication unit 213. Note that when it is determined that the face of the target P1 is not reflected in the tracking area TA1, the tracking unit 212 does not need to generate a face image.
• the facial authentication unit 213 performs facial authentication processing using the facial image sent from the tracking unit 212. For example, the facial authentication unit 213 may calculate facial feature amounts indicating features related to the face of the target P1 from the facial image. The facial authentication unit 213 may calculate a matching score from the calculated facial feature amounts and facial information (e.g., facial feature amounts) of a registrant registered in the facial information DB 26.
• facial information e.g., facial feature amounts
• the face authentication unit 213 determines that face authentication has been successful. In this case, one registrant who corresponds to the target (here, target P1) indicated by the face image sent from the tracking unit 212 is identified from among the multiple registrants registered in the face information DB 26. On the other hand, if the calculated matching score is less than the predetermined threshold, the face authentication unit 213 determines that face authentication has failed. In this case, it is highly likely that the person who corresponds to the target (here, target P1) indicated by the face image sent from the tracking unit 212 is not registered in the face information DB 26. Note that if the calculated matching score and the predetermined threshold are "equal,” then either case can be included.
• the facial authentication unit 213 links the authentication ID indicating the registrant registered in the facial information DB 26, which corresponds to the target (target P1 in this case) indicated by the facial image sent from the tracking unit 212, to the tracking ID linked to the facial image, and registers it in the ID correspondence table 221.
• the facial authentication unit 213 may register the authentication time, which is the time when the facial authentication process was performed, in the ID correspondence table 221. If the facial authentication is unsuccessful, the facial authentication unit 213 may register information indicating that there is no corresponding person (for example, "N/A (Not Applicable)") in the ID correspondence table 221, linking it to the tracking ID linked to the facial image sent from the tracking unit 212.
• the "authentication ID" corresponds to an example of the "identification information for identifying one registrant" described above.
• the ID correspondence table 221 will be described with reference to FIG. 10.
• a tracking ID e.g., the position of the feet of the target P1
• an authentication ID e.g., the authentication time
• an authentication time may be linked to one another.
• a tracking ID when a tracking ID is linked to a specific authentication ID, it indicates that a registrant corresponding to the person for whom the tracking ID is set is registered in the face information DB 26.
• a tracking ID is linked to the character string "N/A"
• it indicates that a person corresponding to the person for whom the tracking ID is set is not registered in the face information DB 26.
• a tracking ID is not linked to an authentication ID (i.e., when the authentication ID field is blank), it indicates that face authentication processing has never been performed.
• the tracking unit 212 may use images IMG2 and IMG1 to identify target P1 contained in image IMG2. Identifying target P1 contained in image IMG2 is synonymous with associating target P1 contained in image IMG1 with target P1 contained in image IMG2. For this reason, at least one of a matching method and an optical flow method relating to association between images can be applied to identify target P1 contained in image IMG2. Note that since various existing aspects can be applied to each of the matching method and optical flow method, detailed explanations thereof will be omitted.
• the tracking unit 212 detects the head of target P1. Based on the detected head of target P1, the tracking unit 212 sets the area including the head of target P1 as tracking area TA2. Since target P1 included in image IMG1 and target P1 included in image IMG2 are the same target, the tracking ID of target P1 related to tracking area TA2 is the same as the tracking ID of target P1 related to tracking area TA1. The tracking unit 212 calculates the position of target P1 based on tracking area TA2.
• the tracking unit 212 links the position of the target P1 to the tracking ID related to the target P1 and registers it in the ID correspondence table 221. In this case, since the position of the target P1 calculated based on the tracking area TA1 is registered in the ID correspondence table 221, the tracking unit 212 registers the position of the target P1 calculated based on the tracking area TA2 in the ID correspondence table 221, thereby updating the position of the target P1.
• the tracking unit 212 judges whether or not the face of the target P1 is reflected in the tracking area TA2. In other words, the tracking unit 212 performs face detection on the tracking area TA2. When it is judged that the face of the target P1 is reflected in the tracking area TA2, the tracking unit 212 generates a face image including the face area of the target P1. The tracking unit 212 links the generated face image with the tracking ID related to the target P1 and transmits it to the face authentication unit 213. Note that, when it is judged that the face of the target P1 is not reflected in the tracking area TA2, the tracking unit 212 does not need to generate a face image.
• the tracking unit 212 does not need to generate a face image.
• the face authentication unit 213 performs face authentication processing using the face image sent from the tracking unit 212. If face authentication is successful, the face authentication unit 213 links an authentication ID indicating a registrant registered in face information DB 26 who corresponds to the target (here, target P1) indicated by the face image sent from the tracking unit 212 to the tracking ID linked to the face image, and registers it in the ID correspondence table 221. In addition to the authentication ID, the face authentication unit 213 may register the authentication time, which is the time when the face authentication processing was performed, in the ID correspondence table 221. Note that if the authentication time has already been registered in the ID correspondence table 221 (i.e., if face authentication has been successful in the past), the face authentication unit 213 may update the authentication time registered in the ID correspondence table 221. If face authentication fails, the face authentication unit 213 may register information indicating that there is no match (e.g., "N/A") in the ID correspondence table 221 by linking it to the tracking ID linked to the face image sent from the tracking unit 212.
• the tracking unit 212 judges whether or not the face of the target P1 is reflected in the tracking area TA3. In other words, the tracking unit 212 performs face detection on the tracking area TA3. When it is judged that the face of the target P1 is reflected in the tracking area TA3, the tracking unit 212 generates a face image including the face area of the target P1. The tracking unit 212 links the generated face image with the tracking ID related to the target P1 and transmits it to the face authentication unit 213. Note that, when it is judged that the face of the target P1 is not reflected in the tracking area TA3, the tracking unit 212 does not need to generate a face image.
• the tracking unit 212 does not need to generate a face image.
• the face authentication unit 213 performs face authentication processing using the face image sent from the tracking unit 212. If face authentication is successful, the face authentication unit 213 links an authentication ID indicating a registrant registered in face information DB 26 who corresponds to the target (here, target P1) indicated by the face image sent from the tracking unit 212 to the tracking ID linked to the face image, and registers it in the ID correspondence table 221. In addition to the authentication ID, the face authentication unit 213 may register the authentication time, which is the time when the face authentication processing was performed, in the ID correspondence table 221. Note that if the authentication time has already been registered in the ID correspondence table 221 (i.e., if face authentication has been successful in the past), the face authentication unit 213 may update the authentication time registered in the ID correspondence table 221. If face authentication fails, the face authentication unit 213 may register information indicating that there is no match (e.g., "N/A") in the ID correspondence table 221 by linking it to the tracking ID linked to the face image sent from the tracking unit 212.
• the tracking unit 212 may perform the following process. After tracking of the target P1 is interrupted, the tracking unit 212 may determine whether a new person has been detected from an image acquired by the image acquisition unit 211 (for example, an image captured by the camera CAM1). A "new person" means a person for whom a tracking ID has not been set.
• the tracking unit 212 may compare the feature amount of the tracking area related to the target P1 (e.g., at least one of the tracking areas TA1, TA2, and TA3) with the feature amount of the tracking area related to the new person to determine whether the target P1 and the new target are the same target. If it is determined that the target P1 and the new person are the same target, the tracking unit 212 may set a tracking ID related to the target P1 to the new person. As a result, the tracking unit 212 can track the target P1 again.
• the feature amount may be a feature amount related to the head of the target, a feature amount related to the upper body of the target, or a feature amount related to the entire body of the target.
• the tracking area (e.g., at least one of the tracking areas TA1, TA2, and TA3) may include the head of the target, the upper body of the target, or the entire body of the target.
• the feature amount may be obtained, for example, by Person Re-Identification technology.
• the tracking unit 212 and the face authentication unit 213 may repeat the above-mentioned process until the target P1 passes through the first lane.
• the tracking unit 212 and the face authentication unit 213 may perform the same process as the above-mentioned process for all targets passing through the first lane.
• the tracking unit 212 and the face authentication unit 213 may perform the same process as the above-mentioned process for all targets passing through the second lane.
• the determination unit 214 determines which of the multiple targets passing through the first lane or the second lane have been successfully facially recognized (in other words, which targets have been identified) based on the ID correspondence table 221.
• the determination unit 214 may determine that the target associated with the tracking ID has been successfully facially recognized.
• the determination unit 214 may determine that an object passing through the first lane has entered Station A if the tracking position linked to the tracking ID in the ID correspondence table 221 is closer to the premises of Station A (e.g., the second area in FIG. 2) than the first specified position in the first lane.
• the determination unit 214 may determine that an object passing through the second lane has exited Station A if the tracking position linked to the tracking ID in the ID correspondence table 221 is closer to the premises of Station A (e.g., the first area in FIG. 2) than the second specified position in the second lane.
• the notification unit 215 may link the authentication ID of the subject whose face has been successfully authenticated (in other words, the authentication ID of the registered user corresponding to the subject) with information indicating that the subject has entered or exited Station A, and transmit the linked information to the center server 50.
• a lane through which an object entering the railway station passes e.g., the first lane in FIG. 2
• a lane through which an object exiting the railway station passes e.g., the second lane in FIG. 2 are set in the ticket gate area of a railway station (e.g., at least one of Station A and Station B).
• a railway station e.g., at least one of Station A and Station B.
• the camera CAM1 can be arranged in advance so that the camera CAM1 can capture the front of an object passing through the first lane.
• the camera CAM2 can be arranged in advance so that the camera CAM2 can capture the front of an object passing through the second lane.
• the tracking unit 212 can relatively efficiently detect the face of the target (in other words, the target to be authenticated) from each of the image captured by camera CAM1 (i.e., the first image) and the image captured by camera CAM2 (i.e., the second image).
• an upward camera that captures a planar image of the ticket gate area from above may be installed.
• the image acquisition unit 211 of the information processing device 20 may acquire the image captured by the upward camera.
• the determination unit 214 may detect the number of objects passing through the first lane and the number of objects passing through the second lane based on the image captured by the upward camera.
• the determination unit 214 may determine whether the number of targets passing through the first lane is equal to the number of tracking IDs linked to positions in the first lane in the ID correspondence table 221 (in other words, the number of targets for which face recognition is performed using images captured by camera CAM1). If it is determined that the number of targets passing through the first lane is different from the number of tracking IDs linked to positions in the first lane in the ID correspondence table 221, the notification unit 215 may issue a warning to station staff, for example, via a terminal device used by the station staff at Station A.
• the determination unit 214 may determine whether the number of targets passing through the second lane is equal to the number of tracking IDs linked to positions in the second lane in the ID correspondence table 221 (in other words, the number of targets for which face recognition is performed using images captured by the camera CAM2). If it is determined that the number of targets passing through the second lane is different from the number of tracking IDs linked to positions in the second lane in the ID correspondence table 221, the notification unit 215 may issue a warning to station staff, for example, via a terminal device used by the station staff at Station A.
• an upper camera that captures a planar image of the ticket gate area from above may be installed.
• the resolution of the upper camera may be lower than that of the cameras CAM1 and CAM2.
• the image acquisition unit 211 of the information processing device 20 may acquire the image captured by the upper camera.
• the determination unit 214 may determine whether or not there is an object moving in the opposite direction to the first lane (in other words, a person running in the wrong direction) by performing a predetermined image processing on a plurality of temporally consecutive images captured by the upper camera.
• the notification unit 215 may issue a warning to a station attendant, for example, via a terminal device used by the station attendant of Station A.
• the tracking unit 212 and face authentication unit 213 of the information processing device 20 may perform the above-mentioned tracking process and face authentication process using an image captured by the second wrong-way runner camera.
• the determination unit 214 may determine that an object passing through area Ar2 has entered Station A when the tracking position linked to the tracking ID in the ID correspondence table 221 is, for example, closer to the second lane than the end of area Ar2 on the second lane side.
• a first camera e.g., camera CAM1
• a second camera e.g., camera CAM2
• the tracking unit 212 and face authentication unit 213 of the information processing device 20 may perform the above-mentioned tracking process and face authentication process using an image captured by the first camera.
• the determination unit 214 may determine that an object passing through area Ar3 has entered Station A when the tracking position linked to the tracking ID in the ID correspondence table 221 is, for example, on the area Ar4 side from the end of area Ar3 on the area Ar4 side.
• the tracking unit 212 and face authentication unit 213 of the information processing device 20 may perform the above-mentioned tracking process and face authentication process using an image captured by the second camera.
• the determination unit 214 may determine that an object passing through area Ar4 has exited from Station A when the tracking position linked to the tracking ID in the ID correspondence table 221 is, for example, closer to area Ar3 than the end of area Ar4 on the area Ar3 side.
• a face information database (e.g., face information DB 60 and 26) may have face information (e.g., face features) related to a suspicious target registered therein.
• the determination unit 214 may determine whether or not an authentication ID registered in the ID correspondence table 221 is an authentication ID related to a suspicious target. If it is determined that an authentication ID registered in the ID correspondence table 221 is an authentication ID related to a suspicious target, the notification unit 215 may issue a warning to a station attendant via a terminal device used by the station attendant at Station A, for example.
• the face information database (e.g., face information DB 60 and 26) may link the balance of the electronic money to identification information (e.g., authentication ID) relating to each of multiple registered users.
• the determination unit 214 of the information processing device 20 may determine whether the balance of electronic money linked to the identification information of the registered user corresponding to the target exiting from Station A is less than the railway fare (e.g., fare) to be paid. If it is determined that the balance of the electronic money is less than the railway fare to be paid, the notification unit 215 may notify, for example, a terminal device (e.g., a smartphone) carried by the target, that the balance is insufficient. In this case, the notification unit 215 may issue a warning to station staff, for example, via a terminal device used by the station staff at Station A.
• the railway fare e.g., fare
• a first foot camera that captures an image of the feet of a target passing through the first lane may be installed near the end of the first lane on the side inside the station A (e.g., the second area in FIG. 2 ), and a second foot camera that captures an image of the feet of a target passing through the second lane may be installed near the end of the second lane on the side outside the station A (e.g., the first area in FIG. 2 ).
• the determination unit 214 may detect the shoes of an object passing through the first lane from the image captured by the first foot camera. The determination unit 214 may determine whether the direction in which the toes of the detected shoes are pointing is the same as the traveling direction set for the first lane. If the direction in which the toes of the detected shoes are pointing is the opposite direction to the traveling direction set for the first lane, the determination unit 214 may determine that the object is a wrong-way runner.
• the determination unit 214 may detect the shoes of an object passing through the second lane from the image captured by the second foot camera. The determination unit 214 may determine whether the direction in which the toes of the detected shoes are pointing is the same as the traveling direction set for the second lane. If the direction in which the toes of the detected shoes are pointing is opposite to the traveling direction set for the second lane, the determination unit 214 may determine that the object is a wrong-way runner.
• the ID correspondence table 221 may include a history of the tracking position of each target.
• the determination unit 214 may determine the traveling direction of each target based on the history of the tracking position. In this case, when the first step of a target is from the inside of the A station (e.g., the second area in FIG. 2) side of the first lane, the determination unit 214 may determine that the traveling direction of the target is the opposite direction to the traveling direction set in the first lane. When the first step of a target is from the outside of the A station (e.g., the first area in FIG. 2) side of the second lane, the determination unit 214 may determine that the traveling direction of the target is the opposite direction to the traveling direction set in the second lane. Alternatively, the determination unit 214 may determine the traveling direction of the target from the first and second steps of the target.
• the determination unit 214 may determine that an object passing through the first lane is running in the opposite direction to the direction of travel set for the first lane, and may determine that the object is running in the wrong direction.
• the determination unit 214 may determine that an object passing through the second lane is running in the opposite direction to the direction of travel set for the second lane, and may determine that the object is running in the wrong direction.
• the determination unit 214 may determine a trajectory of the wrong-way runner based on the position of the wrong-way runner calculated by the tracking unit 212. The determination unit 214 may determine a reason why the wrong-way runner is running in the wrong direction based on the determined trajectory of the wrong-way runner.
• the determination unit 214 may determine that the wrong-way runner is running due to poor health.
• the notification unit 215 may notify station staff of the presence of a person who is feeling unwell, for example, via a terminal device used by the station staff at Station A.
• the determination unit 214 may determine that the wrong-way runner is intentionally driving in the wrong direction. In this case, the determination unit 214 may determine that the wrong-way runner has entered or exited Station A. As a result, the wrong-way runner may be charged a railway fare.
• the determination unit 214 may determine that the object associated with the tracking ID (in other words, the object to be authenticated) has moved from the first lane to the second lane.
• the notification unit 215 may, for example, notify a terminal device (e.g., a smartphone) carried by the target with a message such as "Please return to the first lane.”
• the notification unit 215 may notify the station staff, via a terminal device used by the station staff, of information for identifying the target (e.g., an image including the target) and information indicating that the target has moved from the first lane to the second lane.
• information for identifying the target e.g., an image including the target
• information indicating that the target has moved from the first lane to the second lane e.g., an image including the target
• the information processing device 20 may change the target from an authenticated target passing through the first lane to an authenticated target passing through the second lane.
• an image acquisition means for acquiring at least one of a first image including at least a portion of a first lane through which a first target to be authenticated traveling from a first area to a second area passes, and a second image including at least a portion of a second lane through which a second target to be authenticated traveling from the second area to the first area passes;
• an authentication means for performing at least one of authentication of the first subject using the first image and authentication of the second subject using the second image;
• Appendix 2 At least one of the first lane and the second lane includes a slope, The information processing system described in Appendix 1, wherein the image acquisition means acquires at least one of the first image including the first authenticated subject passing through the sloped road, or the first authenticated subject before entering the sloped road or after passing through the sloped road, and the second image including the second authenticated subject passing through the sloped road, or the second authenticated subject before entering the sloped road or after passing through the sloped road.
• Appendix 5 The information processing system of any one of appendix 1 to 4, further comprising an alarm means for issuing a warning when the number of people passing through the first lane differs from the number of people to be authenticated in the authentication performed by the authentication means using the first image, and/or when the number of people passing through the second lane differs from the number of people to be authenticated in the authentication performed by the authentication means using the second image.
• Appendix 8 acquiring at least one of a first image including at least a portion of a first lane through which a first target to be authenticated traveling from a first area to a second area passes and a second image including at least a portion of a second lane through which a second target to be authenticated traveling from the second area to the first area passes;
• An information processing method comprising: performing at least one of authentication of the first subject to be authenticated using the first image and authentication of the second subject to be authenticated using the second image.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Multimedia (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Health & Medical Sciences (AREA)
• General Health & Medical Sciences (AREA)
• Oral & Maxillofacial Surgery (AREA)
• Human Computer Interaction (AREA)
• Collating Specific Patterns (AREA)

Priority Applications (3)

Applications Claiming Priority (1)

Publications (1)

Family

ID=92904212

Family Applications (1)

Country Status (3)

Citations (7)

Family Cites Families (8)

• 2023
  • 2023-03-24 JP JP2025509219A patent/JPWO2024201579A1/ja active Pending
  • 2023-03-24 EP EP23930220.1A patent/EP4693173A4/en active Pending
  • 2023-03-24 WO PCT/JP2023/011873 patent/WO2024201579A1/ja not_active Ceased

Patent Citations (7)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events