WO2022172430A1 - 判定方法、判定プログラム、及び情報処理装置 - Google Patents

判定方法、判定プログラム、及び情報処理装置 Download PDF

Info

Publication number
WO2022172430A1
WO2022172430A1 PCT/JP2021/005432 JP2021005432W WO2022172430A1 WO 2022172430 A1 WO2022172430 A1 WO 2022172430A1 JP 2021005432 W JP2021005432 W JP 2021005432W WO 2022172430 A1 WO2022172430 A1 WO 2022172430A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
area
person
determination
photographed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/005432
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
壮一 ▲浜▼
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujitsu Ltd
Original Assignee
Fujitsu Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujitsu Ltd filed Critical Fujitsu Ltd
Priority to JP2022581139A priority Critical patent/JP7524980B2/ja
Priority to PCT/JP2021/005432 priority patent/WO2022172430A1/ja
Priority to CN202180091227.XA priority patent/CN116762098A/zh
Priority to EP21925681.5A priority patent/EP4293612A4/en
Publication of WO2022172430A1 publication Critical patent/WO2022172430A1/ja
Priority to US18/347,340 priority patent/US12591978B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/223—Analysis of motion using block-matching
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/40—Spoof detection, e.g. liveness detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/193—Preprocessing; Feature extraction

Definitions

  • the present invention relates to image judgment technology.
  • Biometric authentication technology is a technology that verifies a person's identity using biometric features such as fingerprints, face, and veins.
  • biometric authentication technology a biometric feature acquired in a situation where confirmation is required is compared (verified) with a pre-registered biometric feature, and identity verification is performed by determining whether or not the two match.
  • Face recognition technology which is one of biometric authentication technologies, is attracting attention as a means of contactless identity verification. Face recognition technology is used for a variety of purposes, including access control for personal terminals such as personal computers (PCs) and smartphones, room entry/exit control, and personal identification at airport boarding gates.
  • the facial image information used as biometric features in this face recognition technology differs from the information used as biometric features in other biometric authentication technologies such as fingerprint authentication and palm vein authentication. It can also be acquired by shooting with a simple camera.
  • face images are often published on the Internet through social networking services (SNS) and the like. For this reason, there is a concern that by presenting a photograph printed with a publicly available face image or a screen of a smartphone or the like on which the face image is displayed to a camera, a third party may impersonate the person himself/herself.
  • SNS social networking services
  • the photographed image taken by the camera is a photograph of the actual person (a person who is actually at the shooting location), or a photograph of the person or a display of the person such as a display screen showing the person.
  • Several techniques have been proposed for determining whether or not
  • Patent Document 1 See Patent Document 3
  • the captured image is a photograph of a display object of a person, it is impossible for such a display object to respond to the request on the spot.
  • a technology that allows the person to be authenticated to input a predetermined movement a technology that sees the response of the person to be authenticated to the display of the device, and furthermore, a person's biometrics by detecting natural human actions (such as blinking).
  • Techniques for determining whether or not are proposed (for example, see Patent Documents 4 to 9).
  • the photographed image is an image of the actual person by using the characteristics of the person's image area and the characteristics of the image area other than the person's image area (background image area) in the photographed image.
  • Several techniques have been proposed for performing More specifically, for example, a technique has been proposed for determining that an object is non-living when there is a variation of a predetermined value or more in the feature amount of a background area, which is an area other than a human area in a captured image.
  • a technology has been proposed that determines whether the object to be photographed is a photograph or a person by using the similarity of the motion feature amounts of the face region and the background region in the photographed image (for example, See Patent Documents 10 to 12).
  • Non-Patent Documents 1 to 4 techniques have been proposed for detecting the image area of an object or the image area of a person's face from a captured image (see, for example, Non-Patent Documents 1 to 4).
  • Non-Patent Document 5 a technique has been proposed for extracting the motion of an image by using an optical flow obtained from changes in the luminance gradient of each pixel that constitutes a time-series image (see, for example, Non-Patent Document 5).
  • the image taken during face authentication may be blurred.
  • blurring occurs, for example, when a notebook PC is placed on one's lap in a car such as a train, or when the camera is shaken due to vibrations in the surroundings because the camera is not rigidly fixed. If such a blur caused by camera shake at the time of photographing is present in the photographed image, the accuracy of determining whether or not the photographed image is a person's display object may be degraded.
  • a technique has been proposed for determining that an object is non-living when there is a variation of a predetermined value or more in the feature amount of the background area, which is the area other than the human area in the captured image.
  • This technique focuses on the fact that when the photographed image is a photographed image of a real person, the feature amount of the background region hardly changes, and the above determination is performed by detecting such a change.
  • this technique detects variations in the feature amount of the background area even from a photographed image in which there is blurring as described above. For this reason, if there is blurring in the captured image, this technique may erroneously discriminate the object as a non-living object even if the object is a living object.
  • a technique of determining whether the object to be photographed is a photograph or a person by using the similarity of the motion feature amounts of the face region and the background region in the photographed image.
  • This technique focuses on the interlocking movement of the face area and the background area in a photographed image obtained by photographing a photograph of a person, and performs the above-described determination by detecting this interlocking.
  • the movements of the face area and the background area are interlocked. Therefore, when there is blurring in the captured image, this technology may erroneously determine that the captured image is a photograph even if the captured image is a photograph of the actual person.
  • an object of the present invention is to improve the accuracy of determining whether or not a photographed image is a photograph of a displayed object of a person.
  • the computer acquires a captured image containing a person's image area captured by a camera.
  • the computer specifies an image area other than the image area of the person from the acquired photographed image.
  • the computer determines whether or not the photographed image is a photograph of a person's display object, according to the distribution of motions at a plurality of positions included in the specified image area.
  • FIG. 10 is a diagram (part 1) for explaining the state of synchronization and asynchronization of movement of each image area of a captured image when camera shake occurs at the time of capturing
  • FIG. 11 is a diagram (part 2) for explaining the state of synchronization and asynchronization of movement of each image area of a captured image when camera shake occurs at the time of capturing
  • It is a figure which shows the structure of an exemplary information processing apparatus. It is a figure which shows the hardware configuration example of a computer.
  • 7 is a flow chart showing the details of a captured image determination process
  • 4 is a flow chart showing the processing contents of image region identification processing.
  • FIG. 10 is a diagram (part 1) for explaining the state of synchronization and asynchronization of movement of each image area of a captured image when camera shake occurs at the time of capturing
  • FIG. 11 is a diagram (part 2) for explaining the state of synchronization and asynchronization of movement of each image area of a captured image when camera shake occurs at the time of capturing
  • FIG. 11 is a diagram (part 1) for explaining an example of a technique for identifying a person area;
  • FIG. 11 is a diagram (part 2) for explaining an example of a technique for identifying a person area;
  • FIG. 10 is a diagram illustrating an example of a technique for specifying a background area;
  • 4 is a flow chart showing the details of motion extraction processing.
  • 4 is a flow chart showing the details of determination processing.
  • FIG. 10 is a diagram illustrating an example of obtaining a motion vector of an image using a plurality of pairs of captured images;
  • the photographed image is of a display object of a person according to the distribution of movements at a plurality of positions included in an image area other than the person's image area in the photographed image photographed by the camera. judgment is made. This method will be explained.
  • each image area is detected from the captured image captured by the camera.
  • FIG. 1 is a diagram for explaining each image area of the captured image 10.
  • FIG. 1 image areas of a peripheral area 11 , a person area 12 and a background area 13 are detected from the captured image 10 .
  • the peripheral area 11 is an area on the outer periphery of the captured image 10, and is an annular area with the edge of the captured image 10 as the outer periphery.
  • Both the person area 12 and the background area 13 are areas surrounded by the inner circumference of the peripheral area 11 .
  • the person area 12 is an image area in which a person is represented.
  • the background area 13 is an area other than the person area 12, and is an area where an object other than the person is represented.
  • the photographed image 10 is a photograph of a real person
  • the person is displayed in the person area 12, and the actual background of the person at the time of photographing the photographed image 10 is the background area 13 and the peripheral area 11. displayed in both.
  • the peripheral area 11 the surrounding scene of the background displayed in the background area 13 is displayed.
  • the photographed image 10 is a photograph of a display object of a person
  • the image displayed on the display object at the time of photographing the photographed image 10 is displayed in both the person region 12 and the background region 13, and the image is photographed.
  • a peripheral area 11 displays a scene around the display object when the image 10 is captured.
  • the person area 12 displays the image of the person represented in the display object
  • the background area 13 displays the background image represented with the person in the display object.
  • the solid line graphs represent the behavior of the magnitude of the difference vector for the photographed image 10 obtained by photographing the actual person, and the broken line graphs are photographed images obtained by photographing the display object.
  • 3 illustrates the magnitude behavior of the difference vector for image 10.
  • the horizontal axis in each of the graphs of FIGS. 2A and 2B represents the shooting time of the captured image 10.
  • FIG. In the graph of FIG. 2A the magnitude of the difference vector between the motion vector representing the motion of the person region 12 and the motion vector representing the motion of the background region 13 is represented along the vertical axis.
  • the magnitude of the difference vector between the motion vector representing the motion of the peripheral region 11 and the motion vector representing the motion of the background region 13 is represented along the vertical axis.
  • the magnitude of the difference vector for the motions of the two regions is small, and when the motions of the two regions are not synchronized, the The magnitude of the difference vector for motion of the two regions increases.
  • the magnitude of the difference vector for the photographed image 10 of the display object is small, and the magnitude of the difference vector for the photographed image 10 of the actual person is large. Therefore, the movements of the person region 12 and the background region 13 in the photographed image 10 of the displayed object are substantially synchronized, and on the other hand, the person region 12 and the background region of the photographed image 10 in which the real person is photographed. It can be seen that the movement with 13 is not synchronized.
  • the magnitude of the difference vector for the photographed image 10 of the actual person is small, and the magnitude of the difference vector for the photographed image 10 of the display object is large. Therefore, the movements of the peripheral region 11 and the background region 13 in the photographed image 10 of the real person are substantially synchronized. It can be seen that the movement with 13 is not synchronized.
  • shooting is performed according to the distribution of motion of each position included in each image area. It is determined whether or not the image 10 is an image of a displayed object.
  • FIG. 3 shows the configuration of an exemplary information processing device 20. As shown in FIG.
  • a camera 30 is connected to the information processing device 20 .
  • a camera 30 photographs an object to be photographed and outputs a photographed image 10.
  • the camera 30 originally captures a person. For example, when performing face authentication, the camera 30 captures the face of the person to be authenticated.
  • the camera 30 repeatedly captures images of an object to be captured and outputs time-series captured images 10 .
  • the time-series captured images 10 are used to extract the motion of each area of the captured images 10 .
  • the information processing device 20 includes an image acquisition unit 21, an area identification unit 22, a motion extraction unit 23, and a determination unit 24 as components.
  • the image acquisition unit 21 acquires and stores the captured image 10 captured by the camera 30 .
  • the area specifying unit 22 extracts each image area described with reference to FIG. and background area 13).
  • the motion extracting unit 23 extracts the motion of each image region specified by the region specifying unit 22 from the captured image 10, and obtains the distribution of the motion of each position included in each image region.
  • the determination unit 24 determines whether or not the photographed image 10 is a photograph of a person's display object, according to the motion distribution of each position included in each image region acquired by the motion extraction unit 23 . .
  • the information processing device 20 in FIG. 3 may be configured by a combination of a computer and software.
  • FIG. 4 shows an example of the hardware configuration of the computer 40.
  • the computer 40 includes hardware such as a processor 41, a memory 42, a storage device 43, a reader 44, a communication interface 46, and an input/output interface 47 as components. These components are connected via a bus 48, and data can be exchanged between the components.
  • the processor 41 may be, for example, a single processor or a multiprocessor and multicore.
  • the processor 41 uses the memory 42 to execute, for example, a photographed image determination processing program describing the procedure of the photographed image determination processing described later.
  • the memory 42 is, for example, a semiconductor memory, and may include a RAM area and a ROM area.
  • the storage device 43 is, for example, a hard disk, a semiconductor memory such as a flash memory, or an external storage device. Note that RAM is an abbreviation for Random Access Memory. Also, ROM is an abbreviation for Read Only Memory.
  • the reading device 44 accesses the removable storage medium 45 according to instructions from the processor 41 .
  • the removable storage medium 45 is, for example, a semiconductor device (USB memory, etc.), a medium for inputting/outputting information by magnetic action (magnetic disk, etc.), a medium for inputting/outputting information by optical action (CD-ROM, DVD, etc.).
  • USB is an abbreviation for Universal Serial Bus.
  • CD is an abbreviation for Compact Disc.
  • DVD is an abbreviation for Digital Versatile Disk.
  • the communication interface 46 transmits and receives data via a communication network (not shown) according to instructions from the processor 41, for example.
  • the input/output interface 47 acquires various data such as the image data of the captured image 10 sent from the camera 30 . Also, the input/output interface 47 outputs the result of the captured image determination process, which will be described later, output from the processor 41 .
  • a program executed by the processor 41 of the computer 40 is provided, for example, in the following form. (1) Pre-installed in the storage device 43 . (2) provided by a removable storage medium 45; (3) provided from a server such as a program server to communication interface 46 via a communication network;
  • the hardware configuration of the computer 40 is an example, and the embodiment is not limited to this.
  • some or all of the functions of the functional units described above may be implemented as hardware such as FPGA and SoC.
  • FPGA is an abbreviation for Field Programmable Gate Array.
  • SoC is an abbreviation for System-on-a-chip.
  • FIG. 5 is a flow chart showing the details of this photographed image determination process.
  • the time-series photographed images 10 photographed by the camera 30 sent from the camera 30 are acquired through the input/output interface 47 and stored in the memory 42 .
  • the outer periphery of the photographed image 10 is a horizontally long rectangle.
  • the direction of the long side of this rectangle is the horizontal direction of the captured image 10 .
  • the direction of the short side of the rectangle (the direction orthogonal to the horizontal direction of the captured image 10) is the vertical direction of the captured image 10, and the direction of the head of the person represented in the captured image 10 is the top of the captured image 10. and the direction of the torso of the person is the downward direction of the photographed image 10 .
  • the processor 41 provides the function of the image acquisition unit 21 in FIG. 3 by executing the process of S101.
  • image area specifying processing is performed.
  • This process is a process of specifying the person area 12 and areas other than the person area 12 (peripheral area 11 and background area 13) from the captured image 10 obtained by the process of S101. The details of this processing will be described later.
  • This process is a process of extracting the motion of each image area specified by the process of S102 from the captured image 10 and acquiring the motion distribution state of each position included in each image area. The details of this processing will be described later.
  • This process is a process of extracting the motion of each image area specified by the process of S102 from the captured image 10 and acquiring the motion distribution state of each position included in each image area. The details of this processing will be described later.
  • FIG. 6 is a flow chart showing the processing contents of the image region identification processing.
  • the processor 41 provides the function of the area identifying section 22 in FIG. 3 by executing this image area identifying process.
  • the width of the ring which is the peripheral area 11
  • this width is set to 5% of the length of the horizontal width of the captured image 10 .
  • S ⁇ b>202 processing is performed to identify the person area 12 in each of the time-series captured images 10 stored in the memory 42 .
  • Many techniques are known as techniques for identifying a person's region from an image, and any of these known techniques may be used as the processing of S202.
  • a technique called semantic segmentation that extracts pixels that correspond to people in an image is known.
  • a technique for realizing semantic segmentation for example, a technique using a Convolutional Neural Network (CNN) is known.
  • CNN Convolutional Neural Network
  • PSPNet Pulid Scene Parsing Network
  • this PSPNet may be used to specify the person area 12 from the area surrounded by the inner periphery of the peripheral area 11 in the captured image 10.
  • a technique for detecting a rectangular area also called a bounding box
  • a method using CNN is also known as a method for realizing detection of this rectangular area.
  • the "Single Shot MultiBox Detector" (SSD) proposed in Non-Patent Document 2 cited above and the “You Only Look Once” (YOLO) proposed in Non-Patent Document 3 cited above use CNN.
  • This is an example of a technique for detecting such a rectangular area.
  • the "Multi-task Cascaded Convolutional Networks" (MTCNN) proposed in Non-Patent Document 4 cited above is also an example of a technique for detecting such a rectangular area. It is a specialized method.
  • any one of these rectangular area detection techniques may be used to identify the person area 12 from the area surrounded by the inner circumference of the peripheral area 11 in the captured image 10.
  • FIG. 1 any one of these rectangular area detection techniques may be used to identify the person area 12 from the area surrounded by the inner circumference of the peripheral area 11 in the captured
  • the identification is performed using semantic segmentation such as PSPNet
  • the body part of the person including the head and torso in the area surrounded by the inner circumference of the peripheral area 11 is represented.
  • the area where the person is is identified as the person area 12 as shown in FIG. 7A.
  • a rectangular area is detected by a method such as SSD, YOLO, or MTCNN
  • the rectangular area including the head of a person, among the areas surrounded by the inner circumference of the peripheral area 11 is the face area. 14 is detected. In this case, as shown in FIG.
  • an area included in a rectangle obtained by extending the rectangle of the face area 14 downward in the photographed image 10 to a position in contact with the inner periphery of the peripheral area 11 is defined as the person area 12.
  • a part of the person's body part is also included in the person area 12 .
  • a process of specifying the background area 13 in each of the time-series captured images 10 stored in the memory 42 is performed.
  • the background area 13 is specified as the remaining area of the captured image 10 excluding the peripheral area 11 specified by the process of S201 and the person area 12 specified by the process of S202.
  • the processing up to the above is the image region identification processing.
  • FIG. 9 is a flow chart showing the contents of the motion extraction process.
  • the processor 41 provides the function of the motion extraction unit 23 of FIG. 3 by executing this motion extraction processing.
  • a process of acquiring an image motion vector for each pixel forming the captured image 10 is performed.
  • a motion vector is extracted based on changes in luminance gradients in two of the time-series captured images 10 stored in the memory 42 by the process of S101 in FIG.
  • Non-Patent Document 5 is also an example of the optical flow calculation method.
  • the optical flow calculated using the method proposed in Non-Patent Document 5 may be used to obtain a two-dimensional motion vector for each pixel of the captured image 10.
  • a process of calculating the average vector for the peripheral area 11 is performed.
  • a process of calculating the average of all pixels of the motion vectors obtained by the process of S301 for each pixel of the captured image 10 included in the peripheral area 11 is performed.
  • the average vector vp calculated by this process is an example of a motion vector representing the motion of positions included in the peripheral area 11 .
  • the average vector vp for the peripheral area 11 of the captured image 10 is a two-dimensional vector.
  • the component vpx of the average vector vp in the horizontal direction (x direction) and the component vpy in the vertical direction (y direction) of the captured image 10 are calculated by the following formula (1). be.
  • vx(i, j) and v(i, j) are positions (i, j) on the two-dimensional coordinates defined by the x direction and the y direction of the captured image 10, respectively. are the values of the x component and the y component of the motion vector for the pixel specified by (pixel included in the peripheral area 11). Also, np is the number of pixels included in the peripheral area 11 . That is, the formula [Formula 1] divides the sum of the x-component and the y-component of the motion vector for each pixel included in the peripheral region 11 by the number of pixels in the peripheral region 11 to obtain the average vector vp , the components vpx and vpy of are calculated respectively.
  • a process of calculating an average vector for the person area 12 is performed.
  • a process of calculating the average of all pixels of the motion vectors obtained by the process of S301 for each pixel of the captured image 10 included in the person area 12 is performed.
  • the average vector vf calculated by this process is an example of a motion vector representing the motion of positions included in the person region 12 .
  • the method for calculating the average vector vf for the person region 12 may be the same as the method for calculating the average vector vp for the peripheral region 11, which has been described with regard to the process of S302.
  • a process of calculating the average vector for the background area 13 is performed.
  • a process of calculating the average of all pixels of the motion vectors obtained by the process of S301 for each pixel of the captured image 10 included in the background area 13 is performed.
  • the average vector vb calculated by this process is an example of a motion vector representing the motion of positions included in the background area 13 .
  • the method of calculating the average vector vb for the background area 13 may also be the same as the method of calculating the average vector vp for the peripheral area 11, which has been described with regard to the process of S302.
  • the motion vector of each pixel constituting the captured image 10 is obtained by the processing of S301, and the average vector of the pixels included in each region is obtained by the subsequent processing of S302, S303, and S304. Calculation is performed for each region.
  • the captured image 10 may be divided into regions, and then the motion vector of each pixel included in the divided captured image 10 may be obtained, and then the average vector may be calculated for each region. good.
  • FIG. 10 is a flow chart showing the contents of the determination process.
  • the processor 41 provides the function of the determination unit 24 of FIG. 3 by executing this determination processing.
  • the first difference vector vdiff1 is the difference between the motion vector representing the motion of the position included in the person region 12 and the motion vector representing the motion of the position included in the background region 13.
  • the following [Equation 2 ] are calculated by calculating the formulas.
  • vf and vb are average vectors for the person area 12 and the background area 13, respectively.
  • vfx and vfy are the values of the x component and the y component of the average vector vf for the person region 12
  • vbx and vby are the values of the x component and the y component of the average vector vb for the background region 13, respectively. value.
  • the first difference vector vdiff1 calculated in this way is an example of an index representing the difference between the movement of the position included in the background area 13 and the movement of the position included in the person area 12. This is an example of the distribution of the movement of .
  • the second difference vector vdiff2 is the difference between the motion vector representing the motion of the position included in the background area 13 and the motion vector representing the motion of the position included in the peripheral area 11.
  • the following [Equation 3 ] are calculated by calculating the formulas.
  • vb and vp in the [Equation 3] are the average vectors for the background area 13 and the peripheral area 11, respectively.
  • vbx and vby are the values of the x component and the y component of the average vector vb for the background region 13, respectively
  • vpx and vpy are the values of the x component and the y component of the average vector vp for the peripheral region 11, respectively. value.
  • the second difference vector vdiff2 calculated in this manner is an example of an index representing the difference between the movement of the position included in the background area 13 and the movement of the position included in the peripheral area 11, and This is an example of the distribution of the movement of .
  • the magnitude of the first difference vector vdiff1 is calculated by calculating the square root of the sum of the squares of the x component value and the y component value for the first difference vector vdiff1.
  • the first threshold is a preset value.
  • the magnitude of the average vector vb of the background region 13 in the photographed image 10 including the blur of the person's display object captured while shaking the camera 30 is preliminarily estimated by a plurality of experiments, and the obtained estimation is A value about 1/2 of the value is set as the first threshold.
  • processing is performed to determine that the captured image 10 is a photograph of a real person.
  • the magnitude of the second difference vector vdiff2 is calculated by calculating the square root of the sum of the squares of the x component value and the y component value for the second difference vector vdiff2.
  • the second threshold is a preset value.
  • the magnitude of the average vector vb of the background region 13 in the photographed image 10 including the blur of the person's display object captured while shaking the camera 30 is preliminarily estimated by a plurality of experiments, and the obtained estimation is A value about 1/2 of the value is set as the second threshold.
  • processing is performed to determine that the photographed image 10 is an image of a displayed object of a person.
  • the above processing is the judgment processing.
  • the processor 41 operates as the information processing apparatus 20 shown in FIG. 3 by executing the above photographed image determination processing by the processor 41, and judges whether or not the photographed image 10 is a person's display object. Allows you to do it with precision.
  • two of the time-series captured images 10 are used to acquire the motion vector of the image in each pixel that constitutes the captured images 10 .
  • a plurality of pairs of captured images 10 each composed of two of the time-series captured images 10 are used to acquire a motion vector for each pixel for each pair, and the obtained An average of a plurality of motion vectors may be used as a motion vector for each pixel.
  • the motion vector of each pixel is acquired for each pair of the four captured images 10, and the average motion vector of the obtained four motion vectors is calculated for each pixel to form the captured image 10.
  • 4 shows an example of obtaining a motion vector of an image at each pixel that is displayed. By doing so, the accuracy of the motion vector of the acquired image is improved.
  • a moving average may be calculated.
  • the weighted average according to the area of each region may be calculated.
  • two adjacent frames of the captured images 10 of five frames that are consecutive in time series form one pair, forming four pairs.
  • the two frames that make up a pair should be two frames with some frames in between, rather than two adjacent frames. good too. By doing so, the image difference between the two frames forming a pair becomes large, so even if the camera 30 shoots at a very high frame rate, for example, Detected image motion may be stabilized.
  • a general camera 30 is used as the camera 30 connected to the information processing device 20 of FIG.
  • an infrared camera or a depth camera capable of outputting a grayscale image may be used as camera 30 .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Human Computer Interaction (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Geometry (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Collating Specific Patterns (AREA)
PCT/JP2021/005432 2021-02-15 2021-02-15 判定方法、判定プログラム、及び情報処理装置 Ceased WO2022172430A1 (ja)

Priority Applications (5)

Application Number Priority Date Filing Date Title
JP2022581139A JP7524980B2 (ja) 2021-02-15 2021-02-15 判定方法、判定プログラム、及び情報処理装置
PCT/JP2021/005432 WO2022172430A1 (ja) 2021-02-15 2021-02-15 判定方法、判定プログラム、及び情報処理装置
CN202180091227.XA CN116762098A (zh) 2021-02-15 2021-02-15 判定方法、判定程序、以及信息处理装置
EP21925681.5A EP4293612A4 (en) 2021-02-15 2021-02-15 DETERMINATION METHOD, DETERMINATION PROGRAM AND INFORMATION PROCESSING DEVICE
US18/347,340 US12591978B2 (en) 2021-02-15 2023-07-05 Determination method for determining identity of person depending on movement of positions in specified image region, non-transitory computer-readable storage medium, and information processing apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2021/005432 WO2022172430A1 (ja) 2021-02-15 2021-02-15 判定方法、判定プログラム、及び情報処理装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/347,340 Continuation US12591978B2 (en) 2021-02-15 2023-07-05 Determination method for determining identity of person depending on movement of positions in specified image region, non-transitory computer-readable storage medium, and information processing apparatus

Publications (1)

Publication Number Publication Date
WO2022172430A1 true WO2022172430A1 (ja) 2022-08-18

Family

ID=82838557

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/005432 Ceased WO2022172430A1 (ja) 2021-02-15 2021-02-15 判定方法、判定プログラム、及び情報処理装置

Country Status (5)

Country Link
US (1) US12591978B2 (https=)
EP (1) EP4293612A4 (https=)
JP (1) JP7524980B2 (https=)
CN (1) CN116762098A (https=)
WO (1) WO2022172430A1 (https=)

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001126091A (ja) 1999-10-27 2001-05-11 Toshiba Corp 乗員顔画像処理システムおよび通行料金収受システム
JP2005259049A (ja) 2004-03-15 2005-09-22 Omron Corp 顔面照合装置
JP2006099614A (ja) 2004-09-30 2006-04-13 Toshiba Corp 生体判別装置および生体判別方法
JP2006330936A (ja) 2005-05-24 2006-12-07 Matsushita Electric Works Ltd 顔認証装置
JP2008000464A (ja) 2006-06-23 2008-01-10 Glory Ltd 認証装置および認証方法
JP2008090452A (ja) 2006-09-29 2008-04-17 Omron Corp 検出装置、方法およびプログラム
WO2009107237A1 (ja) 2008-02-29 2009-09-03 グローリー株式会社 生体認証装置
WO2009110323A1 (ja) 2008-03-03 2009-09-11 日本電気株式会社 生体判定システム、生体判定方法および生体判定用プログラム
JP2010225118A (ja) 2009-03-25 2010-10-07 Toshiba Corp 画像処理装置、方法及びプログラム
JP2016152029A (ja) 2015-02-19 2016-08-22 大阪瓦斯株式会社 顔認証装置、画像処理装置及び生体判定装置
JP2016173813A (ja) 2015-03-16 2016-09-29 株式会社リコー 情報処理装置、情報処理システム、認証方法、及びプログラム
WO2019151368A1 (ja) 2018-02-01 2019-08-08 日本電気株式会社 生体認証装置、システム、方法および記録媒体

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070098303A1 (en) * 2005-10-31 2007-05-03 Eastman Kodak Company Determining a particular person from a collection
JP2007272488A (ja) * 2006-03-31 2007-10-18 Yokogawa Electric Corp 画像処理装置、監視カメラ及び画像監視システム
JP5940474B2 (ja) * 2012-09-27 2016-06-29 富士フイルム株式会社 体動検出装置および方法
US11003961B2 (en) * 2014-06-03 2021-05-11 Nec Corporation Image processing system, image processing method, and program storage medium
US10079827B2 (en) 2015-03-16 2018-09-18 Ricoh Company, Ltd. Information processing apparatus, information processing method, and information processing system

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001126091A (ja) 1999-10-27 2001-05-11 Toshiba Corp 乗員顔画像処理システムおよび通行料金収受システム
JP2005259049A (ja) 2004-03-15 2005-09-22 Omron Corp 顔面照合装置
JP2006099614A (ja) 2004-09-30 2006-04-13 Toshiba Corp 生体判別装置および生体判別方法
JP2006330936A (ja) 2005-05-24 2006-12-07 Matsushita Electric Works Ltd 顔認証装置
JP2008000464A (ja) 2006-06-23 2008-01-10 Glory Ltd 認証装置および認証方法
JP2008090452A (ja) 2006-09-29 2008-04-17 Omron Corp 検出装置、方法およびプログラム
WO2009107237A1 (ja) 2008-02-29 2009-09-03 グローリー株式会社 生体認証装置
WO2009110323A1 (ja) 2008-03-03 2009-09-11 日本電気株式会社 生体判定システム、生体判定方法および生体判定用プログラム
JP2010225118A (ja) 2009-03-25 2010-10-07 Toshiba Corp 画像処理装置、方法及びプログラム
JP2016152029A (ja) 2015-02-19 2016-08-22 大阪瓦斯株式会社 顔認証装置、画像処理装置及び生体判定装置
JP2016173813A (ja) 2015-03-16 2016-09-29 株式会社リコー 情報処理装置、情報処理システム、認証方法、及びプログラム
WO2019151368A1 (ja) 2018-02-01 2019-08-08 日本電気株式会社 生体認証装置、システム、方法および記録媒体

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
GUNNAR FARNEBACK: "Two-Frame Motion Estimation Based on Polynomial Expansion", PROCEEDINGS OF THE 13TH SCANDINAVIAN CONFERENCE ON IMAGE ANALYSIS, 2003, pages 363 - 370, XP055500834, DOI: 10.1007/3-540-45103-X_50
HENGSHUANG ZHAO ET AL.: "Pyramid Scene Parsing Network", PROCEEDINGS OF THE IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR, 2017
JOSEPH REDMON ET AL.: "You Only Look Once:Unified, Real-Time Object Detection", IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR, pages 779 - 788
KAIPENG ZHANG ET AL.: "Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks", IEEE SIGNAL PROCESSING LETTERS (SPL, vol. 23, October 2016 (2016-10-01), pages 1499 - 1503
See also references of EP4293612A4
WEI LIU ET AL.: "European Conference on Computer Vision (ECCV) 2016", SPRINGER INTERNATIONAL PUBLISHING, article "SSD:Single Shot MultiBox Detector", pages: 21 - 37

Also Published As

Publication number Publication date
US20230342947A1 (en) 2023-10-26
CN116762098A (zh) 2023-09-15
EP4293612A4 (en) 2024-04-03
US12591978B2 (en) 2026-03-31
JP7524980B2 (ja) 2024-07-30
JPWO2022172430A1 (https=) 2022-08-18
EP4293612A1 (en) 2023-12-20

Similar Documents

Publication Publication Date Title
CN108197586B (zh) 脸部识别方法和装置
US9652663B2 (en) Using facial data for device authentication or subject identification
CN106096582B (zh) 区分真人面部与平坦表面
Das et al. Recent advances in biometric technology for mobile devices
CN106326832B (zh) 基于物体区域处理图像的装置及方法
JP2004192378A (ja) 顔画像処理装置およびその方法
EP3859662B1 (en) Authentication device, authentication method, and storage medium
JP6071002B2 (ja) 信頼度取得装置、信頼度取得方法および信頼度取得プログラム
JPWO2010137157A1 (ja) 画像処理装置、方法、プログラム
JP5087037B2 (ja) 画像処理装置、方法及びプログラム
JP5170094B2 (ja) なりすまし検知システム、なりすまし検知方法およびなりすまし検知用プログラム
KR20160093523A (ko) 안구 위치 정보 확정 방법 및 장치
JP6157165B2 (ja) 視線検出装置及び撮像装置
KR102807290B1 (ko) 화상 처리 장치, 화상 처리 방법 및 화상 처리 프로그램
JP2022526468A (ja) 二次元顔画像の２つ以上の入力に基づいて三次元顔モデルを適応的に構築するためのシステムおよび方法
CN110008943B (zh) 一种图像处理方法及装置、一种计算设备及存储介质
Dospinescu et al. Face detection and face recognition in android mobile applications
CN111263955A (zh) 一种目标对象的移动轨迹确定方法和装置
Mishra et al. Integrating state-of-the-art face recognition and anti-spoofing techniques into enterprise information systems
JPWO2020065851A1 (ja) 虹彩認証装置、虹彩認証方法、虹彩認証プログラムおよび記録媒体
CN110799986A (zh) 一种基于面部特征点的眨眼动作识别的系统和方法
JP7524980B2 (ja) 判定方法、判定プログラム、及び情報処理装置
CN110753931A (zh) 基于面部特征点的点头动作识别的系统和方法
JP2022040244A (ja) 情報処理装置、方法およびプログラム
JP2006277146A (ja) 照合方法および照合装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21925681

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022581139

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 202180091227.X

Country of ref document: CN

WWE Wipo information: entry into national phase

Ref document number: 2021925681

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021925681

Country of ref document: EP

Effective date: 20230915