WO2022059066A1 - 検出システム、検出方法、及びコンピュータプログラム - Google Patents

検出システム、検出方法、及びコンピュータプログラム Download PDF

Info

Publication number
WO2022059066A1
WO2022059066A1 PCT/JP2020/034900 JP2020034900W WO2022059066A1 WO 2022059066 A1 WO2022059066 A1 WO 2022059066A1 JP 2020034900 W JP2020034900 W JP 2020034900W WO 2022059066 A1 WO2022059066 A1 WO 2022059066A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature
detection system
image
living body
feature point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/034900
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
貴裕 戸泉
巧一 高橋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Corp
Original Assignee
NEC Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Corp filed Critical NEC Corp
Priority to JP2022550069A priority Critical patent/JP7666823B2/ja
Priority to PCT/JP2020/034900 priority patent/WO2022059066A1/ja
Priority to US17/640,160 priority patent/US11837023B2/en
Publication of WO2022059066A1 publication Critical patent/WO2022059066A1/ja
Anticipated expiration legal-status Critical
Priority to JP2024227884A priority patent/JP7798165B2/ja
Priority to JP2025280675A priority patent/JP2026040643A/ja
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/60—Rotation of whole images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G06V10/242—Aligning, centring, orientation detection or correction of the image by image rotation, e.g. by 90 degrees
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • This disclosure relates to the technical fields of detection systems, detection methods, and computer programs that detect a part of a living body from an image.
  • Patent Document 1 discloses that a pupil circle and an iris circle are detected from an image.
  • Patent Document 2 discloses that a face is detected from an image and eyes are detected from the position information of the face.
  • Patent Document 3 discloses that feature points are extracted from a facial image.
  • Patent Document 4 it is detected that a circular region is detected from the ROI (Region Of Interest) and a plurality of regions that are candidates for the iris are detected.
  • the subject of this disclosure is to provide a detection system, a detection method, and a computer program capable of solving the above-mentioned problems.
  • One aspect of the detection system of the present disclosure is an acquisition means for acquiring an image including a living body, and a feature figure corresponding to a substantially circular first portion in the living body is detected from the image, and the first in the living body.
  • a detection means for detecting a feature point corresponding to a second portion around the portion is provided.
  • One aspect of the detection method of the present disclosure is to acquire an image including a living body, detect a feature figure corresponding to the first portion of a substantially circular shape in the living body from the image, and detect a feature figure corresponding to the first portion of the living body in the living body.
  • the feature point corresponding to the second part is detected.
  • One aspect of the computer program of the present disclosure is to acquire an image including a living body, detect a feature figure corresponding to a substantially circular first portion in the living body from the image, and detect a feature figure corresponding to the first portion of the living body, and surround the first portion in the living body. Operate a computer that detects the feature points corresponding to the second part.
  • FIG. 1 is a block diagram showing a hardware configuration of the detection system according to the first embodiment.
  • the detection system 10 includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, and a storage device 14.
  • the detection system 10 may further include an input device 15 and an output device 16.
  • the processor 11, the RAM 12, the ROM 13, the storage device 14, the input device 15, and the output device 16 are connected via the data bus 17.
  • Processor 11 reads a computer program.
  • the processor 11 is configured to read a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14.
  • the processor 11 may read a computer program stored in a computer-readable recording medium by using a recording medium reading device (not shown).
  • the processor 11 may acquire (ie, read) a computer program from a device (not shown) located outside the detection system 10 via a network interface.
  • the processor 11 controls the RAM 12, the storage device 14, the input device 15, and the output device 16 by executing the read computer program.
  • a functional block for detecting a part of a living body from an image is realized in the processor 11.
  • processor 11 a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (field-programmable gate array), a DSP (Demand-Side Platform), and an ASIC (Application) are used. Alternatively, a plurality of them may be used in parallel.
  • CPU Central Processing Unit
  • GPU Graphics Processing Unit
  • FPGA field-programmable gate array
  • DSP Demand-Side Platform
  • ASIC Application Specific integrated circuit
  • the RAM 12 temporarily stores the computer program executed by the processor 11.
  • the RAM 12 temporarily stores data temporarily used by the processor 11 while the processor 11 is executing a computer program.
  • the RAM 12 may be, for example, a D-RAM (Dynamic RAM).
  • the ROM 13 stores a computer program executed by the processor 11.
  • the ROM 13 may also store fixed data.
  • the ROM 13 may be, for example, a P-ROM (Programmable ROM).
  • the storage device 14 stores the data stored in the detection system 10 for a long period of time.
  • the storage device 14 may operate as a temporary storage device of the processor 11.
  • the storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.
  • the input device 15 is a device that receives an input instruction from the user of the detection system 10.
  • the input device 15 may include, for example, at least one of a keyboard, a mouse and a touch panel.
  • the output device 16 is a device that outputs information about the detection system 10 to the outside.
  • the output device 16 may be a display device (for example, a display) capable of displaying information about the detection system 10.
  • FIG. 2 is a block diagram showing a functional configuration of the detection system according to the first embodiment.
  • the detection system 10 is configured to detect a part of a living body from an image.
  • the detection system 10 includes an image acquisition unit 110 and a detection unit 120 as a processing block for realizing the function or as a physical processing circuit.
  • the image acquisition unit 110 and the detection unit 120 can be realized by, for example, the processor 11 (see FIG. 1) described above.
  • the image acquisition unit 110 is configured to be able to acquire an image (that is, an image to be detected) input to the detection system 10.
  • the image acquisition unit 110 may include a storage unit that stores acquired images. Information about the image acquired by the image acquisition unit 110 is output to the detection unit 120.
  • the detection unit 120 is configured to be able to detect a part of a living body from an image acquired by the image acquisition unit 110. Specifically, the detection unit 110 is configured to be able to detect a feature figure corresponding to the first part of the living body and a feature point corresponding to the second part of the living body.
  • the "first part” here is a part having a substantially circular shape in a living body.
  • the "second part” is a part located around the first part in the living body. It should be noted that which part of the living body is to be the first part and which part is to be the second part may be set in advance. In this case, a plurality of parts having different types may be set as the first part, or a plurality of parts having different types may be set as the second part.
  • the detection unit 120 may have a function of outputting information regarding the detected feature points and feature figures.
  • FIG. 3 is a flowchart showing the operation flow of the detection system according to the first embodiment.
  • the image acquisition unit 110 first acquires an image (step S101).
  • the detection unit 120 detects the feature figure corresponding to the first portion from the image acquired by the image acquisition unit 110 (step S102).
  • the detection unit 120 further detects a feature point corresponding to the second portion from the image acquired by the image acquisition unit 110 (step S103).
  • the detected feature figure and feature point can be represented by coordinates, mathematical formulas, or the like.
  • the processes of steps S102 and S103 may be executed before and after each other, or may be executed in parallel at the same time. That is, the order of detection of the feature figure corresponding to the first part and the feature point corresponding to the second part is not limited, and the feature figure and the feature point may be detected at the same time.
  • FIG. 4 is a diagram showing an example of a feature figure detected by the detection system according to the first embodiment.
  • the detection unit 120 detects a circle (including an ellipse) as a feature figure corresponding to the second portion.
  • the circle detected by the detection unit 120 may include a vertically elongated ellipse, a horizontally elongated ellipse, and an oblique ellipse (that is, an ellipse rotated at an arbitrary angle) in addition to a perfect circle.
  • an oblique ellipse that is, an ellipse rotated at an arbitrary angle
  • what kind of circle is actually detected may be set in advance.
  • the shape may be set according to the part of the living body to be detected.
  • the detection unit 120 may be configured to be able to detect a circle that is partially missing or a circle that is partially hidden.
  • the feature figure corresponding to the first portion and the feature point corresponding to the second portion are detected from the image. That is, the first part and the second part are detected by different methods. This makes it possible to appropriately detect the first portion and the second portion having different shape characteristics from the image of the living body. ..
  • the detection system 10 according to the second embodiment will be described with reference to FIGS. 5 to 7.
  • the second embodiment differs from the first embodiment described above only in a part of the configuration and operation.
  • the hardware configuration may be the same as that of the first embodiment (see FIG. 1). Therefore, in the following, the description of the parts that overlap with the embodiments already described will be omitted as appropriate.
  • FIG. 5 is a block diagram showing a functional configuration of the detection system according to the second embodiment.
  • the same reference numerals are given to the same components as those shown in FIG. 2.
  • the detection system 10 has an image acquisition unit 110, a detection unit 120, and iris recognition as a processing block for realizing the function or as a physical processing circuit. It is provided with a unit 130. That is, the detection system 10 according to the second embodiment is configured to further include an iris recognition unit 130 in addition to the components of the first embodiment (see FIG. 2).
  • the iris recognition unit 130 can be realized by, for example, the processor 11 (see FIG. 1) described above.
  • the iris authentication unit 130 is configured to be able to execute iris authentication using the feature points and feature figures detected by the detection unit 120.
  • the iris recognition unit 130 identifies the iris region based on, for example, a feature point corresponding to the eyelid, which is an example of the first part, and a feature figure (see FIG. 7) corresponding to the iris and the pupil, which is an example of the second part.
  • the iris recognition unit 130 configured to be able to execute the iris recognition process using the iris region may have a function of outputting the result of the iris recognition.
  • the iris authentication unit 130 may be configured to execute a part of the iris authentication process outside the system (for example, execute it on an external server, a cloud, or the like).
  • FIG. 6 is a flowchart showing the operation flow of the detection system according to the second embodiment.
  • the same reference numerals are given to the same processes as those shown in FIG.
  • the image acquisition unit 110 first acquires an image (step S101). After that, the detection unit 120 detects the feature figure corresponding to the first portion from the image acquired by the image acquisition unit 110 (step S102). The detection unit 120 further detects a feature point corresponding to the second portion from the image acquired by the image acquisition unit 110 (step S103).
  • the iris recognition unit 130 identifies the eyelid region (that is, the region where the eyelid exists) from the feature points corresponding to the eyelid, and generates a mask for the eyelid region (step S201).
  • the eyelid area mask is used to remove the eyelid area that is unnecessary for iris recognition (in other words, has no iris information).
  • the iris recognition unit 130 identifies the iris region (that is, the region from which the iris information can be obtained) from the feature figures corresponding to the iris and the pupil, and executes iris recognition using the iris region (step S202).
  • the existing technology can be appropriately adopted, and therefore detailed description thereof is omitted here.
  • FIG. 7 is a diagram showing an example of detecting a feature figure and a feature point by the detection system according to the second embodiment.
  • the "iris and pupil” are detected as the first part, and the “eyelid” is detected as the second part.
  • the detection unit 120 detects a circle corresponding to the iris and a circle corresponding to the pupil.
  • the detection unit 120 may detect only one of the circle corresponding to the iris and the circle corresponding to the pupil. Since the shape of the iris and pupil is close to a circle, it is suitable for detecting as a characteristic figure of a substantially circle. If an iris or pupil is to be detected at a feature point (for example, a point on the circumference), the number and position of the points depend on the system design and directly affect the detection accuracy. However, if the iris or pupil is detected as a circle, the position of the iris or pupil can be determined as a circular equation. Since the formula of the circle is uniquely determined, it does not depend on the system design and does not affect the detection accuracy. From this point as well, it can be said that the iris is suitable for detection in a circle.
  • the detection unit 120 detects a plurality of feature points indicating the position (contour) of the eyelid.
  • the detection unit 120 detects two feature points corresponding to the inner and outer corners of the eye, three feature points corresponding to the upper eyelid, and three feature points corresponding to the lower eyelid.
  • the number of feature points described above is only an example, and a feature point smaller than this may be detected, or a feature point larger than this may be detected.
  • the individual differences in the eyelids in the living body are relatively large, and the shape of the eyelids varies greatly depending on the individual, such as single eyelids, double eyelids, eyelids and sagging eyes. Therefore, it is suitable for detecting as a feature point, not as a feature figure.
  • the shape of the eyelids varies from person to person, it is common that they are located around the iris and pupil. Therefore, if it is detected together with the feature figure, it can be detected relatively easily as a feature point.
  • iris recognition is executed using the detected feature figures and feature points.
  • this embodiment in particular, since a plurality of parts existing around the eyes are appropriately detected, it is possible to appropriately perform iris recognition.
  • the ratio of the distance from the inner corner of the eye to the outer corner of the eye and the difference between the radii of the two circles is determined. Data matching or weighting in iris recognition may be used.
  • the detection system 10 according to the third embodiment will be described with reference to FIGS. 8 and 9.
  • the third embodiment differs from each of the above-described embodiments only in a part of the configuration and operation.
  • the hardware configuration may be the same as that of the first embodiment (see FIG. 1). Therefore, in the following, the description of the parts that overlap with the embodiments already described will be omitted as appropriate.
  • FIG. 8 is a block diagram showing a functional configuration of the detection system according to the third embodiment.
  • the same components as those shown in FIGS. 2 and 5 are designated by the same reference numerals.
  • the detection system 10 has an image acquisition unit 110, a detection unit 120, and a line-of-sight estimation as a processing block for realizing the function or as a physical processing circuit. It is provided with a unit 140. That is, the detection system 10 according to the third embodiment is configured to further include a line-of-sight estimation unit 140 in addition to the components of the first embodiment (see FIG. 2).
  • the line-of-sight estimation unit 140 can be realized by, for example, the processor 11 (see FIG. 1) described above.
  • the line-of-sight estimation unit 140 is configured to be able to perform line-of-sight direction estimation using the feature points and feature figures detected by the detection unit 120. Specifically, the line-of-sight estimation unit 140 estimates the line-of-sight direction based on the "feature points corresponding to the eyelids" and the "feature figures corresponding to the iris and the pupil" (see FIG. 7) described in the second embodiment. It is configured to be able to execute the processing to be performed. The feature points corresponding to the eyelids and the feature figures corresponding to the iris and the pupil may be detected as described in the second embodiment.
  • the line-of-sight estimation unit 140 may have a function of outputting the result of line-of-sight estimation. Further, the line-of-sight estimation unit 140 may be configured to execute a part of the line-of-sight estimation process outside the system (for example, execute it on an external server, cloud, or the like).
  • FIG. 9 is a flowchart showing the operation flow of the detection system according to the third embodiment.
  • the same reference numerals are given to the same processes as those shown in FIGS. 3 and 6.
  • the image acquisition unit 110 first acquires an image (step S101). After that, the detection unit 120 detects the feature figure corresponding to the first portion from the image acquired by the image acquisition unit 110 (step S102). The detection unit 120 further detects a feature point corresponding to the second portion from the image acquired by the image acquisition unit 110 (step S103).
  • the line-of-sight estimation unit 140 estimates the line-of-sight direction based on the feature figure and the feature point (step S301).
  • the specific processing contents of the line-of-sight direction estimation will be described in detail in the fourth embodiment described later.
  • the line-of-sight estimation is executed using the detected feature figures and feature points.
  • the portion used for estimating the line-of-sight direction (for example, each part around the eyes) is appropriately detected, it is possible to appropriately estimate the line-of-sight direction.
  • the detection system 10 according to the fourth embodiment will be described with reference to FIG.
  • the fourth embodiment shows a more specific configuration (that is, a specific estimation method in the line-of-sight direction) of the third embodiment described above, and the configuration and the flow of operations are described in the third embodiment. It may be the same as (see FIGS. 8 and 9). Therefore, in the following, the description of the portion overlapping with the portion already described will be omitted as appropriate.
  • FIG. 10 is a diagram showing an example of a line-of-sight estimation method by the detection system according to the fourth embodiment.
  • the line-of-sight direction is estimated using the feature points corresponding to the eyelids and the feature figures corresponding to the iris and the pupil.
  • the feature points corresponding to the eyelids and the feature figures corresponding to the iris and the pupil may be detected as described in the second embodiment (see FIG. 7).
  • Feature points 1 and 2 in the figure are feature points corresponding to the inner and outer corners of the eyes, respectively. Further, the feature point 3 and the feature point 4 are points that intersect the eyelids on the midline of the feature point 1 and the feature point 2. Therefore, the feature points 1 to 4 are in the same position regardless of the direction in which the eyeball is facing, as long as the direction of the face does not change. When the direction of the eyeball is changed, the circles 5 and 6 which are the characteristic figures corresponding to the iris and the pupil move. Therefore, the direction of the eyeball (that is, the line-of-sight direction) can be estimated by using the relative relationship between the position of the eyelid that can be specified from each feature point and the position of the iris and the pupil that can be specified from each feature figure.
  • the relative relationship between the position of the eyelids and the position of the eyeball should be obtained in advance and the relationship with which position is currently being viewed. This association may be calculated as a function or created as a table.
  • the line-of-sight direction When calculating the line-of-sight direction, first normalize the image using the circle 6 corresponding to the iris. Next, the origin is the intersection of the line connecting the feature points 1 and 2 and the line connecting the feature points 3 and 4, and the information on how many pixels are separated from this origin in the x and y directions. , Calculate the relative positional relationship between the eyelids and the eyeballs. Then, the line-of-sight direction is estimated using the calculated positional relationship between the eyelid and the eyeball.
  • the line-of-sight direction is estimated from the positional relationship between the eyelids and the eyeballs.
  • the relative positional relationship between the feature points corresponding to the eyelids and the feature figures corresponding to the iris and the pupil can be appropriately calculated, so that the line-of-sight direction can be appropriately estimated. ..
  • the detection system 10 according to the fifth embodiment will be described with reference to FIGS. 11 to 13.
  • the fifth embodiment differs from each of the above-described embodiments only in a part of the configuration and operation.
  • the hardware configuration may be the same as that of the first embodiment (see FIG. 1). Therefore, in the following, the description of the parts that overlap with the embodiments already described will be omitted as appropriate.
  • FIG. 11 is a block diagram showing a functional configuration of the detection system according to the fourth embodiment.
  • the same reference numerals are given to the same components as those shown in FIGS. 2, 5, and 8.
  • the detection system 10 has an image acquisition unit 110, a detection unit 120, and an angle of rotation as a processing block for realizing the function or as a physical processing circuit. It includes an estimation unit 150 and an image rotation unit 160. That is, the detection system 10 according to the fifth embodiment is configured to further include a rotation angle estimation unit 150 and an image rotation unit 160 in addition to the components of the first embodiment (see FIG. 2).
  • the rotation angle estimation unit 150 and the image rotation unit 160 can be realized by, for example, the processor 11 (see FIG. 1) described above.
  • the rotation angle estimation unit 150 can estimate the rotation angle (that is, the inclination) of the image acquired by the image acquisition unit 110 based on the feature points detected by the detection unit 120. For example, as shown in the second embodiment or the like, when the feature point of the eyelid is detected (see FIG. 7 or the like), the image acquisition unit 110 estimates the rotation angle of the image from the detected inclination of the eyelid. .. The rotation angle estimation unit 150 may estimate the rotation angle of the image in consideration of the feature figure in addition to the feature points detected by the detection unit 120. For example, as shown in the second embodiment or the like, when the feature point of the eyelid and the feature figure of the iris or the pupil are detected (see FIG. 7 etc.), the image acquisition unit 110 determines that the detected eyelid. The angle of rotation of the image may be estimated from the positional relationship with the iris and the pupil.
  • the image rotation unit 160 is configured to be able to rotate the image acquired by the image acquisition unit 110 based on the rotation angle estimated by the rotation angle estimation unit 150. That is, the image rotation unit 160 is configured to be able to perform tilt correction of the image based on the estimated rotation angle.
  • the image rotation unit 160 may have a function of storing the rotated image as a corrected image.
  • FIG. 12 is a flowchart showing the operation flow of the detection system according to the fifth embodiment.
  • the same reference numerals are given to the same processes as those shown in FIGS. 3, 6, and 9.
  • the image acquisition unit 110 first acquires an image (step S101). After that, the detection unit 120 detects the feature figure corresponding to the first portion from the image acquired by the image acquisition unit 110 (step S102). The detection unit 120 further detects a feature point corresponding to the second portion from the image acquired by the image acquisition unit 110 (step S103).
  • the rotation angle estimation unit 150 estimates the rotation angle of the image based on the detected feature points (step S401). Then, the image rotation unit 160 rotates the image according to the estimated rotation angle (step S402). In particular, the image rotation unit 160 rotates the image about the center of the substantially circle detected as the feature figure as the rotation axis.
  • FIG. 13 is a diagram showing a specific operation example by the detection system according to the fifth embodiment.
  • the rotation angle of the image is estimated from the feature points of the eyelids.
  • the image is tilted to the left (counterclockwise). It should be noted that it can be calculated by comparing the numerical value of the rotation angle, for example, the position of the feature point in the normal state set in advance with the position of the feature point detected this time.
  • existing techniques can be appropriately adopted.
  • the image rotation unit 160 rotates the image by the estimated rotation angle.
  • the image rotation unit 160 rotates the image to the right (clockwise).
  • the image rotation unit 160 rotates the image about the center of the circle corresponding to the iris or the pupil detected as the feature figure as the rotation axis.
  • the image rotation unit 160 rotates the image with the center of one of the feature figures as the rotation axis. Just do it. (Technical effect) Next, the technical effect obtained by the detection system 10 according to the fifth embodiment will be described.
  • the rotation angle of the image is estimated based on the detected feature points, and the image is rotated about the center of the feature figure as the rotation axis. To. By doing so, even if the image acquired by the image acquisition unit 110 is tilted, the tilt can be appropriately corrected.
  • the rotated image can also be used, for example, for the iris recognition described in the third embodiment and the line-of-sight estimation described in the fourth embodiment. In this case, since the tilt is corrected by the rotation of the image, iris recognition and line-of-sight estimation can be performed with higher accuracy.
  • the detection system 10 according to the sixth embodiment will be described with reference to FIGS. 14 to 20.
  • the sixth embodiment differs from each of the above-described embodiments only in a part of the configuration and operation.
  • the hardware configuration may be the same as that of the first embodiment (see FIG. 1). Therefore, in the following, the description of the parts that overlap with the embodiments already described will be omitted as appropriate.
  • FIG. 14 is a block diagram showing a functional configuration of the detection system according to the sixth embodiment.
  • the same reference numerals are given to the same components as those shown in FIGS. 2, 5, 8 and 11.
  • the detection system 10 has an image acquisition unit 110, a detection unit 120, and a display unit as a processing block for realizing the function or as a physical processing circuit. It is equipped with 170. That is, the detection system 10 according to the sixth embodiment is configured to further include a display unit 170 in addition to the components of the first embodiment (see FIG. 2).
  • the display unit 170 is configured as, for example, a monitor having a display.
  • the display unit 170 may be configured as a part of the output device 16 shown in FIG.
  • the display unit 170 is configured to be able to display information on feature figures and feature points detected by the detection unit 120.
  • the display unit 170 may be configured so that its display mode can be changed by, for example, an operation of a system user.
  • FIG. 15 is a flowchart showing the operation flow of the detection system according to the seventh embodiment.
  • the same reference numerals are given to the same processes as those shown in FIGS. 3, 6, 9, and 12.
  • the image acquisition unit 110 first acquires an image (step S101). After that, the detection unit 120 detects the feature figure corresponding to the first portion from the image acquired by the image acquisition unit 110 (step S102). The detection unit 120 further detects a feature point corresponding to the second portion from the image acquired by the image acquisition unit 110 (step S103).
  • the display unit 170 displays information on the detected feature figure and feature point (step S501).
  • the display unit 170 may display not only the information directly related to the feature figure and the feature point, but also the information that can be estimated from the feature figure and the feature point.
  • FIG. 16 is a diagram (No. 1) showing a display example of a feature point and a feature figure in the display unit.
  • the display unit 170 may display an image drawn by superimposing feature figures and feature points.
  • the display unit 170 may display only the feature figure or only the feature point.
  • the display unit 170 may switch between drawing and non-drawing of feature figures and feature points by, for example, a user's operation.
  • the display unit 170 may display an operation button (that is, a button for switching the display) at the bottom of the image or the like.
  • the display unit 170 may further display information indicating the position of the feature figure and the feature point (for example, the coordinates of the feature point, the formula of the feature figure, etc.). Further, the display unit 170 paints a color or draws a boundary line so that the range of the region (in the example of the figure, the eyelid region, the iris region, and the pupil region) that can be identified from the feature figure and the feature point can be identified. May be displayed.
  • FIG. 17 is a diagram (No. 2) showing a display example of a feature point and a feature figure in the display unit.
  • the display unit 170 may display the original image (that is, the input image) and the detection result (that is, the image in which the feature figure and the feature point are drawn on the input image) side by side.
  • the display unit 170 may display only one of the original image and the detection result, for example, by the operation of the user. Further, the display modes of the original image and the detection result may be changed separately.
  • FIG. 18 is a diagram (No. 3) showing a display example of a feature point and a feature figure in the display unit.
  • the display example of FIG. 18 assumes a second embodiment (that is, a configuration including the iris recognition unit 130).
  • the display unit 170 may display the registered image for iris recognition and the captured image this time (that is, an image in which a feature figure and a feature point are drawn on an input image) side by side.
  • the display unit 170 may display only one of the registered image and the captured image, for example, by the operation of the user. Further, the display mode of each of the registered image and the captured image may be changed separately.
  • FIG. 19 is a diagram (No. 4) showing a display example of a feature point and a feature figure in the display unit.
  • the display example of FIG. 19 assumes the third and fourth embodiments (that is, the configuration including the line-of-sight estimation unit 140).
  • the display unit 170 may display the estimation result in the line-of-sight direction in addition to the image drawn by superimposing the feature figure and the feature point. Specifically, the display unit 170 may display an arrow indicating the line-of-sight direction as shown in the figure. In this case, the larger the deviation from the front of the line of sight, the longer or larger the arrow may be displayed.
  • FIG. 20 is a diagram (No. 5) showing a display example of a feature point and a feature figure in the display unit.
  • the display example of FIG. 20 assumes a fifth embodiment (that is, a configuration including a rotation angle estimation unit 150 and an image rotation unit 160).
  • the display unit 170 displays the image before rotation (that is, the image before correcting the tilt) and the image after rotation (that is, the image after correcting the tilt) side by side. May be good.
  • the display unit 170 may display only one of the image before rotation and the image after rotation by, for example, a user's operation. Further, the display mode of the image before rotation or the image after rotation may be changed separately.
  • the detection system 10 As described with reference to FIGS. 14 to 20, in the detection system 10 according to the sixth embodiment, information on the detected feature figure and feature point is displayed. Therefore, the detection result of the feature figure and the feature point and the result of various processing using the feature figure and the feature point can be presented to the user in an easy-to-understand manner.
  • the detection system 10 according to the seventh embodiment will be described with reference to FIGS. 21 and 22. It should be noted that the seventh embodiment differs from each of the above-described embodiments only in a part of the configuration and operation, and for example, the hardware configuration may be the same as that of the first embodiment (see FIG. 1). Therefore, in the following, the description of the parts that overlap with the embodiments already described will be omitted as appropriate.
  • FIG. 21 is a block diagram showing a functional configuration of the detection system according to the seventh embodiment.
  • the same reference numerals are given to the same components as those shown in FIGS. 2, 5, 8, 11, and 14.
  • the detection system 10 according to the seventh embodiment has an image acquisition unit 110, a detection unit 120, and a learning unit as a processing block for realizing the function or as a physical processing circuit. It is equipped with 180. That is, the detection system 10 according to the seventh embodiment is configured to further include a learning unit 180 in addition to the components of the first embodiment (see FIG. 2).
  • the learning unit 180 is configured to be able to learn a model for detecting a feature figure and a feature point (for example, a model of a neural network).
  • the image acquisition unit 110 acquires an image which is training data.
  • the learning unit 180 executes learning using the feature figures and feature points detected by the detection unit 120 from the training data. That is, the learning unit 180 executes learning using the detected feature figure and feature point as a composite target. More specifically, the learning unit 180 compares the correct answer data of the feature figure and the feature point input as training data with the feature figure and the feature point detected by the detection unit 120, and executes learning.
  • the learning unit 180 may be configured to execute a part of the learning process outside the system (for example, execute it on an external server, a cloud, or the like).
  • the detection system 10 may have a function of expanding the input training data.
  • the image acquisition unit 110 may perform data expansion such as luminance change, vertical / horizontal shift, enlargement / reduction, and rotation.
  • FIG. 22 is a flowchart showing the operation flow of the detection system according to the seventh embodiment.
  • the same reference numerals are given to the same processes as those shown in FIGS. 3, 6, 9, 12, and 15.
  • the image acquisition unit 110 first acquires an image (step S101). After that, the detection unit 120 detects the feature figure corresponding to the first portion from the image acquired by the image acquisition unit 110 (step S102). The detection unit 120 further detects a feature point corresponding to the second portion from the image acquired by the image acquisition unit 110 (step S103).
  • the learning unit 180 calculates an error function from the detected feature figure and feature point (step S601). Specifically, the learning unit 180 calculates the distance between the detected feature figure and the vector showing the feature point and the feature figure of the training data (that is, the correct answer data) and the vector showing the feature point, so that the error between the two is calculated. To calculate.
  • the error calculation method for example, the L1 norm and the L2 norm can be used, but other methods may be used.
  • the learning unit 180 performs error back propagation based on the error and calculates the gradient of the parameters of the detection model (step S602). After that, the learning unit 180 updates (optimizes) the parameters of the detection model based on the calculated gradient (step S603).
  • the optimization method for example, a method such as SDG (Stochastic Gradient Descent) or Adam can be used, but other methods may be used for optimization.
  • the learning unit 180 may perform regularization such as Weight Decay (weight attenuation) when optimizing the parameters.
  • the detection model is a neural network, it may include a layer for regularization such as dropout and batchnorm.
  • steps S601 to S603 are merely examples, and learning may be executed using another method as long as the feature figure and the feature point can be used as a composite target. ..
  • the learning unit 180 determines whether or not the learning is completed (step S604).
  • the learning unit 180 determines whether or not the learning is completed, for example, depending on whether or not the processes up to this point have looped a predetermined number of times.
  • step S604: YES the series of processes is completed.
  • step S604: NO the process is repeated from step S101.
  • learning is executed with the feature figure and the feature point as a composite target. Therefore, it is possible to optimize the detection model of the feature figure and the feature point and realize more appropriate detection.
  • the learning unit 180 executes the learning process by using the information regarding the distribution of the relative positional relationship between the feature figure and the feature point. For example, the learning unit 180 learns a model for detecting a feature figure and a feature point by using the distribution of the position of the iris detected as the feature figure and the position of the eyelid detected as the feature point.
  • each of the iris and the eyelid will be detected independently (that is, the relative positional relationship is taken into consideration). Therefore, there is a possibility that the part that is not the iris is detected as the iris, or the part that is not the eyelid is detected as the eyelid.
  • Appendix 1 The detection system according to Appendix 1 detects an image including a living body and a feature figure corresponding to a substantially circular first portion in the living body from the image, and detects a feature figure corresponding to the first portion of the living body, and the periphery of the first portion in the living body.
  • the detection system is provided with a detection means for detecting a feature point corresponding to the second portion of the above.
  • Appendix 2 The detection system according to Appendix 2 is based on the feature figure corresponding to at least one of the iris and the pupil, which is the first part, and the feature point corresponding to the eyelid, which is the second part, with respect to the living body.
  • the detection system according to Appendix 3 is based on the feature figure corresponding to at least one of the iris and the pupil, which is the first part, and the feature point corresponding to the eyelid, which is the second part, of the living body.
  • the detection system according to Appendix 1 or 2 further comprising a line-of-sight estimation means for executing a line-of-sight estimation process for estimating a line-of-sight.
  • the line-of-sight estimation means is based on the relative positional relationship between the feature figure corresponding to at least one of the iris and the pupil and the feature point corresponding to the eyelid.
  • the detection system according to Appendix 5 uses the feature points corresponding to the eyelids, which is the second part, to estimate the rotation angle of the image, and the iris and pupil, which are the first parts.
  • Appendix 7 The detection system according to Appendix 7 is further provided with a learning means for executing a learning process of the detection means by using the feature points and the feature figures detected from the image which is training data.
  • the detection system according to any one of 6 to 6.
  • Appendix 8 The detection system according to the appendix 8 is described in the appendix 6, wherein the learning means executes the learning process by using the information regarding the relative positional relationship between the feature figure and the feature point. It is a detection system.
  • the detection method according to the appendix 9 acquires an image including a living body, detects a feature figure corresponding to the first portion of a substantially circular shape in the living body from the image, and detects a second portion around the first portion in the living body. It is a detection method characterized by detecting a feature point corresponding to a portion.
  • the computer program according to the appendix 10 acquires an image including a living body, detects a feature figure corresponding to the first portion of a substantially circular shape in the living body from the image, and detects a feature figure corresponding to the first portion of the substantially circular shape in the living body, and a second around the first portion in the living body. It is a computer program characterized by operating a computer so as to detect a feature point corresponding to a part.
  • Appendix 11 The recording medium described in Appendix 11 is a recording medium characterized in that the computer program described in Appendix 10 is recorded.
  • Detection system 11 Processor 110 Image acquisition unit 120 Detection unit 130 Iris recognition unit 140 Line-of-sight estimation unit 150 Rotation angle estimation unit 160 Image rotation unit 170 Display unit 180 Learning unit

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Ophthalmology & Optometry (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Geometry (AREA)
  • Image Analysis (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
PCT/JP2020/034900 2020-09-15 2020-09-15 検出システム、検出方法、及びコンピュータプログラム Ceased WO2022059066A1 (ja)

Priority Applications (5)

Application Number Priority Date Filing Date Title
JP2022550069A JP7666823B2 (ja) 2020-09-15 2020-09-15 検出システム、検出方法、及びコンピュータプログラム
PCT/JP2020/034900 WO2022059066A1 (ja) 2020-09-15 2020-09-15 検出システム、検出方法、及びコンピュータプログラム
US17/640,160 US11837023B2 (en) 2020-09-15 2020-09-15 Detection system, detection method, and computer program
JP2024227884A JP7798165B2 (ja) 2020-09-15 2024-12-24 検出システム、学習方法、及びコンピュータプログラム
JP2025280675A JP2026040643A (ja) 2020-09-15 2025-12-24 検出システム、検出方法、及びコンピュータプログラム

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/034900 WO2022059066A1 (ja) 2020-09-15 2020-09-15 検出システム、検出方法、及びコンピュータプログラム

Publications (1)

Publication Number Publication Date
WO2022059066A1 true WO2022059066A1 (ja) 2022-03-24

Family

ID=80775571

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/034900 Ceased WO2022059066A1 (ja) 2020-09-15 2020-09-15 検出システム、検出方法、及びコンピュータプログラム

Country Status (3)

Country Link
US (1) US11837023B2 (https=)
JP (3) JP7666823B2 (https=)
WO (1) WO2022059066A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024142286A1 (ja) * 2022-12-27 2024-07-04 日本電気株式会社 情報処理システム、学習装置、照合装置、情報処理方法及び記録媒体

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4441696A4 (en) * 2022-05-27 2025-04-02 Samsung Electronics Co., Ltd. METHOD AND ELECTRONIC DEVICE FOR TILT CORRECTION OF VIDEOS

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003132355A (ja) * 2001-10-25 2003-05-09 Matsushita Electric Ind Co Ltd 虹彩認証方法及びその装置
JP2006004294A (ja) * 2004-06-18 2006-01-05 Oki Electric Ind Co Ltd 個体識別装置および個体識別方法
JP2006172209A (ja) * 2004-12-16 2006-06-29 Advanced Telecommunication Research Institute International 視線方向の推定装置、視線方向の推定方法およびコンピュータに当該視線方向の推定方法を実行させるためのプログラム
JP2008152608A (ja) * 2006-12-19 2008-07-03 Oki Electric Ind Co Ltd 生体情報処理装置
JP2018124733A (ja) * 2017-01-31 2018-08-09 ソニー株式会社 電子機器、情報処理方法およびプログラム

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4008282B2 (ja) 2002-04-24 2007-11-14 沖電気工業株式会社 瞳孔・虹彩円検出装置
JP4260728B2 (ja) 2004-11-26 2009-04-30 住友重機械工業株式会社 昇降装置
JP4449723B2 (ja) * 2004-12-08 2010-04-14 ソニー株式会社 画像処理装置、画像処理方法、およびプログラム
JP4414401B2 (ja) 2006-02-10 2010-02-10 富士フイルム株式会社 顔特徴点検出方法および装置並びにプログラム
JP2010033305A (ja) * 2008-07-29 2010-02-12 Hitachi Ltd 画像情報処理方法、及び装置
US8345922B2 (en) * 2008-09-03 2013-01-01 Denso Corporation Apparatus for detecting a pupil, program for the same, and method for detecting a pupil
JP4893863B1 (ja) * 2011-03-11 2012-03-07 オムロン株式会社 画像処理装置、および画像処理方法
JP4893862B1 (ja) * 2011-03-11 2012-03-07 オムロン株式会社 画像処理装置、および画像処理方法
JP5975828B2 (ja) 2012-10-05 2016-08-23 セコム株式会社 顔特徴抽出装置および顔認証システム
JP6751324B2 (ja) * 2016-09-14 2020-09-02 株式会社デンソーアイティーラボラトリ 虹彩検出装置、虹彩検出方法、およびプログラム
CA3071819A1 (en) * 2017-09-01 2019-03-07 Magic Leap, Inc. Detailed eye shape model for robust biometric applications

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003132355A (ja) * 2001-10-25 2003-05-09 Matsushita Electric Ind Co Ltd 虹彩認証方法及びその装置
JP2006004294A (ja) * 2004-06-18 2006-01-05 Oki Electric Ind Co Ltd 個体識別装置および個体識別方法
JP2006172209A (ja) * 2004-12-16 2006-06-29 Advanced Telecommunication Research Institute International 視線方向の推定装置、視線方向の推定方法およびコンピュータに当該視線方向の推定方法を実行させるためのプログラム
JP2008152608A (ja) * 2006-12-19 2008-07-03 Oki Electric Ind Co Ltd 生体情報処理装置
JP2018124733A (ja) * 2017-01-31 2018-08-09 ソニー株式会社 電子機器、情報処理方法およびプログラム

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024142286A1 (ja) * 2022-12-27 2024-07-04 日本電気株式会社 情報処理システム、学習装置、照合装置、情報処理方法及び記録媒体
JPWO2024142286A1 (https=) * 2022-12-27 2024-07-04

Also Published As

Publication number Publication date
JPWO2022059066A1 (https=) 2022-03-24
JP2026040643A (ja) 2026-03-09
JP2025031893A (ja) 2025-03-07
JP7798165B2 (ja) 2026-01-14
JP7666823B2 (ja) 2025-04-22
US11837023B2 (en) 2023-12-05
US20220392260A1 (en) 2022-12-08

Similar Documents

Publication Publication Date Title
JP7798165B2 (ja) 検出システム、学習方法、及びコンピュータプログラム
JP6946831B2 (ja) 人物の視線方向を推定するための情報処理装置及び推定方法、並びに学習装置及び学習方法
US8677282B2 (en) Multi-finger touch adaptations for medical imaging systems
JP4829141B2 (ja) 視線検出装置及びその方法
US20160328825A1 (en) Portrait deformation method and apparatus
JP5664064B2 (ja) 視線検出装置および補正係数算出プログラム
US20040233224A1 (en) Image processor, image processing method, recording medium, computer program and semiconductor device
US20120050269A1 (en) Information display device
US20230377230A1 (en) Style transfer program, style transfer device, and style transfer method
JP2010250420A (ja) 顔の特徴部位の座標位置を検出する画像処理装置
CN120359547A (zh) 视频透视（vst）增强现实（ar）系统中扩展视图几何图形的生成和渲染
US20240119711A1 (en) Learning apparatus, estimation apparatus, learning method, estimation method, and program and non-transitory storage medium
US20130195323A1 (en) System for Generating Object Contours in 3D Medical Image Data
JP2019159787A (ja) 人物検出方法および人物検出プログラム
JP2021064043A (ja) 画像処理装置、画像処理システム、画像処理方法及び画像処理プログラム
US12499642B2 (en) Image processing apparatus, image processing method, and computer-readable recording medium
US20260037071A1 (en) Recording Medium, Image Processing Device, Image Processing Method, and In-Vehicle Interface Device
JP7765611B2 (ja) 情報処理装置、情報処理方法及びプログラム
US10080963B2 (en) Object manipulation method, object manipulation program, and information processing apparatus
WO2023102356A1 (en) Visual field map expansion
CN115798026A (zh) 一种眼球活体检测方法、装置、存储介质和计算机设备
CN118570277B (zh) 面部器官的闭合度计算方法及装置、介质及电子设备
WO2023043442A1 (en) Correcting raw coordinates of facial feature point
CN115187703B (zh) 一种眉妆指导方法、装置、设备及介质
JP6723822B2 (ja) 画像処理装置、画像処理方法及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20954052

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022550069

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20954052

Country of ref document: EP

Kind code of ref document: A1