JP2018077683A - Image processing device, authentication method and authentication program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device, an authentication method and an authentication program capable of authenticating a user who is not imaged at a position where the image processing device is operated but is imaged at a position apart from the image processing device.SOLUTION: An image processing device includes: a camera that outputs an image acquired by image pickup; a storage control unit 59 that stores the image output by the camera; an operation detecting unit 55 that detects an operation input by a user; a first authenticating unit 53 that authenticates the user on the basis of the image picked up and output by the camera after an operation of detecting the operation input by the user; and a second authenticating unit 69 that authenticates the user on the basis of the image stored before the operation when the image picked up and output by the camera after the operation does not contain an object to be authenticated.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an image processing device, an authentication method, and an authentication program, and more particularly, to an image processing device including a camera, an authentication method and an authentication program executed by the image processing device.

  An image processing apparatus represented by an MFP (Multi Function Peripheral) having a plurality of functions such as a fax, a copy, and a scanner is often used by a plurality of people, and in order to specify a user who operates the image processing apparatus, Some have the ability to authenticate users. As a technique for authenticating a user, a technique for performing authentication using an image obtained by imaging a user's face with a camera is known. In this case, since the position and the angle of view where the camera is arranged are fixed, the range captured by the camera is a predetermined range. For this reason, it is necessary to capture the face of the user who operates the MFP with the camera.

  For example, Japanese Patent Laid-Open No. 2015-011537 discloses a first image capturing unit that captures an image of a person who is in an imaging range in an on state, and an image of a human face that is used for user authentication in an on state. A second imaging unit for photographing, and when the person is detected by the human sensor, the first imaging unit is turned on, and the person is approaching based on the image photographed by the first imaging unit. If it is determined, the main system is returned from the sleep mode to the standby mode by supplying power necessary for operation, and from the time when the human sensor detects a person to before the main system returns from the sleep mode. Describes that authentication is performed by turning on the second imaging unit.

  Since the user often operates the MFP in a standing posture, the position and angle of view of the camera are set so that the user's face standing in a position where the MFP can be operated is within the imaging range of the camera. Often set. Also in the technique described in Japanese Patent Application Laid-Open No. 2015-011537, a user who operates the MFP in a standing posture can be authenticated.

However, there may be a difference in the heights of a plurality of users, and a face of a certain user can be imaged, but a face of a user having a height difference from that user may not be imaged. In addition, the height of the face of the user who rides in the wheelchair is lower than the height of the face of the healthy person. Therefore, when the camera imaging range is adjusted to the height of the face of the healthy person, the user who gets in the wheelchair gets closer to the MFP. There is a problem that the user's face falls outside the imaging range of the camera.
Japanese Patent Laying-Open No. 2015-011537

  The present invention has been made to solve the above-described problems, and one of the objects of the present invention is that a user who is not imaged at a position where the image processing apparatus is operated but is imaged at a position away from the image processing apparatus. Is to provide an image processing apparatus capable of authenticating the above.

  Another object of the present invention is to provide an authentication method capable of authenticating a user who is not imaged at a position where the image processing apparatus is operated but is imaged at a position away from the image processing apparatus.

  Still another object of the present invention is to provide an authentication program capable of authenticating a user who is not imaged at a position where the image processing apparatus is operated but is imaged at a position away from the image processing apparatus.

  The present invention has been made to solve the above-described problems. According to one aspect of the present invention, an image processing apparatus stores an image output unit that outputs an image obtained by capturing an image and an image output by the image capture unit. And a first authentication unit that authenticates the user based on an image captured and output by the imaging unit after an operation in which the operation detection unit detects an operation. And a second authentication unit that authenticates the user based on the image stored before the operation when the subject to be authenticated is not included in the image captured and output by the imaging unit after the operation. .

  According to this aspect, the user to be operated is authenticated based on the image obtained after the operation for accepting the operation by the user, and the subject to be authenticated is included in the image obtained after the operation. If not, the user is authenticated based on the image stored before the operation. Therefore, it is possible to provide an image processing apparatus that can authenticate a user who is not imaged at a position where the image processing apparatus is operated but is imaged at a position away from the image processing apparatus.

  Preferably, the image processing device further includes image selecting means for selecting one image from the plurality of stored images.

  According to this aspect, since one is selected from a plurality of images obtained by imaging before operation, user authentication is facilitated.

  Preferably, the image selection means selects an image having the largest area of the face portion of the subject to be authenticated.

  According to this aspect, the authentication is performed based on the image having the largest area of the face portion, so that the authentication accuracy can be made as high as possible.

  Preferably, the image selection means selects an image in which the face of the subject to be authenticated is directed to the front.

  According to this aspect, since authentication is performed based on an image with the face facing the front, the accuracy of authentication can be made as high as possible.

  Preferably, the image selection unit selects an image having the maximum contrast.

  According to this aspect, since authentication is performed based on an image having the maximum contrast, the accuracy of authentication can be made as high as possible.

  Preferably, the image selection means selects an image having the largest area of the skin color portion of the subject to be authenticated.

  According to this aspect, since the skin color portion corresponds to the face portion of the subject, the accuracy of authentication can be made as high as possible.

  Preferably, the image selection means selects an image in which the subject to be authenticated is arranged at the center.

  According to this aspect, since authentication is performed using an image with less distortion, the accuracy of authentication can be made as high as possible.

  Preferably, the image selection unit compares the plurality of images output by the imaging unit at different times, and selects the plurality of images when the area of the face portion of the subject to be authenticated does not have a predetermined change. do not do.

  According to this aspect, authentication is not performed based on an image in which the area of the face portion of the subject to be authenticated does not change in a predetermined manner, so that a subject that is not approaching the image processing apparatus can be prevented from being authenticated.

  Preferably, when the stored image includes a plurality of subjects, the image processing device further includes target specifying means for specifying one of the plurality of subjects as an authentication target.

  According to this aspect, one of the plurality of subjects is specified as an authentication target, so that authentication becomes easy.

  Preferably, the target specifying means specifies a subject having the largest face part area as an authentication target.

  According to this aspect, the subject having the largest area of the face portion among the plurality of subjects is identified as the authentication target, so that the subject closest to the image processing apparatus can be authenticated.

  Preferably, the target specifying unit compares a plurality of images output at different times by the imaging unit, calculates an increase rate of the area of the face portion of each of the plurality of subjects, and a subject having the maximum calculated increase rate Is specified as an authentication target.

  According to this aspect, the subject having the largest increase in the area of the face portion among the plurality of subjects is specified as the authentication target, so that a subject with a fast speed approaching the image processing apparatus can be authenticated.

  Preferably, the imaging means captures images at regular time intervals.

  According to this aspect, since an image obtained by imaging every predetermined time is stored, an image obtained by imaging a user included in the imaging range can be stored before the operation.

  Preferably, a distance detection unit that detects a distance to the subject is further provided, and the imaging unit captures an image every time the subject moves a predetermined distance.

  If this aspect is followed, the image which imaged the user included in the imaging range can be memorize | stored before the time of operation.

  Preferably, when the second image captured after the first image selected by the image selection unit includes at least a part of the subject to be authenticated, the subject to be authenticated included in the first image Is further provided with a composite image generating means for generating a composite image in which a part of the image is complemented by a part of the subject to be authenticated included in the second image, and the second authentication means authenticates the user based on the composite image To do.

  According to this aspect, the second image obtained by imaging a part of the subject included in the first image selected from the plurality of images captured before the imaging is performed after the first image. Since the authentication is performed based on the composite image supplemented with the portion of the subject included in the image, the authentication accuracy can be further increased.

  Preferably, the second authenticating unit includes a predicting unit that predicts an arrival time from when the imaging unit finally outputs an image including the subject to be authenticated before the operation becomes possible until the subject becomes operable. Do not authenticate the user after the predicted arrival time.

  According to this aspect, since the user is not authenticated after the arrival time has elapsed since the subject was last imaged before imaging, it is possible to prevent another user from being authenticated accidentally.

  According to another aspect of the present invention, the authentication method is an authentication method executed by an image processing apparatus including an imaging unit that outputs an image obtained by imaging, and stores an image output by the imaging unit. A storage control step, an operation detection step for detecting an operation input by the user, and an user who operates the operation reception unit are authenticated based on an image captured and output by the imaging unit after the operation is detected. A first authentication step and a step of authenticating a user based on an image stored before the operation when the subject to be authenticated is not included in the image captured and output by the imaging unit after the operation. 2 authentication steps.

  According to this aspect, it is possible to provide an authentication method capable of authenticating a user who is not imaged at a position where the image processing apparatus is operated but is imaged at a position away from the image processing apparatus.

  According to another aspect of the present invention, the authentication program is an authentication program executed by a computer that controls an image processing apparatus including an imaging unit that outputs an image obtained by imaging, and is output by the imaging unit. A storage control step for storing an image, an operation detection step for detecting an operation input by a user, and an operation receiving unit is operated based on an image captured and output by an imaging unit after the operation is detected. Based on the first authentication step for authenticating the user and the image stored before the operation when the subject to be authenticated is not included in the image captured and output by the imaging means after the operation A second authentication step of authenticating the computer.

  According to this aspect, it is possible to provide an authentication program that can authenticate a user who is not imaged at a position where the image processing apparatus is operated but is imaged at a position away from the image processing apparatus.

1 is a perspective view showing an external appearance of an MFP in one embodiment of the present invention. 2 is a block diagram illustrating an outline of a hardware configuration of an MFP. FIG. It is a schematic diagram explaining authentication by MFP in the present embodiment. FIG. 3 is a diagram illustrating an example of functions of a CPU included in an MFP. It is a flowchart which shows an example of the flow of an imaging control process. It is a flowchart which shows an example of the flow of an authentication process.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

  FIG. 1 is a perspective view showing an appearance of an MFP according to one embodiment of the present invention. FIG. 2 is a block diagram showing an outline of the hardware configuration of the MFP. The MFP will be described with reference to FIGS. 1 and 2. An MFP (Multi Function Peripheral) 100 is an example of an image processing apparatus. The main circuit 110, a camera 119 that captures a subject, a document reading unit 130 for reading a document, and a document are conveyed to the document reading unit 130. An automatic document feeder 120, an image forming unit 140 for forming an image on a sheet based on image data, a paper feeding unit 150 for supplying paper to the image forming unit 140, and a user interface And an operation panel 160.

  A camera 119 as an imaging unit is fixedly attached to the main body of the MFP 100 above the operation panel 160. The camera 119 includes a lens and a photoelectric conversion element, and images light collected by the lens on the photoelectric conversion element. The photoelectric conversion element photoelectrically converts the received light and outputs image data to the CPU 111. The photoelectric conversion element is a CMOS (Complementary Metal Oxide Semiconductor) sensor, a CCD (Charge Coupled Device) sensor, or the like. The lens included in the camera 119 is preferably a wide-angle lens. The angle of view of the lens of the camera 119 is fixed. For this reason, the imaging range of the camera 119 is fixed. The angle of view of camera 119 is set so that the imaging range of camera 119 includes the face of the user who operates MFP 100. Specifically, the angle of view and the position of the camera 119 are set so that the imaging range includes the face of the user who operates the operation panel 160 in a standing posture. Since the user's height varies, the angle of view and the arrangement position of the camera 119 may be determined by providing a predetermined range for the height of the user's face. Further, the angle of view and the position of camera 119 are set so that the imaging range includes at least a first distance from MFP 100 so as to include the face of the user on the wheelchair. The first distance is determined by the performance of the camera 119, such as resolution and focal length, and the user's face at a position away from the camera 119 by the first distance is imaged, and the user is determined based on the image output by the camera 119. This is the distance at which the accuracy of the authentication process is a predetermined value or more.

  The automatic document feeder 120 automatically conveys a plurality of documents set on the document tray 125 one by one to the document reading position of the document reading unit 130, and an image formed on the document by the document reading unit 130 The read original is discharged onto the original discharge tray 127. The automatic document feeder 120 includes a document detection sensor that detects a document placed on the document tray 125.

  The document reading unit 130 has a rectangular reading surface for reading a document. The reading surface is formed of platen glass, for example. The automatic document feeder 120 is connected to the main body of the MFP 100 so as to be rotatable about an axis parallel to one side of the reading surface, and can be opened and closed. A document reading unit 130 is disposed below the automatic document feeder 120, and the reading surface of the document reader 130 is exposed when the automatic document feeder 120 is rotated and opened. Therefore, the user can place a document on the reading surface of the document reading unit 130. The automatic document feeder 120 can change a state between an open state in which the reading surface of the document reading unit 130 is exposed and a closed state that covers the reading surface. The automatic document feeder 120 includes a state detection sensor that detects the open state of the automatic document feeder 120.

  The document reading unit 130 includes a light source that emits light and a photoelectric conversion element that receives light, and scans an image formed on a document placed on the reading surface. When a document is placed on the reading area, the light emitted from the light source is reflected by the document, and the reflected light is imaged by the photoelectric conversion element. When the photoelectric conversion element receives the light reflected from the document, the photoelectric conversion element generates image data obtained by converting the received light into an electric signal. Document reading unit 130 outputs image data to CPU 111.

  The paper feed unit 150 conveys the paper stored in the paper feed tray to the image forming unit 140. The image forming unit 140 is controlled by the CPU 111 and forms an image by a well-known electrophotographic method. The image forming unit 140 forms an image on a sheet conveyed by the paper feeding unit 150 based on image data input from the CPU 111. Then, the paper on which the image is formed is discharged to the paper discharge tray 155. The image data output from the CPU 111 to the image forming unit 140 includes image data such as print data received from the outside in addition to the image data input from the document reading unit 130.

  The main circuit 110 includes a CPU (Central Processing Unit) 111 that controls the entire MFP 100, a communication interface (I / F) unit 112, a ROM (Read Only Memory) 113, a RAM (Random Access Memory) 114, A hard disk drive (HDD) 115 as a mass storage device, a facsimile unit 116, a human sensor 117, and an external storage device 118 are included. CPU 111 is connected to camera 119, automatic document feeder 120, document reading unit 130, image forming unit 140, sheet feeding unit 150, and operation panel 160, and controls the entire MFP 100.

  The ROM 113 stores a program executed by the CPU 111 or data necessary for executing the program. The RAM 114 is used as a work area when the CPU 111 executes a program. The RAM 114 temporarily stores image data continuously sent from the document reading unit 130.

  Operation panel 160 is provided on the upper surface of MFP 100. Operation panel 160 includes a display unit 161 and an operation unit 163. The display unit 161 is, for example, a liquid crystal display device (LCD), and displays an instruction menu for a user, information about acquired image data, and the like. For example, an organic EL (electroluminescence) display can be used in place of the LCD as long as it is an apparatus that displays an image.

  The operation unit 163 includes a touch panel 165 and a hard key unit 167. The touch panel 165 is a capacitive type. Note that the touch panel 165 is not limited to the capacitance method, and other methods such as a resistance film method, a surface acoustic wave method, an infrared method, and an electromagnetic induction method can be used.

  The touch panel 165 is provided with its detection surface superimposed on the display unit 161 on the upper or lower surface of the display unit 161. Here, the size of the detection surface of the touch panel 165 and the size of the display surface of the display unit 161 are the same. For this reason, the coordinate system of the display surface and the coordinate system of the detection surface are the same. The touch panel 165 detects the position where the user points the display surface of the display unit 161 on the detection surface, and outputs the coordinates of the detected position to the CPU 111. Since the coordinate system of the display surface and the coordinate system of the detection surface are the same, the coordinates output from the touch panel 165 can be replaced with the coordinates of the display surface.

  Hard key portion 167 includes a plurality of hard keys. The hard key is, for example, a contact switch. The touch panel 165 detects a position designated by the user on the display surface of the display unit 161. When the user operates the MFP 100, the user often takes an upright posture, and thus the display surface of the display unit 161, the operation surface of the touch panel 165, and the hard key unit 167 are arranged facing upward. This is because the user can easily visually recognize the display surface of the display unit 161 and the user can easily instruct the operation unit 163 with a finger.

  Communication I / F unit 112 is an interface for connecting MFP 100 to a network. The communication I / F unit 112 communicates with other computers or data processing devices connected to the network using a communication protocol such as TCP (Transmission Control Protocol) or FTP (File Transfer Protocol). Note that the network to which the communication I / F unit 112 is connected is a local area network (LAN), and the connection form may be wired or wireless. The network is not limited to a LAN, and may be a wide area network (WAN), a public switched telephone network (PSTN), the Internet, or the like.

  The facsimile unit 116 is connected to the public switched telephone network (PSTN) and transmits facsimile data to the PSTN or receives facsimile data from the PSTN. The facsimile unit 116 stores the received facsimile data in the HDD 115, converts it into print data that can be printed by the image forming unit 140, and outputs the print data to the image forming unit 140. As a result, the image forming unit 140 forms an image of facsimile data received by the facsimile unit 116 on a sheet. Further, the facsimile unit 116 converts the data stored in the HDD 115 into facsimile data, and transmits the facsimile data to a facsimile machine connected to the PSTN.

  Human sensor 117 detects a person within a predetermined range from MFP 100 and measures the distance to the person. The human sensor 117 outputs the measured distance to the CPU 111. In the present embodiment, an infrared sensor is used as the human sensor 117. A pyroelectric sensor may be used separately from the infrared sensor as long as a person can be detected and the distance to the detected person can be measured.

  The external storage device 118 is controlled by the CPU 111, and a CD-ROM (Compact Disk Read Only Memory) 118A or a semiconductor memory is mounted. In this embodiment, an example in which the CPU 111 executes a program stored in the ROM 113 will be described. However, the CPU 111 controls the external storage device 118 to read a program to be executed by the CPU 111 from the CD-ROM 118A. The read program may be stored in the RAM 114 and executed.

  The recording medium for storing the program to be executed by the CPU 111 is not limited to the CD-ROM 118A, but is a flexible disk, cassette tape, optical disk (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile Disc). )), A semiconductor memory medium such as an IC card, an optical card, a mask ROM, an EPROM (Erasable Programmable ROM), and an EEPROM (Electrically EPROM). Further, the CPU 111 downloads a program from a computer connected to the network and stores it in the HDD 115, or loads the program stored in the HDD 115 into the RAM 114 so that the computer connected to the network writes the program to the HDD 115. Then, it may be executed by the CPU 111. The program here includes not only a program directly executable by the CPU 111 but also a source program, a compressed program, an encrypted program, and the like.

  FIG. 3 is a schematic diagram for explaining authentication by the MFP in the present embodiment. Referring to FIG. 3, the imaging range of camera 119 when MFP 100 is viewed from the side is indicated by hatching. In the imaging range 300 of the camera 119, the lower limit position becomes higher as it is closer to the MFP 100. For this reason, when the user A getting on the wheelchair is located at a distance L1 from the MFP 100, the face of the user A is within the imaging range of the camera 119. On the other hand, when the user A getting on the wheelchair is located at a distance L2 from the MFP 100 where the operation panel 160 of the MFP 100 can be operated, the face of the user A is out of the imaging range of the camera 119. MFP 100 according to the present embodiment enables face authentication of user A when user A in the wheelchair reaches a position at distance L1 from MFP 100 and operates operation panel 160. Specifically, face authentication is performed on the user based on an image captured when the user is located at a distance L1 from MFP 100.

  FIG. 4 is a diagram illustrating an example of functions of the CPU provided in the MFP. 4 are functions formed in CPU 111 when CPU 111 provided in MFP 100 executes an authentication program stored in ROM 113, HDD 115, or CD-ROM 118A.

  With reference to FIG. 4, the CPU 111 includes an imaging control unit 51, a first authentication unit 53, an operation detection unit 55, a timing determination unit 57, a storage control unit 59, a target identification unit 61, and an image selection unit. 63, a distance detection unit 65, a prediction unit 67, and a second authentication unit 69.

  The imaging control unit 51 controls the camera 119 to cause the camera 119 to capture an image. Since the camera 119 outputs a still image when the image is captured, the imaging control unit 51 acquires the still image output from the camera 119. When a first imaging instruction is input from a first authentication unit 53 described later, the imaging control unit 51 causes the camera 119 to capture an image and outputs a still image output from the camera 119 to the first authentication unit 53. When a second imaging instruction is input from the timing determination unit 57 described later, the imaging control unit 51 causes the camera 119 to capture an image and outputs a still image output from the camera 119 to the storage control unit 59.

  The storage control unit 59 stores the still image in the HDD 115 in association with the imaging date and time in response to the input of the still image from the imaging control unit 51. The imaging date and time may be the date and time when a still image is input from the imaging control unit 51.

  The operation detection unit 55 controls the operation unit 163 to detect an operation input by the user to the operation unit 163. For example, when the user touches the touch panel 165, the touch panel 165 detects the position touched by the user on the display surface of the display unit 161, and outputs position information indicating the detected position. When the position information output from is detected, an operation by the user is detected. In addition, when the user designates one of a plurality of hard keys included in the hard key unit 167, the hard key unit 167 outputs key identification information for identifying the designated hard key. When the key identification information output from the hard key unit 167 is detected, an operation by the user is detected. The operation detection unit 55 detects a user operation when a document detection sensor or an open state detection sensor included in the automatic document feeder 120 is turned on.

  The operation detection unit 55 receives an operation start signal indicating that the user's operation has started in response to detecting the user's operation after a predetermined waiting period has elapsed without detecting the user's operation. The data is output to the first authentication unit 53.

  The first authentication unit 53 outputs a first imaging instruction to the imaging control unit 51 in response to an operation start signal input from the operation detection unit 55 and acquires a still image output from the imaging control unit 51. . The first authentication unit 53 authenticates the user based on the still image. Authentication is performed by extracting the face portion of the subject included in the still image and comparing it with user authentication information registered in advance. A well-known technique can be used as the technique of face authentication processing for authenticating a user based on an image obtained by imaging a subject. For example, the authentication information may be an image itself obtained by capturing an image of the user's face, or feature data extracted from the face image. When the authentication information is an image, the first authentication unit 53 determines whether the two images are similar by comparing the face portion extracted from the still image with the image of the authentication information. . In the case where the authentication information is feature data obtained by quantifying the facial feature portion, the feature portion is extracted from the face portion extracted from the still image, and the value obtained by quantifying the feature portion and the feature predetermined by the authentication information It is determined whether or not they are similar by comparing the data. The feature data is, for example, a value obtained by quantifying the shape of the facial part, the relative positional relationship between the plurality of parts, the colors of the plurality of parts, and the like. The characteristic portion may be an iris.

  The first authentication unit 53 outputs the authentication result to the operation detection unit 55. If the face part cannot be extracted from the still image acquired from the imaging control unit 51, the first authentication unit 53 outputs an authentication instruction to the second authentication unit 69 without authenticating the user. When the face portion cannot be extracted from the still image acquired from the imaging control unit 51, it is a case where the face portion necessary for authentication is not included. Including the case where part of the part is included but not the whole part. When the authentication information is feature data with the feature portion as an iris, the first authentication unit 53 outputs an authentication instruction to the second authentication unit 69 when the still image does not include an iris.

  Distance detection unit 65 detects a distance to a user existing around MFP 100. The distance detection unit 65 controls the human sensor 117 to detect whether there is a user around the MFP 65 at predetermined time intervals. When detecting the user around the MFP 65, the distance detection unit 65 causes the human sensor 117 to measure the distance to the user at a predetermined time interval until no user is detected thereafter, and the measured distance Is output to the timing determination unit 57 and the prediction unit 67. The distance detection unit 65 outputs nothing to the timing determination unit 57 when the human sensor 177 does not detect a user around the MFP 65.

  The timing determination unit 57 determines the timing at which the imaging control unit 51 images the subject in response to the distance input from the distance detection unit 65, and issues a second imaging instruction to the imaging control unit 51 at the determined timing. Output. The timing determination unit 57 outputs a second imaging instruction to the imaging control unit 51 every time the subject moves a predetermined distance. For this reason, when the distance input from the distance detection unit 65 becomes equal to any of a plurality of distances that the distance from the MFP 100 increases by a predetermined distance, the timing determination unit 57 issues a second imaging instruction to the imaging control unit 61. Output. The timing determining unit 57 calculates the moving speed of the subject based on two or more distances input from the distance detecting unit 65, and the timing at which the subject moves a predetermined distance based on the calculated moving speed. May be determined. As a result, a plurality of still images are stored in the storage control unit 59, and the distance between the subject and the MFP 100 at the time when each of the plurality of still images is captured is a still image stored before the still image. This distance is shorter than the distance between the subject and the MFP 100 at a predetermined time by a predetermined distance, and longer than the distance between the subject and the MFP 100 at the time when a still image stored after the still image is captured.

  Accordingly, the interval at which the timing determination unit 57 outputs the second imaging instruction becomes longer when the moving speed is slow than when the moving speed is fast. However, the timing determination unit 57 does not determine the timing when the moving speed of the subject is negative, in other words, when the moving speed is in a direction away from the MFP 100.

  Each time a still image is stored in the HDD 115 in each of a plurality of still images stored in the HDD 115 by the storage control unit 59, the target specifying unit 61 extracts a face part to be authenticated from the still image. For example, feature amounts such as the shape and color of each part of the face such as the face, eyes, nose, mouth, and eyebrows are determined in advance, and the face portion in the still image is extracted with reference to the feature amounts. The target specifying unit 61 specifies the same subject among a plurality of still images based on the position, shape, and the like in the still images, and detects a change in the area of the face portion. The target specifying unit 61 does not specify a face part of a subject whose face area is not large as an authentication target. This is because if the area of the face portion does not increase with the passage of time, it is highly likely that the subject has not moved toward MFP 100, and the possibility of a user operating MFP 100 is low.

  When the still image includes one face part, the target specifying part 61 specifies the face part as an authentication target. When the still image includes face portions of a plurality of subjects, the target specifying unit 61 specifies the face portion having the largest area as an authentication target. This is because the user of the face part with the largest area is more likely to be a short distance to the MFP 100 than the user of the face part with the largest area and is more likely to be a user who is trying to operate the MFP 100.

  The target specifying unit 61 specifies an authentication target for each still image, and sets a set of image identification information for identifying the still image and region identification information indicating a region in the still image to be authenticated to the image selection unit 63. Output. Here, the area identification information is stored in association with the still image stored in the HDD 115. When the authentication target is not specified, the target specifying unit 61 outputs image identification information for identifying a still image that has not specified the authentication target to the image selection unit 63.

  The target specifying unit 61 specifies an authentication target every time a still image is stored in the HDD 115 by the storage control unit 59. The target specifying unit 61 outputs a prediction instruction to the prediction unit 67 in response to a change from the state in which the authentication target is specified from the still image to the state in which the same authentication target as the authentication target cannot be specified. When the still image includes one face part, the target specifying unit 61 outputs a prediction instruction when the still image changes from a state including the face part to a state not including the face part. When the still image includes a plurality of face portions, the target specifying unit 61 specifies one authentication target from the still image.

  As described above, when the first still image includes a plurality of face parts, the target specifying unit 61 specifies one face part as an authentication target. For example, when the first still image includes a first face part and a second face part, it is assumed that the target specifying unit 61 specifies the first face part as an authentication target. In this case, the user corresponding to the first face portion may move outside the imaging range of the camera 119, and the user corresponding to the second face portion may remain within the imaging range of the camera 119. In this case, the second still image captured after the first still image includes the second face portion, but does not include the first face portion. The target specifying unit 61 specifies the second face portion as the authentication target when the first face portion specified as the authentication target in the first still image does not exist in the second still image. In addition, a prediction instruction is output to the prediction unit 67. By comparing the second face portion included in the second still image with the first face portion included in the first still image, the second still image is included in the first still image. What is necessary is just to judge that the 1st face part does not exist.

  The image selection unit 63 determines one of a plurality of still images stored in the HDD 115 by the storage control unit 59 in response to the selection instruction input from the second authentication unit 69, and determines the determined still image A set of an image and area information indicating an area to be authenticated is output to the second authentication unit 69. When a selection instruction is input from the second authentication unit 69, the still image output by the camera 119 at that time does not include the entire face portion. For this reason, the image selection unit 63 selects a still image whose authentication target is specified by the target specifying unit 61 from among a plurality of still images stored in the HDD 115. However, the image selection unit 63 does not select a still image in which the area of the face part to be authenticated does not increase. This is because when the area of the face portion does not increase, there is a high possibility that the subject does not approach the MFP 100, such as when the subject crosses the front of the MFP 100. As a result, it is possible to prevent a user who is not approaching the MFP 100 from being authenticated.

  The image selection unit 63 selects a still image having the largest area of the face portion to be authenticated from among a plurality of still images determined as selection targets. If the area is maximum, the amount of information for authentication increases and the authentication accuracy improves. The face portion in the still image is a skin-colored portion of the head above the person's neck in the still image. Therefore, it is preferable to select a still image having the maximum skin color area in the still image. This is because the amount of information used for authentication increases as the area of the skin color portion increases. Also, when the face is hidden by the hair, or when there is an obstacle between the face of the subject and the camera 119, or when the subject shoots with the camera 119 facing down, the image is taken in that state. The selected still image can be prevented from being selected.

  The second authentication unit 69 outputs a selection instruction to the image selection unit 63 in response to the authentication instruction input from the first authentication unit 53, and is specified by the region information of the still image selected by the image selection unit 63. The portion of the area to be processed is compared with the authentication information to authenticate the user, and the authentication result is output to the operation detection unit 55. In the authentication by the second authentication unit 69, the still image to be processed is not the still image output by the camera 119 after the operation detection unit 55 outputs the operation start signal, but the operation detection unit 55 outputs the operation start signal. Although it differs from the 1st authentication part 53 by the point which is the still image which the camera 119 outputs before, the comparison with authentication information is the same. When the user authentication is completed, the second authentication unit 69 deletes the still image stored in the HDD 115 by the storage control unit 59.

  When the operation detection unit 55 detects an operation input to the operation unit 163 after the authentication result indicating that the user authentication is successful is input from the first authentication unit 53 or the second authentication unit 53, When the detected operation is accepted as an operation input by an authenticated user, but when an authentication result indicating that the user authentication has failed is input, when an operation input to the operation unit 163 is detected thereafter, The detected operation is accepted as an operation input by the guest user. The operations that can be input by the guest user are limited to predetermined operations.

  The second authentication unit 69 may include a composite image generation unit 71. The portion of the region specified by the region information in the still image input from the image selection unit 63 is an image including the entire face. Therefore, a still image captured after the still image input from the image selection unit 63 and stored in the HDD 115 and including a part of the face instead of the entire face is stored in the HDD 115 by the storage control unit 59. May have been. Since such a still image is captured after the still image input from the image selection unit 63, the still image input from the image selection unit 63 is stored between the subject and the MFP 100 more than when the still image input from the image selection unit 63 is stored. The distance is likely to be short. The shorter the distance between the subject and the MFP 100, the larger the area of the face portion and the higher resolution image can be obtained. The composite image generation unit 71 selects the latest still image that is a still image stored in the HDD 115 and includes a part of the face of the subject, identifies a part of the face in the selected still image, Are extracted from the still image input from the image selection unit 63, and the combined image is generated by adjusting the size of the two portions and combining them. Although the latest still image including a part of the face of the subject is selected from the still images stored in the HDD 115 here, another still image may be selected instead of the latest one. A plurality of still images may be selected, and a plurality of portions of the face may be extracted from each of the plurality of still images and combined.

  Prediction unit 67 predicts the time from when the target is not specified by target specifying unit 61 until the subject reaches MFP 100 in response to a prediction instruction input from target specifying unit 61. The prediction unit 67 calculates the distance to the subject when the prediction instruction is input from the target specifying unit 61 and the moving speed of the subject based on the distance input from the distance detection unit 65. Prediction unit 67 then calculates the time to reach MFP 100 from the calculated distance and moving speed. The prediction unit 67 outputs the calculated time to the second authentication unit 69 as a limited period in response to a prediction instruction input from the target specifying unit 61. The time limit may be a time obtained by adding a predetermined time as a margin period to the calculated time.

  The second authentication unit 69 does not authenticate if an authentication instruction is not input from the first authentication unit 53 by the time the limit period elapses after the limit period is input from the prediction unit 67.

  FIG. 5 is a flowchart illustrating an example of the flow of the imaging control process. The imaging control process is a process executed by CPU 111 when CPU 111 provided in MFP 100 executes an authentication program stored in ROM 113, HDD 115, or CD-ROM 118A. Referring to FIG. 5, CPU 111 determines whether a user has been detected (step S01). If the infrared sensor detects a user, the process proceeds to step S02; otherwise, the process returns to step S01.

  In step S02, the camera 119 is controlled to capture an image. And the still image which the camera 119 outputs is acquired (step S03). In the next step S04, a face part is extracted from the still image, and it is determined whether or not the face part has been extracted. When the still image includes face portions of a plurality of subjects, a plurality of face portions are extracted. If the face portion can be extracted, the process proceeds to step S05. If not, the process proceeds to step S16.

  In step S05, the face part whose area does not increase is excluded from the processing targets among the one or more face parts extracted in step S04. If the area of the face portion increases from the time when the image was taken in the past, it can be determined that the subject is approaching the MFP 100. If the area of the face portion does not increase, the MFP 100 is informed when the subject crosses the MFP 100. There is a high possibility that the subject is not approaching. In step S11 or step S22 described later, before the camera 119 captures an image in step S02, a still image obtained by capturing with the camera 119 is stored in the HDD 115. For example, the face portion extracted in step S04 is set as a processing target, and among the still images stored in the HDD 115, the face portion to be compared is determined from among the still images having the latest imaging date and time, and the face to be processed By comparing the area of the part with the area of the face part to be compared, it is determined whether or not the area increases. When there are a plurality of face parts in the still image, the face part near the position in the still image of the processing target face part is determined as the face part to be compared. That's fine.

  In the next step S06, it is determined whether or not there is a face part to be processed. If there is a face portion to be processed, the process proceeds to step S07; otherwise, the process proceeds to step S16. In step S07, it is determined whether there are a plurality of face portions to be processed. If there is more than one, the process proceeds to step S08; otherwise, step S08 is skipped and the process proceeds to step S09. In step S08, the face part with the largest area is selected from the plurality of face parts, and the process proceeds to step S09.

  In step S09, when the process proceeds from step S07, one face part extracted from the still image is determined as an authentication target, and the process proceeds to step S10. When the process proceeds from step S08, the face having the largest area is obtained. The part is determined as an authentication target, and the process proceeds to step S10. In step S10, “1” is set in the detection flag, and the process proceeds to step S11. The detection flag is a flag indicating whether or not the authentication target is determined from the still image. The detection flag is set to “1” when the authentication target is determined from the still image, and “when the authentication target is not determined from the still image”. 0 "is set. In step S11, the still image acquired in step S03, the authentication target, and the imaging date and time are associated with each other and stored in HDD 115. The authentication target is position information indicating the position in the still image of the face portion determined as the authentication target.

  In step S12, the distance to the user is measured, and the process proceeds to step S13. Measure the distance to the user based on the output of the infrared sensor. In step S13, the moving speed is calculated. The moving speed is calculated from two distances measured at different times. In the next step S14, the imaging time is predicted, and the process proceeds to step S15. Using the moving speed calculated in step S13, the time for the user to travel a predetermined distance is calculated, and the time after the time calculated from the current time is predicted as the imaging time. And it will be in a standby state until the current time becomes the predicted imaging time (NO in step S15), and when the current time becomes the predicted imaging time (YES in step S15), the process returns to step S02.

  On the other hand, when the process proceeds to step S16, the face portion is not extracted from the still image (NO in step S04), or the face portion is extracted from the still image, but the area of the face portion does not increase ( NO in step S06). In step S16, it is determined whether or not the detection flag is set to “1”. If the detection flag is set to “1”, the process proceeds to step S17; otherwise, the process proceeds to step S22. In step S17, the timer T starts counting, and the process proceeds to step S18. The timer T measures the elapsed time from when the authentication target is determined from the still image to the state where the authentication target is not determined from the still image.

  In step S18, the distance to the user is measured, and the process proceeds to step S19. Measure the distance to the user based on the output of the infrared sensor. In step S19, the moving speed is calculated. The moving speed is calculated from two distances measured at different times. In the next step S20, a time limit is determined, and the process proceeds to step S21. A value obtained by dividing the distance obtained by measurement in step S19 with the movement speed calculated in step S13 is determined as the time limit.

  In the next step S21, the detection flag is set to “0”, and the process proceeds to step S22. In step S22, the still image acquired in step S03 and the shooting date and time are associated and stored in HDD 115, and the process returns to step S01.

  FIG. 6 is a flowchart illustrating an example of the flow of authentication processing. The authentication process is a process executed by CPU 111 when CPU 111 provided in MFP 100 executes an authentication program stored in ROM 113, HDD 115, or CD-ROM 118A. Further, the CPU 111 executes authentication processing in parallel with the imaging control processing shown in FIG. Referring to FIG. 6, CPU 111 determines whether an operation has been detected (step S31). The operation here is an operation on the operation panel 160 or the automatic document feeder 120 by the user. Specifically, when the touch panel 165 detects the position indicated by the user, the operation on the operation panel 160 is detected when the hard key unit 167 detects pressing of any of the plurality of hard keys. Further, when the open state detection sensor or the document detection sensor included in the automatic document feeder 120 is turned on, an operation on the automatic document feeder 120 is detected. The process waits until an operation is detected (NO in step S31). If an operation is detected, the process proceeds to step S32.

  In step S32, the camera 119 is controlled to capture an image. And the still image which the camera 119 outputs is acquired (step S33). In the next step S34, the face portion is extracted from the still image, and it is determined whether or not the entire face portion has been extracted. If the entire face portion has been extracted, the process proceeds to step S35; otherwise, the process proceeds to step S36. In step S35, the extracted face portion is authenticated by comparing with the authentication information, and the process proceeds to step S44.

  When the process proceeds to step S36, it is determined whether or not the timer T is equal to or shorter than the time limit. The timer T starts measuring time in step S17 of the imaging control process of FIG. The time limit is determined in step S20 of FIG. If timer T is equal to or shorter than the time limit, the process proceeds to step S37; otherwise, the process proceeds to step S46.

  In step S <b> 37, a still image associated with the authentication target is extracted from the still images stored in HDD 115. In the next step S38, a predetermined number of extracted still images are extracted as processing targets in order from the latest captured date and time. Then, a still image having a maximum size to be authenticated is determined from a predetermined number of still images (step S39).

  In the next step S40, it is determined whether or not a part of the face portion is included in the still image acquired in step S33. If part of the face portion is included, the process proceeds to step S41; otherwise, the process proceeds to step S43. In step S41, a part of the face part is extracted from the still image acquired in step S33, and the process proceeds to step S42. In step S42, a composite image is generated based on the part of the face part extracted in step S41 and the face part to be authenticated included in the still image determined in step S39, and the process proceeds to step S43. Proceed. A part of the face part extracted in step S41 is a first part, and a part other than the first part of the face part to be authenticated included in the still image determined in step S39 is a second part. A synthesized image is generated by synthesizing the first part and the second part by adjusting the size.

  In step S43, the face part is authenticated by comparing with the authentication information, and the process proceeds to step S44. When the process proceeds from step S40, authentication is performed by comparing the face portion to be authenticated included in the still image determined in step S39 with the authentication information. When the process proceeds from step S42, authentication is performed by comparing the composite image generated in step S42 with the authentication information.

  In step S44, the process is branched depending on the result of authentication in step S35 or step S43. If the authentication is successful, the process proceeds to step S45; otherwise, the process proceeds to step S46. In step S45, the authenticated user is set as the operating user, and the authentication process is terminated. In step S46, the guest user is set as the operating user, and the process ends.

<First Modification>
When the still image includes face portions of a plurality of subjects, the target specifying unit 61 compares the areas of the plurality of face portions between the two still images before and after the imaging time, and the face having the largest area increase rate Specify the part to be authenticated. The subject corresponding to the face part with the largest area increase rate has the highest speed of approaching the MFP 100, and is likely to be a user trying to operate the MFP 100. For example, for each of a plurality of face parts included in a still image, the corresponding face part in the still image stored before the still image is identified based on the position, shape, and color in the image, and two faces The increase rate of the area of the part is calculated.

  The target specifying unit 61 may specify the authentication target only by the area, may specify the authentication target only by the area increase rate, or specify the authentication target from the area and the increase rate. You may do it. When the authentication target is specified from the area and the increase rate, for example, the face portion having the largest area is specified as the authentication target, and the area having the largest area and one or more face portions having a smaller area is determined. If the difference is less than or equal to a predetermined value, the face portion having the largest area increase rate among the two or more face portions having an area difference equal to or less than the predetermined value is set as an authentication target.

  Since the subject having the largest increase in the area of the face portion among the plurality of subjects is specified as the authentication target, it is possible to authenticate the subject with a fast speed approaching the image processing apparatus.

<Second Modification>
The image selection unit 63 may select a still image including a face that faces most in the face portion to be authenticated from among a plurality of still images determined as selection targets. Information for authentication, for example, a feature amount can be obtained more accurately and an authentication accuracy can be further improved when an image facing the front is compared with an image not facing the front. A well-known technique can be used to detect the direction of the face from the face part image. For example, the eye, nose, and mouth parts are extracted from the face part image and the face contour is extracted. The position is determined based on the positional relationship between the eyes, nose and mouth.

<Third Modification>
The image selection unit 63 may select a still image having the maximum contrast from among a plurality of still images determined as selection targets. This is because the feature data for authentication can be obtained more accurately from an image having a high contrast.

<Fourth Modification>
The image selection unit 63 may select a still image whose position in the still image of the face part to be authenticated is closest to the center from among a plurality of still images determined as selection targets. This is because the still image has less distortion due to the lens in the central portion than the peripheral portion of the image, so that special course data for authentication can be obtained more accurately.

<Fifth Modification>
In the embodiment described above, the detection of the user and the distance to the user are measured by the infrared sensor, but a pyroelectric sensor may be used. In this case, since the distance cannot be measured, the distance detection unit 65 controls the pyroelectric sensor to detect the user in the timing determination unit 57 while the pyroelectric sensor detects the user. Outputs a presence signal indicating that The timing determination unit 57 outputs an imaging instruction to the imaging control unit 51 at predetermined time intervals while the presence signal indicating that the user is detected is input from the distance detection unit 65. Thus, camera 119 captures an image of the subject at predetermined time intervals while the user is within a predetermined distance from MFP 100. In addition, the prediction unit 67 sets a limit period in advance, and outputs a predetermined limit period to the second authentication unit 69 in response to a prediction instruction input from the target specifying unit 61. It is not limited to the pyroelectric sensor as long as the presence of a person can be detected.

<Sixth Modification>
In the present embodiment, the example in which the camera 119 outputs a still image has been described, but a moving image may be output. In this case, each of a plurality of frames included in the moving image can be analyzed in the same manner as a still image, and a person existing around MFP 100 can be detected. In addition, the camera 119 can be used instead of the pyroelectric sensor in the fifth modification, and it is determined from the change in the size of the face image in the frame whether the subject is approaching or moving away from the MFP 100. be able to.

<Seventh Modification>
In the above-described embodiment, when the operation is not detected before the prediction time predicted by the prediction unit 67, the second authentication unit 69 does not perform authentication. In the seventh modification, the second authenticating unit 69 authenticates even when an operation is not detected before the predicted time predicted by the predicting unit 67, but authentication is performed according to the elapsed time from the predicted time. Reduce the level. The authentication level indicates authentication accuracy, and the authentication accuracy indicates, for example, a comparison result of a comparison between a face portion in a still image and authentication information, and a percentage of coincidence. When the authentication level is lowered, the second authentication unit 69 performs authentication even when the authentication accuracy is low.

<Eighth Modification>
A plurality of authentication levels in which the authentication accuracy is divided at a plurality of stages are provided, and as a result of authentication by the second authentication unit 69, operations that can be input to the operation unit 163 vary depending on the authentication level to which the authentication accuracy belongs. For example, the second authentication unit 69 prepares a table in which executable processes are assigned for each of a plurality of authentication levels. For example, the table assigns more processing to an authentication level with higher authentication accuracy. As a result of authentication, the second authentication unit 69 outputs the authentication accuracy to the operation detection unit 55, and the operation detection unit 55 accepts an operation for executing a process assigned to the authentication level to which the authentication accuracy belongs by the operation unit 163. Set as possible.

<Ninth Modification>
In the above-described embodiment, the still image that is captured and output by the camera 119 is stored. However, the face portion of the subject specified as the authentication target by the target specifying unit 61 is cut out from the still image and cut out. You may make it memorize | store the image of the part of the face. As a result, the amount of image data to be stored can be reduced.

  As described above, MFP 100 according to the present embodiment functions as an image processing apparatus, stores a still image obtained by imaging with camera 119 before an operation for accepting an operation by a user, When a still image obtained later by the camera 119 includes a face portion, the user is authenticated based on the still image, and the still image obtained by the camera 119 after the operation captures the face portion. If not, the user is authenticated based on a still image obtained by imaging with the camera 119 before operation. Therefore, it is possible to authenticate a user who is not picked up at a position where the operation panel 160 of the MFP 100 is operated, but is picked up at a position away from the MFP 100, for example, a position away from the MFP 100 by a first distance.

  In addition, since MFP 100 selects and authenticates an image having the largest area of the face portion of the subject to be authenticated from among a plurality of still images stored in HDD 115, authentication accuracy can be made as high as possible. .

  In addition, since the MFP 199 selects and authenticates a still image in which the face of the subject to be authenticated faces the front from the plurality of still images stored in the HDD 115, the accuracy of authentication can be made as high as possible.

  In addition, since the MFP 199 selects and authenticates a still image having the maximum contrast among a plurality of still images stored in the HDD 115, the accuracy of authentication can be made as high as possible.

  Further, since the MFP 199 selects and authenticates a still image having the maximum skin color area of the subject to be authenticated from among a plurality of still images stored in the HDD 115, the authentication accuracy can be made as high as possible. it can.

  In addition, since the MFP 199 selects a still image in which the subject to be authenticated is arranged at the center from among a plurality of still images stored in the HDD 115, the MFP 199 can perform authentication using an image with less distortion. Can be as high as possible.

  In addition, when there is no predetermined change in the area of the face portion of the subject to be authenticated among the plurality of still images stored in the HDD 115, the MFP 199 does not select the plurality of images and thus is not approaching the MFP 100. You can prevent users from authenticating.

  Further, when the still image includes a plurality of subjects, MFP 100 identifies one of the plurality of subjects as an authentication target, so that authentication is facilitated. In particular, since MFP 100 identifies the subject having the largest face area as the authentication target, it is possible to authenticate the subject closest to MFP 100.

  In addition, since MFP 100 stores still images obtained by imaging with camera 119 at regular intervals, it stores still images obtained by imaging a user included in the imaging range of camera 119 before operation. Can do.

  Further, since MFP 100 stores a still image obtained by imaging with camera 119 every time the user moves a predetermined distance, MFP 100 can obtain the image included in the imaging range of camera 119 before operation. Still images can be stored.

  In addition, the MFP 100 converts a part of the subject included in the first image among the plurality of still images captured by the camera 119 before imaging to the second image captured after the first image. Since the authentication is performed based on the composite image supplemented with the included subject portion, the accuracy of the authentication can be further increased.

  In addition, the MFP 100 predicts the arrival time from when the camera 119 finally outputs a still image including the subject to be authenticated to when the user can operate the operation panel 160 before the operation. Since the user is not authenticated after the arrival time, it is possible to prevent another user from being authenticated by mistake.

  In the above-described embodiment, the MFP 100 has been described as an example of an image processing apparatus. However, an authentication method that causes the MFP 100 to execute the imaging control process illustrated in FIG. 5 and the authentication process illustrated in FIG. It goes without saying that the invention can be understood as an authentication program that causes the CPU 111 that controls the MFP 100 to execute the method.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  100 MFP, 51 imaging control unit, 53 first authentication unit, 55 operation detection unit, 57 timing determination unit, 59 storage control unit, 61 target identification unit, 63 image selection unit, 65 distance detection unit, 67 prediction unit, 69 first 2 authentication unit, 71 composite image generation unit, 110 main circuit, 111 CPU, 112 communication I / F unit, 113 ROM, 114 RAM, 115 HDD, 116 facsimile unit, 117 human sensor, 118 external storage device, 118A CD- ROM, 119 camera, 120 automatic document feeder, 130 document reading unit, 140 image forming unit, 150 paper feeding unit, 160 operation panel, 161 display unit, 163 operation unit, 165 touch panel, 167 hard key unit.

Claims (17)

Imaging means for outputting an image obtained by imaging;
Storage control means for storing an image output by the imaging means;
Operation detecting means for detecting an operation input by the user;
A first authentication unit that authenticates a user based on an image that the imaging unit captures and outputs after an operation in which the operation detection unit detects an operation;
Second authentication means for authenticating a user based on an image stored before the operation when the subject to be authenticated is not included in the image captured and output by the imaging means after the operation And an image processing apparatus.   The image processing apparatus according to claim 1, further comprising image selection means for selecting one image from the plurality of stored images.   The image processing apparatus according to claim 2, wherein the image selection unit selects an image having the largest area of the face portion of the subject to be authenticated.   The image processing apparatus according to claim 2, wherein the image selection unit selects an image in which a face of a subject to be authenticated faces front.   The image processing apparatus according to claim 2, wherein the image selection unit selects an image having a maximum contrast.   The image processing apparatus according to claim 2, wherein the image selection unit selects an image having an area of a skin color portion of a subject to be authenticated.   The image processing apparatus according to claim 2, wherein the image selection unit selects an image in which a subject to be authenticated is arranged at the center.   The image selection means compares the plurality of images output by the imaging means at different times, and if there is no predetermined change in the area of the face portion of the subject to be authenticated, the image selection means The image processing apparatus according to claim 2, which is not selected.   The image processing according to claim 1, further comprising: a target specifying unit that specifies one of the plurality of subjects as an authentication target when the stored image includes a plurality of subjects. apparatus.   The image processing apparatus according to claim 9, wherein the target specifying unit specifies a subject having a maximum face area as an authentication target.   The target specifying unit compares a plurality of images output by the imaging unit at different times, calculates an increase rate of the area of the face portion of each of the plurality of subjects, and a subject having the maximum calculated increase rate The image processing apparatus according to claim 9, wherein the authentication target is specified.   The image processing apparatus according to claim 1, wherein the imaging unit captures images at regular intervals. It further comprises distance detection means for detecting the distance to the subject,
The image processing apparatus according to claim 1, wherein the imaging unit captures an image every time the subject moves a predetermined distance. When the second image captured after the first image selected by the image selection means includes at least a part of the subject to be authenticated, the subject to be authenticated included in the first image A composite image generating means for generating a composite image in which a part is supplemented with a portion of the subject to be authenticated included in the second image;
The image processing apparatus according to claim 1, wherein the second authentication unit authenticates a user based on the composite image. The second authentication unit includes a prediction unit that predicts an arrival time from when the imaging unit finally outputs an image including a subject to be authenticated before the operation until the subject is operable. Including
The image processing apparatus according to claim 1, wherein the user is not authenticated after the predicted arrival time. An authentication method executed by an image processing apparatus including an imaging unit that outputs an image obtained by imaging,
A storage control step for storing an image output by the imaging means;
An operation detection step for detecting an operation input by the user;
A first authentication step of authenticating a user operating the operation accepting unit based on an image captured and output by the imaging unit after the operation at which the operation is detected;
A second authentication step of authenticating the user based on the image stored before the operation when the subject to be authenticated is not included in the image captured and output by the imaging means after the operation; And an authentication method. An authentication program executed by a computer that controls an image processing apparatus including an imaging unit that outputs an image obtained by imaging,
A storage control step for storing an image output by the imaging means;
An operation detection step for detecting an operation input by the user;
A first authentication step of authenticating a user operating the operation accepting unit based on an image captured and output by the imaging unit after the operation at which the operation is detected;
A second authentication step of authenticating the user based on the image stored before the operation when the subject to be authenticated is not included in the image captured and output by the imaging means after the operation; And an authentication program for causing the computer to execute.

Images