JP5783441B2 - Input device and input method - Google Patents

Description

  The present invention relates to an input device and an input method.

  2. Description of the Related Art In recent years, a technique has been proposed that allows a user to easily change an operation screen in a mobile terminal such as a mobile phone.

  For example, in Patent Document 1, in a portable terminal including a display unit and a camera on the display unit side, the operation screen is moved (scrolled) according to the direction of the user's face in the captured image captured by the camera, or the captured image is displayed. A technique for enlarging or reducing the operation screen according to the size of the user's face is described.

  However, in the technique described in Patent Document 1, in order to change the operation screen as described above, an operation that the user shakes the face to the left or right, moves the face closer to or away from the operation screen is necessary. It is. There is a concern that such an operation is considered to be an unnatural operation in a scene where human eyes are concerned, and there is a problem that the technique described in Patent Document 1 is not practical.

  Further, in Patent Documents 2 to 7, in a portable terminal having means such as an acceleration sensor capable of detecting a tilt or a position change of the main body, an operation screen is scrolled or operated in accordance with an operation of tilting the main body. A technique for enlarging or reducing the screen is described.

  However, in the techniques described in Patent Documents 2 to 7, for example, in order to confirm the selection of the item on which the cursor is positioned on the current operation screen, an unnatural operation such as shaking the main body in that state is necessary. There was a problem of being.

JP 2009-031368 A Japanese Patent No. 3234633 Japanese Patent No. 3582433 Japanese Patent No. 4077469 Japanese Patent Application Laid-Open No. 07-064754 JP 07-134641 A JP 07-271505 A

  As described above, the techniques described in Patent Documents 1 to 7 have a problem that an unnatural operation is required although the user can change the operation screen.

  Accordingly, an object of the present invention is to provide an input device and an input method that can solve the above-described problems.

The input device of the present invention is
A display for displaying the operation screen;
A photographing unit for photographing an image of the object facing the display unit;
A face image extraction unit for extracting a face image from a photographed image photographed by the photographing unit;
A face part detection unit for detecting a change in a predetermined part of the face included in the face image;
A screen display changing unit that changes an operation screen of the display unit when a change in the predetermined part is detected.

The input method of the present invention includes:
A screen display step for displaying an operation screen on the display;
A photographing step of photographing an image of the object facing the display unit;
A face image extraction step of extracting a face image from the photographed photographed image;
A face part detection step for detecting a change in a predetermined part of the face included in the face image;
And a screen changing step for changing the operation screen of the display unit when a change in the predetermined part is detected.

  According to the present invention, the user can easily change the operation screen without performing an unnatural operation such as shaking the face to the left or right.

It is a block diagram which shows the structure of the input device of the 1st Embodiment of this invention. It is a block diagram which shows the structure of the mobile telephone of the 2nd-4th embodiment of this invention. It is a block diagram which shows the structure of the control part of the mobile telephone of the 2nd Embodiment of this invention. It is a flowchart explaining operation | movement of the mobile telephone of the 2nd Embodiment of this invention. It is a flowchart explaining the operation | movement of the mobile telephone of the 2nd Embodiment of this invention in detail. It is a figure which shows the state which is displaying the icon which shows that the face image was extracted in the mobile telephone of the 2nd Embodiment of this invention. FIG. 7 is a diagram showing a state in which the icon shown in FIG. 6 is turned off in the mobile phone according to the second embodiment of the present invention. It is a block diagram which shows the structure of the control part of the mobile telephone of the 3rd Embodiment of this invention. It is a figure explaining operation | movement of the mobile telephone of the 3rd Embodiment of this invention. It is a block diagram which shows the structure of the control part of the mobile telephone of the 4th Embodiment of this invention.

EMBODIMENT OF THE INVENTION Below, the form for implementing this invention is demonstrated with reference to drawings.
(1) First Embodiment FIG. 1 is a block diagram showing a configuration of an input device according to a first embodiment of the present invention.

  As shown in FIG. 1, the input device of the present embodiment includes a display unit 11, a photographing unit 12, a face image extraction unit 13, a face part detection unit 14, and a screen display change unit 15.

  The display unit 11 displays an operation screen.

  Here, the operation screen refers to the entire screen displayed when the display unit 11 is in the ON state.

  The imaging unit 12 is a camera or the like that is provided on the display unit 11 side and that captures an image of a subject that faces the display unit 11.

  The face image extraction unit 13 extracts a face image from the photographed image photographed by the photographing unit 12.

  The face part detection unit 14 detects a change in a predetermined part of the face included in the face image extracted by the face image extraction unit 13.

  Here, the predetermined part is, for example, one or both of the eyes and the mouth, and the change of the predetermined part is, for example, closing the eyes, opening the mouth, and the like. Further, the opening and closing of the eyes and mouth is detected based on the size (area) of the portion appearing black at a position recognized as corresponding to the eyes and mouth, for example.

  The screen display change unit 15 changes the operation screen of the display unit 11 when the face part detection unit 14 detects a change in a predetermined part.

  Here, changing the operation screen means, for example, confirming the selection of the item where the cursor is located on the current operation screen, switching to the operation screen corresponding to the item, or displaying the current operation screen in an enlarged manner. And so on.

  As described above, according to the present embodiment, when a change in a predetermined part included in the face image extracted from the photographed image is detected, the operation screen of the display unit 11 is changed.

Therefore, the user can easily change the operation screen by performing an operation such as closing his eyes or opening his mouth without performing an unnatural operation such as shaking his / her face to the left or right. .
(2) Second Embodiment This embodiment is an example in which the input device of the present invention is applied to a mobile phone and the configuration and operation of the first embodiment are more concrete.

  FIG. 2 is a block diagram showing a configuration of a mobile phone as an input device according to the second embodiment of the present invention.

  As shown in FIG. 2, the cellular phone according to the present embodiment includes an antenna 201, a wireless transmission / reception unit 202, a control unit 203, an operation memory 204, a holding memory 205, an operation unit 206, and a display unit 207. And a camera 208 and an acceleration sensor 209.

  The antenna 201 transmits and receives phase-modulated radio waves.

  The wireless transmission / reception unit 202 demodulates the radio wave received by the antenna 201 into a digital signal and performs phase modulation on the digital signal output from the control unit 203.

  The operation unit 206 is used for inputting a telephone number when a user makes a call, inputting text when registering an address book or creating an e-mail, and inputting setting of various functions. It may be composed of

  The display unit 207 corresponds to the display unit 11 in FIG. 1 and displays an operation screen.

  Here, the operation screen refers to the entire screen displayed on the display unit 207 in the ON state. Specific examples of the operation screen include a screen for displaying the other party's telephone number when making and receiving calls, a screen for displaying various input contents input to the operation unit 206, a screen for displaying the contents of e-mail, and calling various functions There are menu screens.

  The camera 208 corresponds to the photographing unit 12 in FIG. 1, is provided on the display unit 207 side, and photographs an image of a subject facing the display unit 207.

  The acceleration sensor 209 is, for example, a three-axis acceleration sensor that measures acceleration in the X-axis direction, the Y-axis direction, and the Z-axis direction.

  The operation memory 204 is used to store programs executed by the control unit 203 and fixed data, and is also used as a work area when the control unit 203 performs operations.

  The holding memory 205 registers e-mails transmitted / received by the mobile phone, an address book, image data of a photographed image taken by the camera 208, and the like.

  Further, the holding memory 205 stores a predetermined part in the face, details of a predetermined change for the predetermined part, and details of a predetermined process performed according to the contents of the change of the predetermined part. Such data is registered in advance.

  Note that a part of the operation memory 204 and the holding memory 205 is generally realized by dividing an area in the physically same device.

  The control unit 203 realizes various functions by controlling each component in the mobile phone, and includes a CPU (Central Processing Unit) and the like.

  For example, the control unit 203 detects a call number of its own mobile phone from the digital signal demodulated by the wireless transmission / reception unit 202 and notifies the incoming call, or processes the voice signal or the digital signal to perform a voice call. Functions, mail transmission / reception functions, server connection functions, etc. are realized.

  In addition, the control unit 203 controls the camera 208, sets brightness, color tone, angle of view size, and the like, specifies shooting timing, and acquires image data of a captured image captured by the camera 208. In addition, the control unit 203 performs predetermined size conversion and format conversion on the image data acquired from the camera 208 and displays the image data on the display unit 207, and also performs predetermined compression according to a user operation performed on the operation unit 206. The data is processed and stored in the holding memory 205. With these operations, the control unit 203 realizes a camera photographing function.

  In addition, the control unit 203 can acquire a measurement result obtained by measuring acceleration with the acceleration sensor 209 and detect a tilt or a physical movement (movement) of the mobile phone with respect to the gravity direction. Further, the control unit 203 controls the measurement timing of the acceleration sensor 209. However, the acceleration sensor 209 can measure the acceleration independently at a certain interval, and can notify the control unit 203 when the acceleration change exceeds a certain threshold.

  FIG. 3 is a block diagram illustrating a configuration of the control unit 203 of the mobile phone according to the present embodiment. FIG. 3 shows only the characteristic components of the present invention extracted, and other components that perform the above-described operations and the like are omitted.

  As shown in FIG. 3, the control unit 203 includes a face image extraction unit 210, a face part detection unit 211, and a screen display change unit 212 as characteristic components of the present invention.

  The face image extraction unit 210 corresponds to the face image extraction unit 13 in FIG. 1, and extracts a face image from a captured image captured by the camera 208.

  The face part detection unit 211 corresponds to the face part detection unit 14 of FIG. 1, and detects a change in a predetermined part of the face included in the face image extracted by the face image extraction unit 210.

  Here, the predetermined part is, for example, either or both of the eyes and the mouth, and the change of the predetermined part is, for example, closing the right eye or opening the mouth. Further, the opening and closing of the eyes and mouth is detected based on the size (area) of the portion appearing black at the position recognized as corresponding to the eyes and mouth in the face image, for example.

  The screen display change unit 212 corresponds to the screen display change unit 15 in FIG. 1 and changes the operation screen of the display unit 207 when the face part detection unit 211 detects a change in a predetermined part.

  Here, changing the operation screen means, for example, confirming the selection of the item on which the cursor is placed on the current operation screen, switching to the operation screen corresponding to the item, or enlarging the current operation screen. To do.

  Hereinafter, the operation of the mobile phone according to the present embodiment will be described with reference to FIG.

  FIG. 4 is a flowchart for explaining the operation of the mobile phone according to the present embodiment.

  As shown in FIG. 4, first, the control unit 203 activates the camera 208 in a state where the display unit 207 is ON and an operation screen is displayed (step ST11).

  After activation of the camera 208, the face image extraction unit 210 of the control unit 203 extracts a face image from the captured image captured by the camera 208 (step ST12).

  In step ST12, when a face image is extracted (YES in step ST12), the face part detection unit 211 of the control unit 203 obtains a predetermined part from the holding memory 205 and the content of the change of the predetermined part. Reading and detecting whether or not the change occurs in the predetermined part (step ST13).

  When a change is detected in the predetermined part in step ST13 (YES in step ST13), the screen display changing unit 212 of the control unit 203 reads the content of the process corresponding to the content of the change in the predetermined part from the holding memory 205. The operation screen of the display unit 207 is changed by reading and executing the process (step ST14).

  While the display unit 207 is ON, the processes from step ST12 to step ST14 are repeated.

  Thereafter, when the display unit 207 is turned off (YES in step ST15), the control unit 203 ends the camera 208 (step ST16).

  Note that turning off the display unit 207 in step ST15 includes both the case where the power supply of the mobile phone itself is turned off and the case where the display unit 207 is turned off while the power supply of the mobile phone itself is on. . The case where the display unit 207 is turned off while the power supply of the mobile phone is turned on is, for example, that the user performs an operation for shifting to the power saving mode on the operation unit 206 to turn off the display unit 207. Or a case where the user closes the main body of the cellular phone and turns off the display unit 207 when the cellular phone is an open / close type device having the display unit 207 inside. However, in step ST15, when the power supply of the mobile phone itself is turned off and the display unit 207 is turned off, the camera 208 is also terminated at that time. Therefore, in step ST16, the process for terminating the camera 208 is not necessary.

  Here, a specific example of data registered in the holding memory 205 will be described.

  For example, the holding memory 205 registers data indicating that the predetermined part is an eye and a mouth.

  In the holding memory 205, when the right eye is closed for a certain period of time, the selection of the item on which the cursor is positioned on the current operation screen is confirmed, and the operation screen corresponding to the item is switched. Data indicating that the process is performed, and data indicating that the process when the mouth is opened for a certain period of time or more are processes for enlarging and displaying the current operation screen are registered.

  Furthermore, in a situation where the holding memory 205 is displaying content spanning multiple pages such as an electronic book or a newspaper, the processing when the right eye is closed for a certain period of time (for example, a second) is performed as follows. Data indicating that the process is switching to the page, data indicating that the process when the left eye is closed for a certain period of time (for example, a second) is switching to the previous page, or any eye is constant Data may be registered indicating that the processing in the case of closing for a time (for example, b seconds, b> a) or more is processing for repeating the page turning operation toward the closed eyes. Thus, in this embodiment, it is possible to define a process according to a change in a predetermined part for each operation screen.

  Next, the operation of the mobile phone according to the present embodiment will be described more specifically with reference to FIGS. 5, 6, and 7.

  FIG. 5 is a flowchart for explaining the operation of the mobile phone of this embodiment in more detail.

  6 and 7 are external views of the mobile phone according to the present embodiment. FIG. 6 shows a state in which an icon indicating that a face image has been extracted is displayed on the operation screen, and FIG. The state where the display is turned off is shown respectively. 6 and 7, reference numeral 301 denotes a mobile phone body, 207 denotes a display unit, and 208 denotes a camera (camera lens).

  As shown in FIG. 5, first, similarly to FIG. 4, the processes of steps ST11 and ST12 are performed.

  When a face image is extracted in step ST12 (YES in step ST12), the screen display changing unit 211 of the control unit 203, as shown in FIG. 6, shows an icon 302 indicating that the face image has been extracted. Is displayed on a part of the display unit 207 (step ST21).

  On the other hand, when a face image is not extracted in step ST12 (NO in step ST12), the screen display changing unit 211 of the control unit 203 turns off the display of the icon 302 as shown in FIG. 7 (step ST22). .

  At this time, as shown in FIGS. 6 and 7, the captured image 303 captured by the camera 208 may be reduced and displayed on a part of the display unit 207. Thereby, the user using the mobile phone can move the position of his / her face to a position where the face image can be extracted while viewing the captured image 303.

  Thereafter, similarly to FIG. 4, the processes of steps ST13 to ST16 are performed.

  As described above, according to the present embodiment, when a change in a predetermined part included in the face image extracted from the photographed image is detected, the user shakes the face left and right to change the operation screen of the display unit 207. Even if an unnatural operation such as the above is not performed, the operation screen can be easily changed only by performing an operation such as closing an eye or opening a mouth.

  Therefore, even if the user holds his mobile phone with his free hand, such as holding a baggage or holding it in a leather strap in a train or a bus, The mobile phone can be operated without such a movement.

  In addition, when the operation unit 206 is configured with a touch panel, the mobile phone can be operated without touching the screen of the display unit 207, thereby reducing the chance of smearing the surface of the display unit 207 integrated with the touch panel. be able to.

  Further, according to the present embodiment, since the icon 302 indicating that the face image has been extracted is displayed on the display unit 207, the user can understand that the face operation is possible based on the display of the icon 302. . The information indicating that the face image has been extracted is not limited to the icon 302, and may be displayed as a character, a figure, or the like.

In addition, according to the present embodiment, since the captured image 303 captured by the camera 208 is displayed on the display unit 207, the user moves his / her face to a position where a facial image can be extracted while viewing the captured image 303. Can do.
(3) Third Embodiment FIG. 8 is a block diagram illustrating a configuration of a control unit 203 of a mobile phone as an input device according to a third embodiment of the present invention.

  Referring to FIG. 8, the mobile phone of this embodiment is different from the second embodiment shown in FIG. 3 in that a target image specifying unit 213 is provided in the control unit 203.

  When there are a plurality of human face images in the captured image captured by the camera 208, the target image specifying unit 213 detects a change in a predetermined part from the plurality of human face images. A face image to be detected is specified.

  FIG. 9 is a diagram for explaining the operation of the mobile phone according to the present embodiment.

  In FIG. 9, there are three face images of subjects A, B, and C in the photographed image.

  Of these, the subject C cannot be extracted as a face image by the face image extraction unit 210 because one eye is out of the captured image.

  The subject B can be extracted as a face image by the face image extraction unit 210, but the face of the subject B is smaller than the face of the subject A and is located toward the end of the captured image.

  Here, there is a high possibility that the user who actually uses the mobile phone is reflected most in the captured image or in the vicinity of the center of the captured image.

  For example, in FIG. 9, the subject A is the largest in the photographed image, and the subject A is the most centered portion of the photographed image, so the user who actually uses the mobile phone is the subject A. Very likely.

  Therefore, when the face image extraction unit 210 extracts the face images of a plurality of persons, the target image specifying unit 213 displays the face image of the person most reflected in the captured image or the most centered portion of the captured image. The face part detection unit 211 detects a change in a predetermined part included in the face image specified as the detection target by the target image specification unit 213.

Therefore, in the present embodiment, it is possible to prevent an erroneous operation caused by the movement of a person reflected in the background of the user who actually uses the mobile phone.
(4) Fourth Embodiment FIG. 10 is a block diagram showing a configuration of a control unit 203 of a mobile phone as an input device according to a fourth embodiment of the present invention.

  Referring to FIG. 10, the mobile phone according to the present embodiment includes a face authentication unit 214, a part position determination unit 215, and an operation stop in the control unit 203 as compared with the second embodiment illustrated in FIG. 3. The difference is that the portion 216 is provided.

  Note that feature point data of a face image of a predetermined person (such as an owner of a mobile phone) is registered in the holding memory 205 in advance.

  When the face image extraction unit 210 extracts the face image, the face authentication unit 214 compares the feature points of the extracted face image with the feature points of the face image of a predetermined person registered in the holding memory 205 in advance. Authenticate the extracted face image.

  When the facial image extracted by the facial image extraction unit 210 is authenticated, the facial part detection unit 211 detects a change in a predetermined part included in the facial image.

  Therefore, in this embodiment, it is possible to prevent other persons from operating other than the predetermined person registered in advance in the mobile phone, and to ensure security.

  Further, when the authentication of the face image extracted by the face image extraction unit 210 is established, the face part detection unit 211 notifies the part position determination unit 215 to that effect, and the part position determination unit 215 is notified by the face image extraction unit 210. After authentication of the extracted face image is established, the movement of a predetermined part in the photographed image is followed and monitored to determine whether or not the predetermined part has deviated from the photographed image. At this time, it may be determined whether or not all of the predetermined part is out of the captured image, or it may be determined whether or not a part of the predetermined part is out of the captured image.

  The operation stopping unit 216 stops the operations of the face part detecting unit 211 and the screen display changing unit 212 until the face image extracted by the face image extracting unit 210 is re-authenticated when the predetermined part is removed from the captured image.

  Therefore, in the present embodiment, it is possible to ensure security while suppressing the detection process of the face part detection unit 211 and the operation screen change process of the screen display change unit 212.

  In the present embodiment, information indicating whether or not authentication of the face image extracted by the face image extraction unit 210 has been established may be displayed as an icon as shown in FIGS. 6 and 7. , Characters, figures, etc. may be displayed.

  Although the present invention has been described with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

  For example, in the above embodiment, a mobile phone has been described as an example of the input device of the present invention. It is possible.

  In the above embodiment, the third and fourth embodiments have been described as separate embodiments. However, the third and fourth embodiments can be combined.

Further, a part or all of the above embodiment can be described as in the following supplementary notes, but is not limited thereto.
(Appendix 1)
A display for displaying the operation screen;
A photographing unit for photographing an image of the object facing the display unit;
A face image extraction unit for extracting a face image from a photographed image photographed by the photographing unit;
A face part detection unit for detecting a change in a predetermined part of the face included in the face image;
An input device comprising: a screen display changing unit that changes an operation screen of the display unit when a change in the predetermined part is detected.
(Appendix 2)
The screen display change unit
The input device according to appendix 1, wherein information indicating that a face image has been extracted is displayed on the display unit.
(Appendix 3)
A face authentication unit that authenticates the face image extracted by the face image extraction unit based on a face image extracted by the face image extraction unit and a face image of a predetermined person registered in advance;
The face part detection unit is
When the face image extracted by the face image extraction unit is authenticated, a change in the predetermined part included in the face image is detected,
The input device further includes:
A part position determination unit that determines whether or not the predetermined part is off the captured image;
An operation stop unit that stops the operation of the face part detection unit and the screen display change unit until the face image extracted by the face image extraction unit is re-authenticated when the predetermined part is out of the captured image; The input device according to Supplementary Note 1 or 2, further comprising:
(Appendix 4)
The screen display change unit
The input device according to appendix 3, wherein information indicating whether authentication of the face image extracted by the face image extraction unit has been established is displayed on the display unit.
(Appendix 5)
The predetermined site is either the eyes or the mouth, or both,
The input device according to any one of appendices 1 to 4, wherein the change of the predetermined part is that the predetermined part is closed or open.
(Appendix 6)
The screen display change unit
The input device according to appendix 5, wherein the operation screen of the display unit is changed by performing processing according to a time when the predetermined part is closed or open.
(Appendix 7)
A screen display step for displaying an operation screen on the display;
A photographing step of photographing an image of the object facing the display unit;
A face image extraction step of extracting a face image from the photographed photographed image;
A face part detection step for detecting a change in a predetermined part of the face included in the face image;
A screen changing step of changing an operation screen of the display unit when a change in the predetermined part is detected.
(Appendix 8)
The input method according to appendix 7, further comprising an extraction display step of displaying information indicating that a face image has been extracted on the display unit.
(Appendix 9)
A face authentication step for authenticating the extracted face image based on the extracted face image and a face image of a predetermined person registered in advance;
In the face part detection step,
When the extracted face image is authenticated, the change of the predetermined part included in the face image is detected,
The input method further includes:
A part position determining step for determining whether or not the predetermined part is deviated from the captured image;
Or a processing stop step of stopping the processing of the face part detecting step and the screen changing step until the extracted face image is re-authenticated when the predetermined part deviates from the captured image. 9. The input method according to 8.
(Appendix 10)
The input method according to appendix 9, further comprising an authentication display step of displaying information indicating whether or not authentication of the extracted face image has been established on the display unit.
(Appendix 11)
The predetermined site is either the eyes or the mouth, or both,
The input method according to any one of appendices 7 to 10, wherein the change of the predetermined part is that the predetermined part is closed or open.
(Appendix 12)
In the screen change step,
The input method according to appendix 11, wherein the operation screen of the display unit is changed by performing processing according to a time when the predetermined part is closed or open.

DESCRIPTION OF SYMBOLS 11 Display part 12 Image | photographing part 13 Face image extraction part 14 Face part detection part 15 Screen display change part 201 Antenna 202 Radio | wireless transmission / reception part 203 Control part 204 Operation | movement memory 205 Holding | maintenance memory 206 Operation part 207 Display part 208 Camera 209 Acceleration sensor 210 Face image extraction unit 211 Face part detection part 212 Screen display change part 213 Target image identification part 214 Face authentication part 215 Part position determination part 216 Operation stop part

Claims (8)

A display for displaying the operation screen;
A photographing unit for photographing an image of the object facing the display unit;
A face image extraction unit for extracting a face image from a photographed image photographed by the photographing unit;
A face part detection unit for detecting a change in a predetermined part of the face included in the face image;
A screen display changing unit that changes an operation screen of the display unit when a change in the predetermined part is detected;
A face authentication unit that authenticates the face image extracted by the face image extraction unit based on a face image extracted by the face image extraction unit and a face image of a predetermined person registered in advance.
The face part detection unit is
When the face image extracted by the face image extraction unit is authenticated, a change in the predetermined part included in the face image is detected,
further,
A part position determination unit that determines whether or not the predetermined part is off the captured image;
An operation stop unit that stops the operation of the face part detection unit and the screen display change unit until the face image extracted by the face image extraction unit is re-authenticated when the predetermined part is out of the captured image;
When face images of a plurality of persons are present in an image photographed by the photographing unit, a face image to be detected by which the face part detecting unit detects a change in a predetermined part from the face images of the plurality of persons. An input device having a target image specifying unit to be specified . The screen display change unit
The input device according to claim 1, wherein information indicating that a face image has been extracted is displayed on the display unit. The screen display change unit
The input device according to claim 1, wherein information indicating whether authentication of the face image extracted by the face image extraction unit is established is displayed on the display unit. The predetermined site is either the eyes or the mouth, or both,
The input device according to any one of claims 1 to 3, wherein the change of the predetermined part is that the predetermined part is closed or open. The screen display change unit
The input device according to claim 4, wherein an operation screen of the display unit is changed by performing processing according to a time when the predetermined part is closed or open. A screen display step for displaying an operation screen on the display;
A photographing step of photographing an image of the object facing the display unit;
A face image extraction step of extracting a face image from the photographed photographed image;
A face authentication step for authenticating the extracted face image based on the extracted face image and a face image of a predetermined person registered in advance;
A face part detection step of detecting a change in a predetermined part of a face included in the face image when the extracted face image is authenticated;
A screen changing step for changing an operation screen of the display unit when a change in the predetermined portion is detected;
A part position determining step for determining whether or not the predetermined part is deviated from the captured image;
A process stop step of stopping processing of the face part detection step and the screen change step until the extracted face image is re-authenticated when the predetermined part deviates from the captured image ;
Furthermore, when there are a plurality of human face images in the captured image, a detection target face image for detecting a change in a predetermined part in the face part detection step is selected from the plurality of person face images. An input method having a target image specifying step of specifying .   The input method according to claim 6, further comprising an extraction display step of displaying information indicating that a face image has been extracted on the display unit. The input method according to claim 6, further comprising an authentication display step of displaying information indicating whether or not authentication of the extracted face image is established on the display unit.