JP2008276274A - Information processor and information processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To recognize a state of being peeped without specially devising an image display device and regardless of the position of a peeper. <P>SOLUTION: An image pickup part 1 takes an image of an area in front of an image display device of an information processor. A face detection means 21 detects a face on the image taken by the image pickup part 1. A peep determining means 22 determines whether or not (n) (n ≥1) or more faces exist in the image based on the result of detection by the face detection means 21. If the peep determining means 22 determines that the image has (n) or more faces, an alarm generating means 23 generates alarm information and causes an alarm output part 3 to issue an alarm toward the user of the information processor. If the peep determining means 22 determines that the image does not have (n) or more faces, the information processor normally ends a process. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an information processing apparatus and an information processing method having a peeping prevention function, and in particular, when a user uses an image display apparatus attached to an information processing apparatus, it is detected that another person is peeping at a display screen. The present invention relates to an information processing apparatus and an information processing method having a peeping prevention function that warns a user (user) and hides information displayed on an image display device from others.

  A method of preventing others from peeping at the display screen of an image display device attached to an information processing device such as a mobile phone, a PC (Personal Computer), an ATM (Auto Teller Machine), etc. For example, there is a method of sticking a specially processed film sheet for narrowing the viewing angle of the screen to the screen.

  For example, Patent Document 1 describes an image display device that prevents information from being viewed by looking into from behind a train or the like. The image display device described in Patent Literature 1 includes an imaging unit, an image display unit, a detection unit that detects the position of the face eye, and a display position change unit.

  The image display apparatus using the conventional peeping prevention method having the above-described configuration operates as follows. The image display device detects the position of the eyes of the face by image recognition from the image captured by the imaging unit using the detection unit. The image display device uses the display position changing unit to change the display position of the image displayed by the image display unit based on the detection result of the detecting unit. Therefore, the image position displayed on the screen changes according to the relative position between the position of the user's eyes and the position of the imaging means. As a result, in an environment where the heads of the user and other people swing differently (for example, when displaying an image in a train, etc.), the display image appears to be blurred by other people, thus preventing information from being viewed. Can do.

  Further, for example, Non-Patent Document 1 describes an absence management system that prevents a person other than a registered regular user from peeping. The absence management system described in Non-Patent Document 1 includes an imaging unit, an image display unit, a unit for detecting the position of a face, and whether the captured face corresponds to a person's face registered in advance. And face collation means for discriminating whether or not.

  The away management system using the conventional peeping prevention method having the above-described configuration operates as follows. The away desk management system detects a face from the image taken by the imaging means using image recognition by the detection means. Further, the away management system extracts face data from the image based on the detected eye position, and collates with the regular use person data (user face image) registered in advance using the extracted face data. When the absence management system determines that the person is a regular user, the display screen continues to be displayed without changing the output of the image display means. Further, when the absence management system does not determine that the person is a regular user, the information displayed on the image display unit is concealed. As a result, it is possible to prevent the person who is not registered from looking into the information.

Japanese Patent Laying-Open No. 2004-317813 (paragraphs 0052-0064 and FIGS. 5-7) Masahiro Suzuki, “Seating Management System“ FaceMonitor ”Face Detection / Face Matching Engine“ NeoFace ””, Image Lab, March 2005, Japan, 2005, p54-57

  When a specially processed film sheet is pasted on the screen, the image quality is degraded by stretching the film sheet on the screen. Therefore, when using the method of sticking a specially processed film sheet on the screen, the first problem is that even a legitimate user of the information processing apparatus may have difficulty in viewing the screen.

  Also, when pasting a specially processed film sheet on the screen, when looking into the screen from the back of a regular user, etc. It will be within the viewing angle of the screen. For this reason, even a person other than the authorized user can read the information displayed on the screen. Therefore, when using the method of sticking a specially processed film sheet on the screen, the second problem is that it is not possible to prevent snooping depending on the direction in which another person looks into the screen.

  Moreover, in the image display apparatus described in Patent Document 1, it is possible to prevent a peep on the display screen by utilizing the shaking of the head. However, when viewing the image display device under a situation where the head is stationary, it is not always possible to prevent peeping by someone other than the authorized user. For example, when operating a cellular phone at a seat in a coffee shop, operating a PC at a seat in an office, or operating an ATM at a bank, the head does not move as much as in a train. Therefore, even if the image display device described in Patent Document 1 is used, the image displayed on the image display device may not move and may be easily peeped by others. Therefore, when the image display device described in Patent Document 1 is used, a third problem is that peeping cannot be prevented depending on where the image display device is used.

  In addition, when using the away management system described in Non-Patent Document 1, it is necessary to record in advance a registration data of a regular user used for collation by the face collation unit. Therefore, it is necessary to provide a storage device that stores registration data, and the cost of the storage device is increased. In particular, an information processing apparatus such as a portable terminal has a high storage capacity, which increases the cost burden. Accordingly, when the away management system described in Non-Patent Document 1 is used, as a fourth problem, there is a problem that the cost of the information processing apparatus becomes high because the face matching unit is used. Further, in the away management system described in Non-Patent Document 1, since the registered face data of the authorized user and the captured image are collated, the processing load on the information processing apparatus is large.

  Therefore, an object of the present invention is to provide an information processing apparatus and an information processing method that can prevent a peeping of the screen by another person without deteriorating the image quality of the display screen of the image display apparatus attached to the information processing apparatus. . It is another object of the present invention to provide an information processing apparatus and an information processing method that can prevent another person from looking into the screen from the same viewpoint as the user. It is another object of the present invention to provide an information processing apparatus and an information processing method that can prevent peeping on a screen without requiring movement of the head of a person who operates the information processing apparatus. Furthermore, an object of the present invention is to provide an information processing apparatus and an information processing method that can prevent a peep on the screen by a simple method without registering face data or attribute data of a regular user.

  An information processing apparatus according to the present invention is installed in an information processing apparatus so as to photograph a user while the user is looking at a screen displayed by the information processing apparatus (for example, a mobile phone or a personal computer), and includes an image including the user The information displayed by the information processing device based on the number of faces detected by the image detection unit, the face detection unit for detecting a face included in the image captured by the image pickup unit, Peeping judgment means that determines whether or not it is a peeping state that is being peeped by another person, and if the peeping judgment means determines that it is a peeping state, it is a solution to eliminate the peeping state A peeping state elimination unit (for example, realized by the warning output unit 3, the display switching unit 24, and the peeping person recording unit 25) that performs processing is provided.

  By adopting the above-described configuration, the imaging unit images the front of the image display means (for example, a liquid crystal display unit of a monitor or a mobile phone) of the information processing apparatus, and faces of persons of a predetermined number or more are included in the captured image. The object of the present invention can be achieved by determining whether or not it is detected.

  Further, in the information processing apparatus, the peeping determination unit determines whether or not the number of faces detected by the face detection unit is equal to or greater than a predetermined number (for example, n ≧ 1), and detects the face detected by the face detection unit. When it is determined that the number is a predetermined number or more, it may be determined that the peeping state is set.

  The information processing apparatus is installed in the information processing apparatus so as to photograph the user while the user is viewing a screen displayed by the information processing apparatus, and an imaging unit that captures an image including the user, Face detection means for detecting a face included in the photographed image, eye gaze estimation means for estimating the direction of the eye gaze contained in the face detected by the face detection means, and face detection based on the estimation result of the eye gaze estimation means Screen gaze determination means for determining whether or not the face person detected by the means is gazing at the screen displayed by the information processing apparatus; and peep state cancellation means for executing cancellation processing for canceling the peep state And the screen gaze determining means determines that the person is determined to be peeping when determining that the number of persons determined to be watching the screen displayed by the information processing apparatus is greater than or equal to a predetermined number. The state resolution means is a screen gaze When it is determined that the constant section is the peeping state, it may be configured to execute the cancellation processing.

  In addition, the information processing apparatus may include an image display device (for example, a liquid crystal display unit of a monitor or a mobile phone, an image output unit 5) that displays information in accordance with a request instruction from the user of the information processing device. .

  In the information processing apparatus, the peeping state elimination unit is a warning output unit (for example, realized by the warning output unit 3) that outputs a warning for notifying the user of the peeping state. May be.

  In the information processing apparatus, the warning output unit may notify the user that the user is in a peeping state by displaying an image, outputting sound, vibrating the information processing apparatus, or emitting light.

  Further, in the information processing device, the warning output means is in a state of peeping at the user by a combination of any two or more of image display, sound output, information processing device vibration, or light emission. You may notify.

  In the information processing apparatus, the warning output means notifies the user that the user is in the peeping state by using all of image display, sound output, vibration of the information processing apparatus, and light emission. May be.

  In the information processing apparatus, the peeping state elimination unit may be an image display switching unit (for example, realized by the display switching unit 24) that switches information displayed by the image display device.

  In the information processing apparatus, the image display switching unit may switch a screen displayed by the image display apparatus to an image preset by a user of the information processing apparatus.

  In the information processing apparatus, the image display switching unit may switch a screen displayed by the image display apparatus to a black image.

  In addition, the information processing apparatus includes peep image storage means (for example, realized by the peep person storage unit 71) that stores an image in which a person peeping at a screen displayed by the information processing apparatus is shown. The peeping state elimination unit may be a peeping image recording unit that stores an image taken by the imaging unit in the peeping image storage unit when it is determined that the peeping state is in the peeping state.

  In the information processing apparatus, the peep image recording unit extracts a region including the face detected by the face detection unit from the image captured by the imaging unit, and stores the image of the extracted region in the peep image storage unit. It may be allowed.

  Further, the information processing apparatus may include an image display device that displays information in accordance with a request instruction from the user of the information processing apparatus, and the imaging unit may be installed around the image display device.

  An information processing method according to the present invention is based on an imaging step of capturing an image including a user of the information processing apparatus and the surroundings of the user, a face detection step of detecting a face included in the captured image, and the number of detected faces. When the information displayed by the information processing device is in a peeping state, which is a state of being peeped by another person other than the user, And a peeping state cancellation step for executing a cancellation process for canceling the viewing state.

  By adopting the configuration as described above, the front of the image display means of the information processing apparatus is photographed by the imaging unit, and it is determined whether or not the faces of persons of a predetermined number or more are detected in the photographed image, The object of the present invention can be achieved.

  Further, the information processing method determines whether or not the number of faces detected in the face detection step is greater than or equal to a predetermined number in the peek determination step, and determines that the number of detected faces is greater than or equal to the predetermined number Alternatively, it may be determined to be in a peeping state.

  Further, the information processing method includes an imaging step for capturing an image including a user of the information processing apparatus and the surroundings of the user, a face detection step for detecting a face included in the captured image, and a line of sight of eyes included in the detected face. A gaze estimation step for estimating the direction of the image, a screen gaze determination step for determining whether or not the detected person of the face is gazing at the screen displayed by the information processing device based on the estimation result in the gaze estimation step, Including a peeping state cancellation step for performing a cancellation process for canceling the viewing state, and the number of persons determined to be gazing at the screen displayed by the information processing device in the screen gaze determination step is greater than or equal to a predetermined number If it is determined that there is a peeping state when it is determined, and if it is determined in the peeping state elimination step that it is a peeping state, a cancellation process may be executed.

  Further, the information processing method may include a step of displaying information on the image display device in accordance with a request instruction from the user of the information processing device.

  The information processing method may include a warning output step of outputting a warning for notifying the user that the user is in the peeping state in the peeping state elimination step.

  The information processing method may be a warning output step that notifies the user that the user is in a peeping state by displaying an image, outputting sound, vibrating the information processing apparatus, or emitting light.

  Further, the information processing method is a warning output step in which the user is peeping to the user by combining any two or more of image display, sound output, information processing device vibration, and light emission. You may notify.

  The information processing method is a warning output processing step that notifies the user that the user is in a peeping state by using all of image display, sound output, information processing device vibration, and light emission. There may be.

  The information processing method may include an image display switching step of switching information displayed by the image display device in the peeping state elimination step.

  The information processing method may switch the screen displayed by the image display device to an image preset by the user of the information processing device in the image display switching step.

  The information processing method may switch the screen displayed by the image display device to a black image in the image display switching step.

  Also, in the information processing method, when it is determined in the peeping state elimination step that the peeping state is determined, the person who is peeping at the screen displayed by the information processing device is shown in the image captured in the imaging step. It may include a peep image recording step of storing in a peep image storage means for storing a stored image.

  In the information processing method, in the peek image recording step, an area including the face detected in the face detection step is extracted from the image photographed in the imaging step, and the image of the extracted area is stored in the peek image storage means. It may be allowed.

  According to the present invention, a face included in a photographed image is detected, and it is determined whether or not it is a peeping state based on the number of faces in the photographed image. According to the present invention, it is not necessary to attach a special film sheet or the like to the display screen of the information processing apparatus to prevent peeping, and the viewing angle of the image display apparatus attached to the information processing apparatus does not have to be narrowed. Further, there is no need to process the image display device itself, and the user can directly view the screen. Accordingly, it is possible to prevent a person from peeping at the screen without deteriorating the image quality of the display screen of the image display device attached to the information processing device.

  In addition, according to the present invention, a face is detected from an image obtained by photographing a user and a region including the back of the user, and a determination based on the number of faces is performed to prevent a peeping action. Therefore, even when another person looks into the screen from almost the same viewpoint as the user, it is possible to prevent another person from looking into the screen.

  Further, according to the present invention, regardless of the movement of the person who operates the information processing apparatus, the face is detected from the image obtained by photographing the person and the surroundings of the person, and the peeping state is based on the number of detected faces. It is determined whether or not. For this reason, it is possible to prevent a peeping action even when the user is stationary. Therefore, it is possible to prevent peeping on the screen without requiring the movement of the head of the person who operates the information processing apparatus.

  In addition, according to the present invention, it is not necessary to perform face collation, personal attribute determination, or the like in order to determine whether or not the user is peeping, and it is not necessary to store data for specifying a regular user. Therefore, even if information related to the user is not stored in the storage unit in advance, the peeping action can be prevented. In addition, since it is not necessary to perform collation processing using face data, the processing load on the information processing apparatus can be reduced. Therefore, it is possible to prevent the screen from being looked at by a simple method without registering the face data or attribute data of the authorized user.

  Further, in the present invention, if it is configured not only to detect a face in an image but also to determine whether or not it is a peeping state by detecting a point at which the person whose face is detected gazes, It is possible to perform an accurate peep judgment. Therefore, it is possible to prevent the peeping action by others with higher accuracy.

  Further, in the present invention, if the information processing apparatus is configured to switch the information displayed when the peeping state is detected, the user of the information processing apparatus does not perform a specific operation in order to eliminate the peeping by others. However, peeping by others can be prevented.

  Further, in the present invention, when a peeping state is detected, a photographed image when it is determined as a peeping state is stored, so that a user or an administrator of the information processing apparatus performs a peeping action. Can be confirmed after the fact. Therefore, when an illegal act due to peeping occurs, it is possible to easily identify the person who has performed the illegal act. Further, it is possible to expect an effect of suppressing the peeping action by recording the peeping state image, and the peeping action can be prevented in advance.

Embodiment 1 FIG.
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of the configuration of an information processing apparatus having a peep prevention function according to the present invention. As shown in FIG. 1, the information processing device includes an imaging unit 1 that inputs an image, a data processing device 2 that operates under program control, and a warning by sound, image, vibration, light, etc. to a user (user). And a warning output unit 3 for outputting.

  In the present embodiment, the information processing apparatus is specifically realized by an information terminal such as a mobile phone, a personal computer, or a word processor dedicated machine. In addition, the information processing apparatus includes an image display device (for example, a liquid crystal display unit of a monitor or a mobile phone) that displays various types of information in accordance with user request instructions. For example, in the present embodiment, the information processing apparatus is used for the purpose of preventing peeping by others other than the user when the information terminal is used as an information retrieval apparatus that retrieves synonyms from a database. Further, for example, the information processing apparatus is used for the purpose of preventing peeping by others other than the user when the information terminal is used as a kana-kanji conversion apparatus having a function of searching for synonyms.

  Specifically, the imaging unit 1 is a camera capable of capturing still images and moving images, such as a video camera or a CCD (Charge Coupled Devices). The imaging unit 1 is installed in the information processing apparatus so that the user can be photographed while viewing the screen displayed by the information processing apparatus. In the present embodiment, it is installed so as to photograph the front of the image display apparatus so that an image including the user of the information processing apparatus and the user's periphery can be photographed. FIG. 2 is an explanatory diagram illustrating an example of an image captured by the imaging unit 1. As illustrated in FIG. 2, the imaging unit 1 captures an image including a user (user) and others around the user.

  In addition, the imaging unit 1 is arranged around a data output unit such as a liquid crystal monitor screen or an LED display unit of the information processing apparatus that the user gazes in order to be able to photograph the user's face of the information processing apparatus in a good state It is preferable that it is installed in.

  Specifically, the data processing device 2 is realized by a CPU of an information terminal that operates according to a program. As shown in FIG. 1, the data processing device 2 includes a face detection unit 21, a peep determination unit 22, and a warning generation unit 23.

  The face detection unit 21 has a function of detecting a face included in an image captured by the imaging unit 1. In the present embodiment, the face detection unit 21 performs a predetermined face detection process on the image captured by the imaging unit 1 to detect a face included in the image. . For example, the face detection unit 21 first detects a black eye position candidate from the image by assuming that the gray level value of the target pixel is smaller than the gray level values of surrounding pixels as a black eye candidate. The face detection means 21 assumes that the combination of the detected positions of the black eyes is a face candidate. Then, the face detection means 21 uses a discriminator that discriminates between the face candidates obtained in this way and images other than the face previously learned by a neural network or the like, so that each face candidate is really a face. It is determined whether or not. As a result of the determination, the face detection means 21 outputs the position of the face candidate determined to be a face as a place where the possibility that the face actually exists is high.

  The above high-speed face detection processing is described in, for example, “Suzuki, Hosoi, Sakurai, Sato,“ Development of high-speed face detection processing using a ring filter ”, 2003 IEICE General Conference Proceedings, p. 251”. Are listed.

  The peeping determination unit 22 has a function of determining whether or not the display screen of the information processing apparatus is being peeped by someone other than the user based on the face detection result (number of detected faces) by the face detection unit 21. Prepare. Hereinafter, the state in which the display screen of the information processing apparatus is being peeped by someone other than the user is also referred to as a “peeping state”. In the present embodiment, the peeping determination unit 22 determines whether or not the number of detected faces detected by the face detection unit 21 is equal to or greater than a predetermined threshold (for example, n predetermined values). When the peeping determination unit 22 determines that the number of detected faces detected by the face detection unit 21 is less than n, the peeping determination unit 22 determines that it is not a peeping state by another person. Further, when the peeping determination unit 22 determines that the number of detected faces is n or more, the peeping determination unit 22 determines that it is a peeping state by another person. In the present embodiment, the predetermined value n is a preset value (for example, n ≧ 1).

  The warning generating unit 23 outputs a warning for notifying the user that the peeping state is present when the peeping determination unit 22 determines the peeping state as a warning output unit. 3 is provided.

  The warning output unit 3 has a function of outputting a warning for notifying the user that the user is peeping according to an instruction from the warning generation unit 23. The warning output unit 3 may be any means capable of sending some information to the user of the information processing device. Specifically, the warning output unit 3 may be an audio output device (not shown) such as a speaker or an image such as a monitor. This is realized by a display device (not shown). For example, when sound is used as a warning, a sound output device such as a speaker is used as the warning output unit 3. In this case, the warning output unit 3 warns the user of the peeping state by outputting a sound or a warning sound. For example, when an image is used as a warning, an image display device such as a monitor is used as the warning output unit 3. In this case, the warning output unit 3 warns the user of the peeping state with the image or the character information.

  In the present embodiment, the warning output unit 3 notifies the user that the user is in a peeping state, for example, by displaying an image, outputting sound, vibrating the information processing apparatus, or emitting light. Further, the warning output unit 3 notifies the user that the user is in a peeping state by combining any two or more of image display, sound output, information processing device vibration, and light emission. The warning output unit 3 notifies the user that the user is in the peeping state by using all of the display of the image, the output of the sound, the vibration of the information processing apparatus, and the light emission.

  In the present embodiment, the warning output unit 3 is a peeping state canceling unit that executes a canceling process for canceling the peeping state when the peeping determination unit 22 determines that the peeping state is in the peeping state. Function.

  Next, the operation will be described. FIG. 3 is a flowchart illustrating an example of a process for determining whether or not a display screen is peeped by another person other than the user and warning the user. First, the imaging unit 1 captures an image obtained by capturing the front of the image display device included in the information processing apparatus at a predetermined timing (step S11). For example, when detecting an instruction input by a user operation (for example, an instruction input for displaying various information on the display screen), the data processing apparatus 2 instructs the imaging unit 1 to capture an image. Then, the imaging unit 1 captures an image in front of the image display device in accordance with an instruction from the data processing device 2.

  Next, the face detection unit 21 of the data processing device 2 performs face detection processing on the image captured by the imaging unit 1 and extracts all the faces included in the captured image (step S12). Further, the peeping determination unit 22 determines whether n (n ≧ 1) or more faces are included in the captured image based on the face detection result detected by the face detection unit 21 (step S13). ).

  When the peeping determination unit 22 determines that n or more faces are included in the photographed image (that is, when it is determined that the peeping state is present), the warning generation unit 23 is peeped at the display screen by another person. Create warning information to alert the user to the effect. Further, the peeping determination unit 22 causes the information processing apparatus user (user) to output a warning to the warning output unit 3 based on the generated warning information (step S14), and ends the process. For example, the warning generation unit 23 causes the warning output unit 3 that is a speaker to output a warning using sound or a warning sound. Further, for example, the warning generation unit 23 causes the warning output unit 3 that is a monitor to output a warning using image or character information.

  When the peeping determination unit 22 determines that n or more faces are not included (that is, when the peeping determination unit 22 determines that the peeping determination unit 22 is not in a peeping state), the information processing apparatus performs the process as it is without outputting a warning to the user. finish.

  As described above, according to the present embodiment, the information processing apparatus captures images of the front and front periphery of the image display apparatus attached to the information processing apparatus using the imaging unit 1. Then, the information processing apparatus performs peeping determination using the number of faces present in the captured image to determine whether or not the peeping state is set. For this reason, even if the image display device attached to the information processing device can be peeped from the same viewpoint as the user of the information processing device, a warning is given to the user that the peeping state is detected by detecting multiple faces in the captured image. Can be notified. Therefore, even when another person looks into the screen from almost the same viewpoint as the user, it is possible to prevent another person from looking into the screen.

  Further, according to the present embodiment, the image capturing unit 1 is installed so as to photograph the front and front periphery of the image display device attached to the information processing apparatus, and a constraint condition (for example, between the user and the image display device) No special film sheet). Therefore, it is possible to notify the user of a peep warning without changing the display characteristics of the image display device (that is, without reducing the image quality of the image display device). Accordingly, it is possible to prevent a person from peeping at the screen without deteriorating the image quality of the display screen of the image display device attached to the information processing device.

  Further, according to the present embodiment, face detection and peeping determination can be performed without a specific operation by the user of the information processing apparatus. Therefore, it is possible to perform peep determination regardless of the use environment of the information processing apparatus. Further, according to the present embodiment, it is possible to determine whether or not the user is in the peeping state without performing the matching process using the face data registered in advance, so that the processing load on the information processing apparatus can be reduced. Therefore, it is possible to prevent the screen from being looked at by a simple method without registering the face data or attribute data of the authorized user.

Embodiment 2. FIG.
Next, a second embodiment of the present invention will be described with reference to the drawings. FIG. 4 is a block diagram illustrating another configuration example of the information processing apparatus having the peeping prevention function. As shown in FIG. 4, in the present embodiment, the data processing device 4 includes the display switching means 24 instead of the alarm generation means 23 of the data processing device 2 shown in FIG. Different from form. The information processing apparatus is different from the first embodiment in that the information processing apparatus includes an image output unit 5 instead of the warning output unit 3. Note that the functions of the imaging unit 1, the face detection unit 21, and the peeping determination unit 22 are the same as those shown in the first embodiment.

  Specifically, the image output unit 5 is realized by an image display device such as a monitor. The image output unit 5 has a function of displaying a screen for displaying various types of information according to an instruction from the data processing device 4 according to an operation of the information processing device user (user).

  The display switching unit 24 has a function of switching the image displayed by the image output unit 5 to a predesignated image when it is determined that the peeping state is determined by the peeping determination unit 22 as a result of the peeping state by another person. Is provided. For example, the display switching unit 24 switches the screen displayed by the image output unit 5 to an image preset by the user. In addition, for example, the display switching unit 24 switches the screen displayed by the image output unit 5 to a black image.

  For example, it is assumed that the information processing apparatus is a mobile phone. In this case, when the image output unit 5 displays personal information such as an email or a telephone book, if the peeping determination unit 22 determines that another person is peeping, the display switching unit 24 displays the image output unit 5. The information displayed by 5 is switched to the standby screen of the mobile phone. By doing so, it is possible to prevent someone other than the user from peeping at the personal information.

  In the present embodiment, the display switching unit 24 is a peeping state canceling unit that executes a canceling process for canceling the peeping state when the peeping determination unit 22 determines that the peeping state is in the peeping state. Function.

  Next, the operation will be described. FIG. 5 is a flowchart illustrating an example of a process for determining whether or not the display screen is peeped by another person other than the user and switching the display screen of the information processing apparatus. In the present embodiment, the imaging unit 1 captures an image in accordance with the same process as that in step S11 described in the first embodiment, and the face detection unit 21 includes steps S12 and S12 described in the first embodiment. A face is detected from the captured image according to similar processing. Then, the peeping determination unit 22 determines whether or not n or more faces are included in the photographed image, according to the same processing as Step S13 shown in the first embodiment.

  In the first embodiment, when the peeping determination unit 22 determines that the peeping state is in the peeping state, the warning output unit 3 simply outputs a warning, and the image displayed by the image display unit attached to the information processing apparatus is displayed as it is. Output (display). In the present embodiment, when the peeping determination unit 22 determines that the peeping state is in the peeping state, the display switching unit 24 switches the image output (displayed) by the image output unit 5 to a predetermined image (step S24). The process is terminated. If it is determined that the data processing device 4 is not in the peeping state, the information processing device ends the process without switching the image output by the image output unit 5.

  As described above, according to the present embodiment, when detecting the peeping state, the information processing apparatus performs control so that the information displayed by the image output unit 5 is not displayed (the image is switched). Therefore, even if the user of the information processing apparatus does not perform a specific operation to eliminate peeping by others, peeping by others can be prevented.

Embodiment 3 FIG.
Next, a third embodiment of the present invention will be described with reference to the drawings. FIG. 6 is a block diagram illustrating still another configuration example of the information processing apparatus having the peeping prevention function. As shown in FIG. 6, in the present embodiment, the data processing device 6 includes a peeping person recording means 25 in place of the alarm generation means 23 of the data processing device 2 shown in FIG. Different from the embodiment. The information processing apparatus is different from the first embodiment in that the information processing apparatus includes a storage device 7 instead of the warning output unit 3. Note that the functions of the imaging unit 1, the face detection unit 21, and the peeping determination unit 22 are the same as those shown in the first embodiment.

  Specifically, the storage device 7 is a memory, a hard disk device, or the like. As shown in FIG. 6, the storage device 7 includes a peeping person storage unit 71 for recording a person who peeped at the image display apparatus attached to the information processing apparatus. In the present embodiment, the storage device 7 stores an image in which a person who is peeping at a screen displayed by the information processing apparatus is shown in accordance with an instruction from the peeping person recording unit 25.

  When the peeping determination unit 22 determines that the peeping determination unit 22 is in the peeping state, the peeping person recording unit 25 peeks out the image to be determined (the image taken by the imaging unit 1) when determined as the peeping state. A function of saving in the person storage unit 71 is provided.

  The peeping person recording unit 25 does not store the image taken by the imaging unit 1 as it is in the peeping person storage unit 71, but extracts only the part where the face is reflected from the image and peeks at the extracted partial image. You may make it memorize | store in the person memory | storage part 71. FIG. In this case, the peeping person recording unit 25 extracts an area including the face detected by the face detection unit 21 from the image captured by the imaging unit 1. Then, the peeping person recording unit 25 stores the extracted area image in the peeping person storage unit 71.

  Further, in this embodiment, the peeping person recording unit 25 executes the cancellation process for eliminating the peeping state when the peeping determination unit 22 determines that the peeping state is in the peeping state. Functions as a means.

  Next, the operation will be described. FIG. 7 is a flowchart illustrating an example of processing for determining whether or not a display screen is peeped by a person other than the user and recording the peeping person. In the present embodiment, the imaging unit 1 captures an image in accordance with the same process as that in step S11 described in the first embodiment, and the face detection unit 21 includes steps S12 and S12 described in the first embodiment. A face is detected from the captured image according to similar processing. Then, the peeping determination unit 22 determines whether or not n or more faces are included in the photographed image, according to the same processing as Step S13 shown in the first embodiment.

  In the first embodiment, when the peeping determination unit 22 determines that the peeping state is present, the warning output unit 3 only outputs a warning, and the image when the peeping state is determined is stored. I didn't go. In the present embodiment, when the peeping determination unit 22 determines that the peeping state is in the peeping state, the peeping person recording unit 25 displays the image to be determined when it is determined that the peeping state is in the peeping state storage unit. 71 (step S34), and the process ends. That is, the peeping person recording unit 25 stores the photographed image taken by the imaging unit 1 in step S11 in the peeping person storage unit 71. If it is determined that the data processing device 6 is not in the peeping state, the information processing device ends the process without storing the photographed image in the peeping person storage unit 71.

  As described above, according to the present embodiment, when the information processing apparatus detects a peeping state, the information processing apparatus records a captured image when it is determined to be a peeping state. Therefore, a user or administrator of the information processing apparatus can confirm the face of the person who has peeped afterwards. Therefore, when an illegal act due to peeping occurs, it is possible to easily identify the person who has performed the illegal act. Further, according to the present embodiment, it is possible to expect the effect of suppressing the peeping action by recording the peeping state image, and the peeping action can be prevented in advance.

Embodiment 4 FIG.
Next, a fourth embodiment of the present invention will be described with reference to the drawings. FIG. 8 is a block diagram illustrating still another configuration example of the information processing apparatus having the peeping prevention function. As shown in FIG. 8, the present embodiment is different from the first embodiment in that the data processing device 8 includes display switching means 24 in addition to the components of the data processing device 2 shown in FIG. Different. Further, the information processing apparatus is different from the first embodiment in that it includes an image output unit 5 in addition to the components shown in FIG. The functions of the imaging unit 1, the face detection unit 21, the peeping determination unit 22, the warning generation unit 23, and the warning output unit 3 are the same as those described in the first embodiment.

  The image output unit 5 is specifically realized by an image display device such as a monitor, similarly to the image output unit 5 described in the second embodiment. Further, the image output unit 5 has a function of displaying a screen for displaying various types of information in accordance with an instruction from the data processing device 8 by an operation of the information processing device user (user).

  As in the case of the display switching unit 24 shown in the second embodiment, the display switching unit 24 determines that the peeping determination unit 22 has determined the peeping state, and as a result, A function of switching the image displayed by the output unit 5 to a predesignated image is provided.

  Next, the operation will be described. FIG. 9 is a flowchart illustrating an example of a process for determining whether or not someone is peeping at the display screen and warning the user and switching the display screen of the information processing apparatus. In the present embodiment, the imaging unit 1 captures an image in accordance with the same process as that in step S11 described in the first embodiment, and the face detection unit 21 includes steps S12 and S12 described in the first embodiment. A face is detected from the captured image according to similar processing. Then, the warning generation unit 23 determines whether or not n or more faces are included in the photographed image, according to the same processing as Step S13 shown in the first embodiment.

  In the first embodiment, when the peeping determination unit 22 determines that the peeping state is in the peeping state, the warning output unit 3 simply outputs a warning, and the image displayed by the image display unit attached to the information processing apparatus is displayed as it is. Output (display). In the second embodiment, when the peeping determination unit 22 determines that the peeping state is present, the image output unit 5 simply switches the image output (displayed) to a preset image, We didn't explicitly inform the user of the status.

  In the present embodiment, when the peeping determination unit 22 determines that the peeping state is present, the warning generation unit 23 warns the user that the peeping state is present via the warning output unit 3 (step S44). . Note that the method by which the warning output unit 3 warns the user of the peeping state is the same as the method described in the first embodiment (for example, notification using voice, warning sound, image, character information, etc.). .

  Further, the display switching unit 24 switches the image output (displayed) by the image output unit 5 to a predetermined image (step S45), and ends the process. Note that the order of the process of step S44 executed by the warning generation unit 23 and the process of step S45 executed by the display switching unit 24 are in no particular order. For example, if the peeping determination unit 22 determines that the peeping state is present, the warning generation unit 23 via the warning output unit 3 after the display switching unit 24 performs processing for causing the image output unit 5 to switch the display image. A warning process may be executed. When the peeping determination unit 22 determines that the peeping state is in the peeping state, the warning generation unit 23 and the display switching unit 24 may execute processing in parallel.

  As described above, according to the present embodiment, it is possible to prevent a peeping action by others as in the first embodiment and the second embodiment. For example, as in the first embodiment, the user can be notified of a warning that the user is in a peeping state, so even if another person looks into the screen from almost the same viewpoint as the user, You can prevent seeing. Further, it is possible to prevent others from peeping at the screen without degrading the image quality of the display screen of the image display device attached to the information processing device. Furthermore, it is possible to prevent a peep on the screen by a simple method without registering the face data or attribute data of the authorized user.

  In addition, according to the present embodiment, since the display image is controlled to be switched when the peeping state is detected, the user of the information processing apparatus does not have to perform a specific operation to eliminate peeping by others. , Peeping by others can be prevented.

Embodiment 5. FIG.
Next, a fifth embodiment of the present invention will be described with reference to the drawings. FIG. 10 is a block diagram illustrating still another configuration example of the information processing apparatus having the peeping prevention function. As shown in FIG. 10, in the present embodiment, the data processing device 9 replaces the peeping determination unit 22 of the data processing device 2 shown in FIG. 1 with a gaze estimation unit 26 and a screen gaze determination unit 27. It is different from the first embodiment in that it is included. Further, in the present embodiment, the function of the warning generating means 23A of the data processing device 9 is different from the function of the warning generating means 23 shown in the first embodiment. The functions of the imaging unit 1, the face detection unit 21, and the warning output unit 3 are the same as those shown in the first embodiment.

  The gaze estimation unit 26 has a function of estimating the gaze direction of the face included in the captured image based on the position of the face eye in the image detected by the face detection unit 21. In the present embodiment, the line-of-sight estimation means 26 estimates the direction of the line of sight of eyes contained in the face detected by the face detection means 21.

  The screen gaze determination means 27 is based on the position of the face in the image detected by the face detection means 21 and the estimation result of the line-of-sight estimation means 26 (the line-of-sight direction corresponding to the face in the image). It has a function of determining whether or not the detected face person is in a state of gazing at the display screen of the information processing apparatus (also referred to as gazing determination). Further, the screen gaze determination unit 27 has a function of determining whether or not there are n or more persons watching the screen. Further, the screen gaze determination unit 27 determines that it is a peeping state when it is determined that the number of persons determined to be watching the screen displayed by the information processing apparatus is n or more.

  In the present embodiment, the warning generating unit 23A causes the warning output unit 3 to look into the user in a state of peeping when the screen gaze determining unit 27 determines that there are n or more persons watching the screen. It has a function to output a warning for notifying that there is.

  Next, the operation will be described. FIG. 11 is a flowchart illustrating another example of processing for determining whether or not a display screen is peeped by another person other than the user and warning the user. In the present embodiment, the imaging unit 1 captures an image in accordance with the same process as that in step S11 described in the first embodiment, and the face detection unit 21 includes steps S12 and S12 described in the first embodiment. A face is detected from the captured image according to similar processing.

  In the first embodiment, after the face detection unit 21 detects a face in step S12, the peeping determination unit 23 counts the number of detected faces, and peeks when n or more faces are detected. It was determined to be in a viewing state. In the present embodiment, after the face detection unit 22 detects a face in the captured image in step S12, the line-of-sight estimation unit 26 estimates the line-of-sight direction of each detected face (step S53).

  FIG. 12 is an explanatory diagram illustrating an example of an estimation method for estimating the gaze direction of a face in an image. In step S53, the line-of-sight estimation means 26 detects, for example, the position of the black eye included in the detected eye position, and the position of the corner of the eye or the head of the eye based on the edge information as shown in FIG. The line-of-sight estimation means 26 estimates the line-of-sight direction of the person in the face in the image based on the positions of the left and right black eyes and the distance between the corners of the eyes and the eyes.

  For example, in the example shown in FIG. 12, the information processing apparatus stores in advance a correspondence table in which a correspondence relationship between the angles and the distances a, b, c, and d between the eye portions is described in a storage device or the like. In the example shown in FIG. 12, a is the distance between the corner of the left eye and the black eye, b is the distance between the left eye and the eye, c is the distance between the right eye and the black eye, and d is the right eye. It is the distance between the black eyes and the corners of the eyes. For example, when the detected distance b between the left eye's black eye and the eye's eye and the distance c between the right eye's eye and the black eye are the same (b = c), the line-of-sight estimation means 26 indicates that the face person in the image is in front. It is determined that it is suitable. If the detected distance b between the left eye's black eye and the eye's eye is larger than the distance c between the right eye's eye and the black eye (b> c), the line-of-sight estimation means 26 looks at the right side of the face person in the image. It is determined that

  The line-of-sight estimation means 26 learns the correspondence between the distances a, b, c, and d between each part of the eye and the line-of-sight direction using a learning model such as a neural network, and the line-of-sight direction based on the learning result. May be estimated.

  After the gaze estimation unit 26 estimates the gaze direction of each detected face, the screen gaze determination unit 27 determines whether each face in the screen is gazing at the screen based on the estimation result of the gaze estimation unit 26. Is determined (step S54).

  FIG. 13 is an explanatory diagram illustrating an example of a determination method for determining whether a face in an image is gazing at the display screen of the information processing apparatus. As shown in FIG. 13, the screen gaze determination means 27 is in a screen gaze state where a face person in the image gazes at the screen based on the position where the face is detected and the gaze direction of the face. Determine whether or not. In the example illustrated in FIG. 13, when it is estimated that the face person detected in the center of the image is looking in front, the screen gaze determination unit 27 determines that the screen is in the screen gaze state. When it is estimated that the face detected in the center of the image is looking in a direction other than the front, the screen gaze determination unit 27 determines that the screen is not in the screen gaze state. Similarly, when it is estimated that the face person detected on the left side of the image is looking at the right side, the screen gaze determination unit 27 determines that the screen is in the screen gaze state. When it is estimated that the face person detected on the left side of the image is looking in a direction other than the right side, the screen gaze determination unit 27 determines that the screen is not in the screen gaze state.

  Further, the screen gaze determination unit 27 determines whether or not the number of faces of the person who has determined that the screen is being watched is n (n ≧ 1) or more (step S55). If it is determined that the number of faces of the person determined to be in the gaze state is less than n, the information processing apparatus ends the process as it is. When it is determined that the number of faces of the person determined to be in the gaze state is n or more, the warning generation unit 23 outputs a signal for warning the peeping state to the warning output unit 3 to warn the user. Is output to the warning output unit 3 (step S56), and the process is terminated.

  In this embodiment, the case where the information processing apparatus outputs a warning to the user when the peeping state is detected will be described. However, the information processing apparatus performs other processing when the peeping state is detected. You may make it perform. For example, the information processing apparatus includes a display switching unit 24 and an image output unit 5 in addition to the components shown in FIG. 9, and when the peeping state is detected, the display screen displayed by the image output unit 5 is switched. Good. Further, for example, the information processing apparatus includes a peeping person recording unit 25 and a peeping person storage unit 71 in addition to the components shown in FIG. 9. When a peeping state is detected, a person who is peeping is detected. The photographed image taken may be stored in the peeping person storage unit 71.

  As described above, according to the present embodiment, the information processing apparatus not only detects a face in an image but also detects a point at which a person whose face is detected gazes, thereby enabling a more accurate peeking. A determination can be made. Therefore, compared with the information processing apparatus shown in the first to fourth embodiments, it is possible to prevent a peeping action by others with higher accuracy.

  For example, when judging whether or not a person is peeping based on only the number of faces in the captured image, the face of a bona fide person who does not intend to peep is accidentally captured. However, there is a possibility that it is determined to be in a peeping state. According to the present embodiment, it is determined whether or not the face person in the image is gazing at the display screen of the information processing apparatus by estimating the direction of the line of sight of the face in the image. And it is determined whether it is a peeping state based on the number of the faces which are gazing at the display screen among the faces in an image. Therefore, it is possible to determine whether or not it is a peeping state with higher accuracy than in the case of determining based only on the number of faces in the image.

  Next, the operation of the information processing apparatus having a peep prevention function will be described using a specific embodiment. FIG. 14 is an explanatory diagram illustrating a specific example of an information processing apparatus having a peeping prevention function. Note that the information processing apparatus illustrated in FIG. 14 corresponds to the information processing apparatus illustrated in the fourth embodiment. Further, as shown in FIG. 14, in the present embodiment, a case where the information processing apparatus is a foldable mobile phone will be described as an example.

  As shown in FIG. 14, the mobile phone includes, as the imaging unit 1, a camera device (for example, a CCD camera) for photographing a user who looks at a liquid crystal monitor. Further, as shown in FIG. 14, the camera device is attached to a surface that becomes the inner side when the cellular phone is folded. The camera device is attached around the liquid crystal monitor.

  In the mobile phone, the data processing device 8 is a CPU of the mobile phone that operates according to a program. In the mobile phone, the warning output unit 3 is realized by a speaker, a vibration unit, or the like. In the mobile phone, the image output unit 5 is a liquid crystal monitor.

  The mobile phone has a central processing unit (CPU) that functions as the face detection means 21, the peek determination means 22, the warning generation means 23, and the display switching means 24.

  In this example, a case will be described in which the user of the mobile phone displays important information that the user does not want to be seen by others, such as a phone book or mail. In this case, the mobile phone displays these important information, and automatically activates the camera device and the face detection means 21, and displays the image including the user viewing the liquid crystal monitor of the mobile phone and the periphery of the user. Take a picture using. The mobile phone performs face detection processing on the captured image. For example, when the number of detected faces is two or more, the mobile phone determines that peeping has occurred. The mobile phone alerts the user by vibration using the vibrator function.

  In addition, the mobile phone performs a warning by vibration, and at the same time, switches the display screen of the liquid crystal monitor unit from the important information screen to a screen displaying the face detection result. For example, as shown in FIG. 15, the mobile phone switches to a screen that displays a detection result of detecting the user's own face and the face of another person who is looking into the screen. As described above, by switching the important information to another image, it is possible to eliminate peeping by others other than the user and notify the user that the peeping state has been established.

  INDUSTRIAL APPLICABILITY The present invention can be applied to an information processing apparatus that displays important information such as a mobile phone or a personal computer, for preventing or preventing peeping by others other than the user. For example, the present invention can be applied to the use of an information search device that searches for synonyms from a database and the use of a program for realizing the information search device using a computer. Further, for example, the present invention can be applied to the use of a personal computer or a word processor dedicated machine equipped with a Kana-Kanji conversion device having a function for searching for synonyms, or the use of a program that realizes a Kana-Kanji conversion function using a computer. .

It is a block diagram which shows an example of a structure of the information processing apparatus provided with the peep prevention function by this invention. It is explanatory drawing which shows the example of the image which an imaging part image | photographs. It is a flowchart which shows an example of the process which determines whether it is peeping at the display screen by others other than a user, and warns a user. It is a block diagram which shows the other structural example of the information processing apparatus provided with the peep prevention function. It is a flowchart which shows an example of the process which determines whether other persons other than a user are peeping at the display screen, and switches the display screen of information processing apparatus. FIG. 11 is a block diagram illustrating still another configuration example of the information processing apparatus having a peep prevention function. It is a flowchart which shows an example of the process which determines whether other persons other than a user are peeping at the display screen, and records the person who is peeping. FIG. 11 is a block diagram illustrating still another configuration example of the information processing apparatus having a peep prevention function. It is a flowchart which shows an example of the process which determines whether other persons other than a user are peeping at a display screen, warns a user, and switches the display screen of an information processing apparatus. FIG. 11 is a block diagram illustrating still another configuration example of the information processing apparatus having a peep prevention function. It is a flowchart which shows other examples of the process which determines whether it is peeping at the display screen by others other than a user, and warns a user. It is explanatory drawing which shows the example of the estimation method which estimates the gaze direction of the face in an image. It is explanatory drawing which shows the example of the determination method which determines whether the face in an image is gazing at the display screen of information processing apparatus. It is explanatory drawing which shows the specific example of the information processing apparatus provided with the peep prevention function. It is explanatory drawing which shows the example of the screen which displays a face detection result.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image pick-up part 2,4,6,8,9 Data processing apparatus 3 Warning output part 5 Image output part 7 Storage device 21 Face detection means 22 Peep determination means 23 Warning generation means 24 Display switching means 25 Peep person recording means 26 Line-of-sight estimation means 27 Screen gaze determination means 71 Peeping person storage unit

Claims (27)

An imaging unit that is installed in the information processing apparatus so as to photograph the user in a state where the user is viewing a screen displayed by the information processing apparatus, and that captures an image including the user;
Face detection means for detecting a face included in an image captured by the imaging unit;
Based on the number of faces detected by the face detection means, it is determined whether or not the information displayed by the information processing device is in a peeping state in which a person other than the user is peeping A determination means;
An information processing apparatus comprising: a peeping state canceling unit that executes a canceling process for canceling the peeping state when the peeping determination unit determines that the peeping state is in a peeping state.   The peeping determination means determines whether or not the number of faces detected by the face detection means is a predetermined number or more, and when determining that the number of faces detected by the face detection means is a predetermined number or more, The information processing apparatus according to claim 1, wherein the information processing apparatus is determined to be in a peeping state. An imaging unit that is installed in the information processing apparatus so as to photograph the user in a state where the user is viewing a screen displayed by the information processing apparatus, and that captures an image including the user;
Face detection means for detecting a face included in an image captured by the imaging unit;
Eye gaze estimation means for estimating the direction of the eye gaze included in the face detected by the face detection means;
Screen gaze determining means for determining whether the person of the face detected by the face detecting means is gazing at the screen displayed by the information processing device based on the estimation result of the gaze estimating means;
A peeping state canceling means for executing a canceling process for canceling the peeping state,
The screen gaze determining means determines that it is in a peeping state when it is determined that the number of persons determined to be gazing at the screen displayed by the information processing apparatus is a predetermined number or more,
The information processing apparatus according to claim 1, wherein the peeping state cancellation unit executes the cancellation process when the screen gaze determination unit determines that the peeping state determination unit is in a peeping state.   The information processing apparatus according to any one of claims 1 to 3, further comprising an image display device that displays information in accordance with a request instruction of a user of the information processing apparatus.   5. The information processing according to claim 1, wherein the peeping state elimination unit is a warning output unit that outputs a warning for notifying the user that the peeping state is in a peeping state. apparatus.   6. The information processing apparatus according to claim 5, wherein the warning output means notifies the user that the user is in a peeping state by displaying an image, outputting sound, vibrating the information processing apparatus, or emitting light.   The warning output means notifies the user that the user is in a peeping state by a combination of any two or more of image display, sound output, vibration of the information processing apparatus, or light emission. Information processing device.   6. The information processing apparatus according to claim 5, wherein the warning output means notifies the user that the user is in a peeping state using all of display of an image, output of sound, vibration of the information processing apparatus, and light emission.   The information processing apparatus according to claim 4, wherein the peeping state elimination unit is an image display switching unit that switches information displayed by the image display device.   The information processing apparatus according to claim 9, wherein the image display switching unit switches a screen displayed by the image display apparatus to an image preset by a user of the information processing apparatus.   The information processing apparatus according to claim 9, wherein the image display switching unit switches a screen displayed by the image display apparatus to a black image. A peek image storage means for storing an image of a person peeping at a screen displayed by the information processing apparatus;
The peeping state eliminating unit is a peeping image recording unit that stores an image taken by the imaging unit in the peeping image storage unit when it is determined that the peeping state is in a peeping state. The information processing apparatus according to any one of the above.   13. The information according to claim 12, wherein the peeping image recording unit extracts a region including the face detected by the face detection unit from the image captured by the imaging unit, and stores the image of the extracted region in the peeping image storage unit. Processing equipment. In accordance with a request instruction from the user of the information processing apparatus, an image display device that displays information is provided.
The information processing apparatus according to claim 1, wherein an imaging unit is installed around the image display apparatus. An imaging step of capturing an image including a user of the information processing apparatus and the periphery of the user;
A face detection step for detecting a face included in the captured image;
Based on the number of detected faces, the information displayed by the information processing device is a peeping determination step for determining whether the information is a peeping state that is being peeped by a person other than the user;
An information processing method comprising: a peeping state elimination step for executing a elimination process for eliminating the peeping state when it is determined that the peeping state is established.   In the peeping determination step, it is determined whether or not the number of faces detected in the face detection step is equal to or greater than a predetermined number, and when it is determined that the number of detected faces is equal to or greater than a predetermined number, the peeping state is established. The information processing method according to claim 15, which is determined as follows. An imaging step of capturing an image including a user of the information processing apparatus and the periphery of the user;
A face detection step for detecting a face included in the captured image;
A line-of-sight estimation step for estimating the direction of the line of sight of eyes contained in the detected face;
Screen gaze determination step for determining whether or not the detected face person is gazing at the screen displayed by the information processing device based on the estimation result in the gaze estimation step;
Including a peeping state cancellation step for executing a cancellation process for canceling the peeping state,
In the screen gaze determination step, when it is determined that the number of persons determined to be gazing at the screen displayed by the information processing apparatus is a predetermined number or more, it is determined that the peeping state is present,
The information processing method according to claim 1, wherein when the peeping state elimination step determines that the peeping state is established, the elimination process is executed.   The information processing method according to any one of claims 15 to 17, further comprising a step of displaying information on the image display device in accordance with a request instruction of a user of the information processing device.   The information processing according to any one of claims 15 to 18, further comprising a warning output step of outputting a warning for notifying the user of the peeping state in the peeping state elimination step. Method.   20. The information processing method according to claim 19, wherein in the warning output step, the user is notified of the peeping state by displaying an image, outputting sound, vibrating the information processing apparatus, or emitting light.   20. The warning output step of notifying the user of a peeping state by a combination of any two or more of image display, sound output, information processing device vibration, or light emission. Information processing method.   20. The information processing method according to claim 19, wherein in the warning output processing step, the user is notified of the peeping state by using all of image display, sound output, vibration of the information processing apparatus, and light emission.   The information processing method according to claim 18, further comprising an image display switching step of switching information displayed by the image display device in the peeping state elimination step.   24. The information processing method according to claim 23, wherein in the image display switching step, a screen displayed by the image display device is switched to an image preset by a user of the information processing device.   24. The information processing method according to claim 23, wherein in the image display switching step, the screen displayed by the image display device is switched to a black image.   When it is determined in the peeping state elimination step that the peeping state is determined, the image taken in the imaging step is stored as an image of a person who is peeping at the screen displayed by the information processing device. The information processing method according to any one of claims 15 to 25, further comprising a peek image recording step of storing in an image storage means.   27. Information according to claim 26, wherein in the peek image recording step, an area including the face detected in the face detection step is extracted from the image taken in the imaging step, and the image of the extracted area is stored in the peep image storage means. Processing method.

Images