WO2007004536A1

WO2007004536A1 - Information processing device and information processing method

Info

Publication number: WO2007004536A1
Application number: PCT/JP2006/313062
Authority: WO
Inventors: Tetsuaki Suzuki
Original assignee: Nec Corporation
Priority date: 2005-07-04
Filing date: 2006-06-30
Publication date: 2007-01-11
Also published as: JP2008276274A

Abstract

An imaging section (1) of an information processing device captures an image of the area in front of an image display additionally provided to the information processing device. Face detecting means (21) detects faces shown in the captured image. Number-of-faces reporting means (22) judges from the results of face detection whether or not n (e.g., n=2) or more faces are shown in the image. Warning issuing means (23) creates warning information if the number-of-faces reporting means (22) judges that n or more faces are shown and instructs a warning outputting section(3) to issue a warning to the user of the information processing device. The information processing device ends the processing without doing anything if the number-of-faces reporting means (22) judges that n or more faces are not shown.

Description

Specification

Information processing apparatus and information processing method

Technical field

TECHNICAL FIELD [0001] The present invention relates to an information processing apparatus and an information processing method having an innocent look prevention function, and in particular, when a user uses an image display apparatus attached to an information processing apparatus, a display screen is displayed by another person. Information processing apparatus and information processing equipped with an anti-seeing function that detects that the pretense is seen, warns the user (user), and hides the information displayed on the image display device from others Regarding the method.

Background art

[0002] The display screens of image display devices attached to information processing devices such as mobile phones, PCs (Personal Computers), ATMs (Auto Teller Machines), etc. For example, there is a method of attaching a specially processed film sheet for narrowing the viewing angle of the screen to the screen.

[0003] Further, for example, Patent Document 1 describes an image display device that prevents information from being viewed by looking into from behind a train in a train or the like. The image display device described in Patent Literature 1 includes an imaging unit, an image display unit, a detection unit that detects the position of the face eye, and a display position change unit.

[0004] An image display apparatus using the conventional method for preventing looking at eyes having the above-described configuration operates as follows. The image display device detects the position of the face eye by image recognition from the image taken by the imaging means using the detection means. Further, the image display device uses the display position changing unit to change the display position of the image displayed by the image display unit based on the detection result of the detecting unit. Therefore, the position of the image displayed on the screen changes according to the relative position between the position of the user's eyes and the position of the imaging means. As a result, in environments where the user's head is different from other people's heads (for example, when displaying an image in a train, etc.), the display image appears to be blurred by others, preventing information from being seen. Can do.

[0005] Further, for example, Non-Patent Document 1 describes a prestigious performance by a person other than a registered regular user. An absence management system that prevents intrusion is described. The away management system described in Non-Patent Document 1 is an imaging means, an image display means, a means for detecting the position of a face, and whether the captured face corresponds to a face of a person registered in advance. A face matching means for determining power failure.

[0006] The away management system using the conventional method for preventing looking at eyes having the above-described configuration operates as follows. The away management system detects the face from the image taken by the image pickup means by using the image recognition by the detection means. Further, the away management system extracts face data from the image based on the detected eye position, and compares it with the regular use person data (user face image) registered in advance using the extracted face data. If the absence management system determines that the person is a regular user, the display screen continues to be displayed without changing the output of the image display means. Also, if the absence management system does not determine that the person is a regular user, the information displayed on the image display means is concealed. As a result, it is possible to prevent the information manager from being viewed by an unregistered person.

Patent Document 1: Japanese Patent Application Laid-Open No. 2004-317813 (paragraphs 0052-0064, FIG. 5-7)

Non-Patent Document 1: Masahiro Suzuki, “Fa _Ce M ₀ nitor”, Face Detection / Face Matching Engine “NeoFace”, Image Lab, March 2005, Japan, 2005, p54—57

Disclosure of the invention

Problems to be solved by the invention

[0008] When a specially processed film sheet is pasted on the screen, the image quality is degraded by stretching the film sheet on the screen. Therefore, when using the method of pasting a specially processed film sheet on the screen, the first problem is that even a legitimate user of an information processing device may see the screen. is there.

[0009] In addition, when a specially processed film sheet is pasted on the screen, when the screen is viewed from the back of an authorized user, when the screen is viewed from almost the same viewpoint as an authorized user, The viewpoint of another person other than the user is within the viewing angle of the screen. For this reason, even a person other than the authorized user can read the information displayed on the screen. Therefore, when using a method of sticking a specially processed film sheet to the screen, the second problem is that it is not possible to prevent snooping depending on the direction in which others look at the screen. There's a problem.

[0010] Further, in the image display device described in Patent Document 1, it is possible to prevent the display screen from being viewed using the shaking of the head. However, when the image display device is viewed in a situation where the head is stationary, it is not always possible to prevent the viewing by others other than the authorized user. For example

When operating a mobile phone at a seat in a coffee shop, operating a PC at an office seat, or operating an ATM at a bank, the head does not move as much as in the train. Therefore, even if the image display device described in Patent Document 1 is used, there is a possibility that an image displayed on the image display device does not move and can be easily seen by others. Therefore, when the image display device described in Patent Document 1 is used, as a third problem, there is a problem that viewing cannot be prevented depending on the place where the image display device is used.

[0011] In addition, when the away management system described in Non-Patent Document 1 is used, it is necessary to record in advance a registered data of a regular user used by the face matching unit for matching. For this reason, it is necessary to provide a storage device for storing the registration data, which increases the cost of the storage device. In particular, an information processing apparatus such as a portable terminal has a high storage capacity cost, which increases the cost burden. Therefore, when the away management system described in Non-Patent Document 1 is used, a fourth problem is that the cost of the information processing apparatus increases because the face matching means is used. Further, in the away management system described in Non-Patent Document 1, since the registered face data of the authorized user and the captured image are collated, the processing load on the information processing apparatus is large.

[0012] Therefore, the present invention provides an information processing apparatus and an information processing method capable of preventing a person from looking at the screen without degrading the image quality of the display screen of the image display apparatus attached to the information processing apparatus. With the goal. It is another object of the present invention to provide an information processing apparatus and an information processing method that can prevent others from looking at the screen even when another person looks at the screen from almost the same viewpoint as the user. To do. It is another object of the present invention to provide an information processing device and an information processing method that can prevent a person from seeing the screen without requiring movement of the head of a person who operates the information processing device. . Furthermore, the present invention prevents screen viewing by a simple method without registering the face data or attribute data of a regular user. It is an object to provide an information processing apparatus and an information processing method that can be stopped. Means for solving the problem

[0013] An information processing apparatus according to the present invention includes an imaging unit that captures an area for a person to browse a screen, a face detection unit that detects a face included in an image captured by the imaging unit, and the face detection unit A number-of-faces notification means for notifying the number of faces detected by. As the face number notification means, for example, a warning sound can be output, a screen can be switched, or a person who has been viewed can be recorded.

[0014] Adopting the configuration as described above, the imaging unit images the front of image display means (for example, a liquid crystal display unit of a monitor or a mobile phone) of the information processing apparatus, and a predetermined number of people or more are included in the captured image. By determining whether or not a human face is detected, it is possible to achieve the object of the present invention.

[0015] Further, in the information processing apparatus, the face number notifying unit performs the operation depending on whether or not the number of faces detected by the face detecting unit is equal to or greater than a preset number n (for example, n = 2). It can be a switch. For example, if the number of faces is equal to or greater than a predetermined number, a warning sound is generated, and if the number is less than the predetermined number, special processing is not performed.

[0016] An information processing apparatus according to another configuration of the present invention includes an imaging unit that captures an area for a person to view a screen, and face detection that detects a face included in an image captured by the imaging unit. And a line of sight estimation means for estimating the direction of the line of sight of eyes included in the face detected by the face detection means, and the person of the face detected by the face detection means based on the estimation result of the line of sight estimation means The screen gaze determining means for determining whether or not the screen is being watched, and the number of faces that are gaze at the screen based on the determination result of the screen gaze determining means. Face number notification means. As the face number notification means, for example, a warning sound can be output, a screen can be switched, or a person who has been viewed can be recorded.

In addition, the information processing device includes an image display device (for example, a liquid crystal display unit of a monitor or a cellular phone, an image output unit 5) that displays information in accordance with a user's request instruction of the information processing device. May be.

[0018] Further, in the information processing apparatus, the face number notifying means warns according to the number of detected faces. May be output.

[0019] Further, in the information processing apparatus, the warning output means notifies the user that the person is in a pretending state by displaying an image, outputting sound, vibrating the information processing apparatus, or emitting light. There may be.

[0020] Further, in the information processing apparatus, the warning output means gives the user an intimidation by combining two or more of image display, sound output, vibration of the information processing apparatus, and light emission. You may notify that it is in a state.

[0021] Further, in the information processing device, the warning output means notifies the user that the user is in a sighting state using all of the display of the image, the output of the sound, the vibration of the information processing device, and the light emission. You may do.

[0022] Further, in the information processing apparatus, the face number notifying unit may be an image display switching unit that switches information displayed by the image display device in accordance with the number of detected faces.

[0023] Further, in the information processing apparatus, the image display switching unit may switch a screen displayed by the image display apparatus to an image preset by a user of the information processing apparatus.

[0024] In the information processing apparatus, the image display switching unit may switch a screen displayed by the image display apparatus to a black image.

[0025] The information processing apparatus includes a browse image storage unit that stores an image of a person browsing a screen displayed by the information processing apparatus. The face number notification unit is captured by the imaging unit. When it is determined that the person included in the image is in a state of browsing the screen, the browsing image recording unit may store the image in the browsing image storage unit.

[0026] Further, the browsing image recording means extracts an area including the face detected by the face detection means from the image taken by the imaging section, and stores the image of the extracted area in the browsing image storage section. May be.

[0027] Further, the information processing device may include an image display device that displays information in accordance with an instruction requested by a user of the information processing device, and the imaging unit may be installed around the image display device. Let's go.

[0028] An information processing method according to the present invention is a method of imaging an area for a person to view a screen. When the information processing apparatus including the image processing unit executes a face detection step for detecting a face included in the image captured by the imaging unit and a face number notification step for notifying the number of faces detected by the face detection step. This is the way.

[0029] Employing the configuration as described above, the front of the image display means of the information processing device is photographed by the imaging unit, and it is determined whether or not faces of a predetermined number of people or more are detected in the photographed image. Thus, the object of the present invention can be achieved.

[0030] Further, in the information processing method, the number-of-faces notification step is notified depending on whether or not the number of faces detected by the face detection means is greater than or equal to a preset number n (for example, n = 2). You can change the operation. For example, if the number of faces is equal to or greater than a predetermined number, a warning sound is generated, and if the number is less than the predetermined number, special processing is not performed.

[0031] Further, the information processing method according to another configuration of the present invention is such that an information processing apparatus including an imaging unit that captures an area for a person to view a screen is captured by the imaging unit. Based on a face detection step for detecting a face included in an image, a line-of-sight estimation step for estimating the direction of the line of sight of an eye included in the face detected by the face detection step, and an estimation result of the line-of-sight estimation step, The screen gaze determination step for determining whether or not the face person detected by the face detection step gazes at the screen and the screen gaze determination step determines whether or not the screen gazes the screen. The number of faces for notifying the number of faces being notified is executed.

[0032] Further, the information processing method may include a step of displaying information on the image display device in accordance with an instruction requested by the user of the information processing device.

[0033] Further, the face number notification step may include a warning output step for outputting a warning according to the number of detected faces.

[0034] Further, in the warning output step, the information processing method notifies the user that the user is in a sighted state by displaying an image, outputting sound, vibrating the information processing device, or emitting light. May be.

[0035] In addition, the information processing method is a warning output step in which the user can view the image by using a combination of two or more of image display, sound output, information processing device vibration, or light emission. You may notify that it is a viewing state. [0036] In addition, the information processing method is a warning output processing step in which the user is in a pretentious state using all of image display, sound output, information processing device vibration, and light emission. May be notified.

[0037] The face number notifying step may include an image display switching step for switching information displayed by the image display device.

[0038] Further, the information processing method may switch the screen displayed by the image display device to an image preset by a user of the information processing device in the image display switching step.

[0039] Further, the information processing method may switch the screen displayed by the image display device to a black image in the image display switching step.

[0040] Further, the face number notifying step may include a browsing image recording step of storing in a browsing image storage unit that stores an image of a person browsing a screen displayed by the information processing apparatus. Good.

[0041] In the browsing image recording step, a region including the face detected in the face detection step is extracted from the image captured by the imaging unit, and the image of the extracted region is stored in the browsing image storage unit. It may be.

The invention's effect

[0042] According to the present invention, a face included in a photographed image is detected, and it is determined whether or not it is a viewing state based on the number of faces in the photographed image. According to the present invention, it is not necessary to narrow the viewing angle of the image display device attached to the information processing device, which does not require a special film sheet or the like to be attached to the display screen of the information processing device in order to prevent prejudice. In addition, the user can directly view a screen that does not need to be processed on the image display device itself. Therefore, it is possible to prevent others from seeing the screen without degrading the image quality of the display screen of the image display device attached to the information processing device.

[0043] Further, according to the present invention, a face is detected from an image obtained by photographing a user and an area including the back of the user, and determination based on the number of faces is performed to prevent a pretending to be seen. Therefore, even if another person looks at the screen from the same viewpoint as the user, it is possible to prevent the other person from looking at the screen. [0044] According to the present invention, a face is detected from an image of a person and the surroundings of the person regardless of the movement of the person operating the information processing apparatus, and the control is performed based on the number of detected faces. Judgment is made as to whether or not it is in a sight-seeing state. Therefore, even if the user is stationary, it is possible to prevent the act of sight. Therefore, it is possible to prevent the screen from being seen without requiring the movement of the head of the person who operates the information processing apparatus.

[0045] Further, according to the present invention, it is necessary to store data for identifying a regular user who does not need to perform face matching, personal attribute determination, or the like in order to determine whether or not the user is in a sighted state. There is no. Therefore, even if information related to the user is not stored in the storage means in advance, it is possible to prevent a viewing act. Further, since it is not necessary to perform collation processing using face data, the processing burden on the information processing apparatus can be reduced. Therefore, it is possible to prevent the screen from being viewed with a simple method without registering the face data or attribute data of the authorized user.

Further, in the present invention, it is configured to determine whether or not the person who has detected the face is gaze-in by detecting a point of gaze by simply detecting the face in the image. If this is done, it is possible to make a more accurate look determination. Therefore, it is possible to prevent a pretending look by another person with higher accuracy.

[0047] Further, in the present invention, if the information processing device is configured to switch the information displayed when the pretending state is detected, the user of the information processing device cancels the pretending look by others. Therefore, even if no specific operation is performed, it is possible to prevent other people from seeing and seeing.

[0048] In the present invention, when the presidency state is detected, if a configuration is made to store a photographed image when it is determined to be the presidency state, a user or an administrator of the information processing apparatus can You can check the face of the person who performed the act. Therefore, in the case where a misconduct occurs due to a sneak peek, it is possible to easily identify the person who performed the misconduct. In addition, it is possible to expect a deterrent effect of pretending to be seen by recording an image of the looking state, and it is possible to prevent the seeing action in advance.

Brief Description of Drawings

[0049] FIG. 1 is a block diagram showing an example of the configuration of an information processing apparatus having a look prevention function according to the present invention. FIG. 2 is an explanatory diagram illustrating an example of an image captured by an imaging unit.

FIG. 3 is a flowchart showing an example of a process for determining whether or not a display screen is viewed by another person other than the user and warning the user.

FIG. 4 is a block diagram showing another configuration example of the information processing apparatus having a look prevention function.

FIG. 5 is a flowchart showing an example of processing for determining whether or not a display screen is viewed by another person other than the user and switching the display screen of the information processing apparatus.

FIG. 6 is a block diagram showing still another configuration example of the information processing apparatus having a look prevention function.

FIG. 7 is a flowchart showing an example of a process for determining whether or not a display screen is viewed by a person other than the user and recording the person being viewed.

FIG. 8 is a block diagram showing still another configuration example of the information processing apparatus having a look prevention function.

FIG. 9 is a flowchart showing an example of processing for determining whether or not a display screen is seen by another person other than the user and warning the user and switching the display screen of the information processing apparatus. .

FIG. 10 is a block diagram showing still another configuration example of the information processing apparatus having the pretending to look at prevention function.

FIG. 11 is a flowchart showing another example of a process for determining whether or not a display screen is viewed by another person other than the user and warning the user.

FIG. 12 is an explanatory diagram showing an example of an estimation method for estimating the gaze direction of a face in an image.

FIG. 13 is an explanatory diagram showing an example of a determination method for determining whether or not a face in an image is gazing at the display screen of the information processing apparatus.

FIG. 14 is an explanatory diagram showing a specific example of an information processing apparatus having a look prevention function.

FIG. 15 is an explanatory diagram showing an example of a screen displaying a face detection result.

Explanation of symbols

1 Imaging unit

2, 4, 6, 8, 9 Data processor

3 Warning output section 5 Image output section

7 Storage device

21 Face detection means

22 Number of faces notification means

23 Warning generation methods

24 Display switching means

25 Browsing person recording means

26 Gaze estimation means

27 Screen gaze judgment means

71 Browsing person storage

BEST MODE FOR CARRYING OUT THE INVENTION

[0051] Embodiment 1.

Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of a configuration of an information processing apparatus having a pretending to look prevention function according to the present invention. As shown in FIG. 1, the information processing device includes an imaging unit 1 for inputting an image, a data processing device 2 that operates under program control, and a warning by sound, image, vibration, light, etc. And a warning output unit 3 for outputting the above.

In the present embodiment, the information processing apparatus is specifically realized by an information terminal such as a mobile phone, a personal computer, a word processor dedicated machine, or the like. In addition, the information processing device includes an image display device (for example, a liquid crystal display unit of a monitor or a mobile phone) that displays various types of information in accordance with user request instructions. For example, in the present embodiment, the information processing apparatus is used for the purpose of preventing a person other than the user from seeing when the information terminal is used as an information retrieval apparatus that retrieves a synonym from a database. . In addition, for example, the information processing device is used for the purpose of preventing a person other than the user from seeing and looking when the information terminal is used as a powerful Kanji conversion device having a function of searching for synonyms.

[0053] Specifically, the imaging unit 1 is a camera capable of capturing still images and moving images, such as a video camera or a CCD (Charge Coupled Device) camera. The imaging unit 1 It is installed in the information processing device so that the user can take a picture while viewing the screen displayed by the information processing device. In the present embodiment, it is installed so as to photograph the front of the image display apparatus so that an image including the user of the information processing apparatus and the user periphery can be photographed. FIG. 2 is an explanatory diagram illustrating an example of an image captured by the imaging unit 1. As shown in FIG. 2, the imaging unit 1 captures an image including a user (user) and others around the user.

[0054] In addition, the imaging unit 1 outputs data such as a liquid crystal monitor screen or an LED display unit of the information processing apparatus that the user is gazing in so that the user's face of the information processing apparatus can be photographed in a good state. It is recommended to be installed around the department.

[0055] Specifically, the data processing device 2 is realized by a CPU of an information terminal that operates according to a program. As shown in FIG. 1, the data processing device 2 includes a face detection unit 21, a face number notification unit 22, and a warning generation unit 23.

[0056] The face detection means 21 has a function of detecting a face included in an image captured by the imaging unit 1. In the present embodiment, the face detection means 21 performs a predetermined face detection process on the image captured by the imaging unit 1 to detect a face included in the image. . For example, the face detection means 21 first detects a black eye position candidate from the image, assuming that the point where the tone value of the target pixel is smaller than the tone values of surrounding pixels is a black eye candidate. Further, the face detection means 21 assumes that the detected combination of black eye positions is a face candidate. Then, the face detection means 21 uses the discriminator for discriminating the face and the image other than the face previously learned by the neural network or the like for the face candidate thus obtained, so that each face candidate is really The power of whether or not it is a face is determined. As a result of the determination, the face detection means 21 outputs the position of the face candidate determined to be a face as a place where the possibility that the face actually exists is high.

[0057] The above high-speed face detection processing is, for example, “Suzuki, Hosoi, Sakurai, Sato,“ Development of high-speed face detection processing using a ring filter ”, Proceedings of the 2003 IEICE General Conference, p. 251 ”.

[0058] The face number notifying means 22 determines whether or not the display screen of the information processing device is being viewed by another person other than the user based on the face detection result (the number of detected faces) by the face detecting means 21. A function to determine is provided. Hereinafter, the state in which the display screen of the information processing apparatus is being viewed by another person other than the user is also referred to as “a viewing state”. In this embodiment The face number notifying unit 22 determines whether or not the number of detected faces detected by the face detecting unit 21 is equal to or greater than a predetermined threshold (for example, a predetermined value n). When it is determined that the number of detected faces is less than the number of detected faces detected by the face detecting means 21, the face number notifying means 22 determines that the face is not being looked at by others. Further, when it is determined that the number of detected faces is greater than or equal to the detected number of faces, the face number notifying means 22 determines that the face is being looked at by another person. In the present embodiment, the predetermined value n is a preset value (for example, n = 2).

[0059] The warning generating means 23 notifies the user that the person is in the sighted state when the face number notifying means 22 determines that the person is in the sighted state as a result of the judgment. The warning output unit 3 is provided with a function for

[0060] The warning output unit 3 has a function of outputting a warning for notifying the user that the user is in the viewing state in accordance with an instruction from the warning generation means 23. The warning output unit 3 may be any means capable of sending some information to the user of the information processing device. Specifically, the warning output unit 3 may be an audio output device (not shown) such as a speaker or an image such as a monitor. This is realized by a display device (not shown). For example, when voice is used as a warning, a voice output device such as a speech power is used as the warning output unit 3. In this case, the warning output unit 3 outputs a voice or a warning sound to warn the user of the pretending status. For example, when an image is used as a warning, an image display device such as a monitor is used as the warning output unit 3. In this case, the warning output unit 3 warns the user of the preoccupation status by image or text information.

[0061] In the present embodiment, the warning output unit 3 is in a state of looking at the user, for example, by any one of image display, sound output, vibration of the information processing device, or light emission. Notify that it is. In addition, the warning output unit 3 notifies the user that the user is in the sighted state by using a combination of two or more of image display, sound output, information processing device vibration, or light emission. . Further, the warning output unit 3 uses all of the display of the image, the output of the sound, the vibration of the information processing device, and the light emission to prompt the user to be in a presiding state.

[0062] In the present embodiment, warning output unit 3 executes a resolution process for canceling the presidency state when it is determined that face number notifying means 22 is in the presidency state. To see Functions as a solution.

[0063] Next, the operation will be described. FIG. 3 is a flowchart illustrating an example of a process for determining whether or not a display screen is being viewed by another person other than the user and warning the user. First, the imaging unit 1 captures an image obtained by capturing the front of the image display device included in the information processing apparatus at a predetermined timing (step S11). For example, when the data processing apparatus 2 detects an instruction input by a user operation (for example, an instruction input for displaying various information on the display screen), the data processing apparatus 2 instructs the imaging unit 1 to capture an image. Then, the imaging unit 1 captures an image in front of the image display device according to the instruction of the data processing device 2.

[0064] Next, the face detection means 21 of the data processing device 2 performs face detection processing on the image captured by the imaging unit 1, and extracts all the faces included in the captured image (step S12). Further, the face number notifying unit 22 determines whether or not n (for example, n = 2) or more faces are included in the captured image based on the face detection result detected by the face detecting unit 21. (Step S13).

[0065] When the number-of-faces notification means 22 determines that n or more faces are included in the captured image (that is, when it is determined that the face is a presidency), the warning generation means 23 Creates warning information to warn the user that the display screen is being viewed. Further, the face number notifying means 22 causes the warning output unit 3 to output a warning to the information processing device user (user) based on the created warning information (step S14), and ends the process. For example, the warning generating means 23 causes the warning output unit 3 that is a speaker to output a warning using sound or warning sound. In addition, for example, the warning generation unit 23 causes the warning output unit 3 that is a monitor to output a warning by using an image or character information.

[0066] When the number-of-faces notification means 22 determines that n or more faces are not included (that is, when it is determined that the face is not a presiding look), the information processing apparatus outputs a warning to the user The process ends as it is.

As described above, according to the present embodiment, the information processing apparatus captures images of the front and front periphery of the image display apparatus attached to the information processing apparatus by the imaging unit 1. Then, the information processing device performs a presidency determination to determine whether or not the presidency state is in use using the number of faces present in the captured image. Therefore, even if the image display device attached to the information processing device is viewed from the same viewpoint as the user of the information processing device, multiple faces are detected in the captured image. Thus, the user can be notified of a warning that the person is in the presidency. Therefore, even when another person looks into the screen from the same viewpoint as the user, it is possible to prevent others from seeing the screen.

[0068] Further, according to the present embodiment, the imaging unit 1 is installed between the user and the image display device except that the imaging unit 1 is installed so as to photograph the front and front periphery of the image display device attached to the information processing device. There are no constraints (eg, special film sheets). As a result, the user can be notified of the pretense warning without changing the display characteristics of the image display device (that is, without degrading the image quality of the image display device). Therefore, it is possible to prevent others from seeing the screen without degrading the image quality of the display screen of the image display device attached to the information processing device.

[0069] Further, according to the present embodiment, face detection and look determination can be performed without a specific operation by the user of the information processing apparatus. For this reason, it is possible to make a prestigious decision regardless of the environment in which the information processing device is used. In addition, according to the present embodiment, it is possible to determine whether or not the person is in a presiding state without performing collation processing using face data registered in advance, thereby reducing the processing burden on the information processing apparatus. it can. Therefore, it is possible to prevent the screen from being viewed by a simple method without registering the face data or attribute data of the authorized user.

[0070] Embodiment 2.

Next, a second embodiment of the present invention will be described with reference to the drawings. FIG. 4 is a block diagram illustrating another configuration example of the information processing apparatus having the pretending to look prevention function. As shown in FIG. 4, in the present embodiment, the data processor 4 includes the display switching means 24 in place of the alarm generating means 23 of the data processor 2 shown in FIG. It is different from the form. Further, the information processing apparatus is different from the first embodiment in that the information processing apparatus includes an image output unit 5 instead of the warning output unit 3. Note that the functions of the imaging unit 1, the face detection means 21, and the face number notification means 22 are the same as those shown in the first embodiment.

[0071] Specifically, the image output unit 5 is realized by an image display device such as a monitor. The image output unit 5 has a function of displaying a screen for displaying various types of information in accordance with instructions from the data processing device 4 according to an operation of the information processing device user (user). [0072] The display switching means 24 predesignates an image to be displayed by the image output unit 5 when it is determined that the number of faces notifying means 22 is a presidency by another person. A function to switch to the selected image is provided. For example, the display switching unit 24 switches the screen displayed by the image output unit 5 to an image preset by the user. Further, for example, the display switching means 24 switches the screen displayed by the image output unit 5 to a black image.

For example, assume that the information processing apparatus is a mobile phone. In this case, when the image output unit 5 displays personal information such as an email or a telephone directory and determines that the face number notification unit 22 is viewed by another person, the display switching unit 24 outputs the image information. Switch the information displayed by section 5 to the standby screen of the mobile phone. By doing so, it is possible to prevent others other than the user from seeing personal information.

In the present embodiment, the display switching unit 24 performs a canceling process for canceling the presidency state when the face number notifying unit 22 determines that the presidency state is in effect. It functions as a means of resolving the preoccupation state to be executed.

Next, the operation will be described. FIG. 5 is a flowchart illustrating an example of processing for determining whether or not a display screen is being viewed by someone other than the user and switching the display screen of the information processing apparatus. In the present embodiment, the imaging unit 1 captures an image in accordance with the same processing as that in step S11 described in the first embodiment, and the face detection unit 21 includes the step S12 described in the first embodiment. A face is detected from the captured image according to the same processing. Then, the number-of-faces notification means 22 determines whether or not n or more faces are included in the photographed image, according to the same processing as Step S13 shown in the first embodiment.

In the first embodiment, when it is determined that the face number notification means 22 is in the state of looking, the warning output unit 3 simply outputs a warning, and the image display means attached to the information processing apparatus The displayed image was output (displayed) as it was. In the present embodiment, when it is determined that the face number notifying unit 22 is in the viewing state, the display switching unit 24 switches the image output (displayed) by the image output unit 5 to a predetermined image (step S24 ), The process ends. If it is determined that the data processing device 4 is not in the presidency state, the information processing device ends the processing without switching the image output by the image output unit 5. [0077] As described above, according to the present embodiment, when the information processing device detects the pretending look state, the information displayed by the image output unit 5 is not displayed (the image is switched). Control. Therefore, even if the user of the information processing device does not perform a specific operation in order to eliminate the pretending by others, the pretending by others can be prevented.

[0078] Embodiment 3.

Next, a third embodiment of the present invention will be described with reference to the drawings. FIG. 6 is a block diagram showing still another configuration example of the information processing apparatus provided with the pretending and seeing prevention function. As shown in FIG. 6, in the present embodiment, the data processing device 6 includes the browsing person recording means 25 in place of the alarm generation means 23 of the data processing device 2 shown in FIG. It differs from the form of implementation. Further, the information processing apparatus is different from the first embodiment in that the information processing apparatus includes a storage device 7 instead of the warning output unit 3. Note that the functions of the imaging unit 1, the face detecting means 21, and the face number notifying means 22 are the same as those shown in the first embodiment.

[0079] Specifically, the storage device 7 is a memory, a hard disk device, or the like. As shown in FIG. 6, the storage device 7 includes a browsing person storage unit 71 for recording a person who has seen the image display device attached to the information processing apparatus. In the present embodiment, the storage device 7 stores an image showing a person who is looking at the screen displayed by the information processing device in accordance with the instruction of the browsing person recording means 25.

[0080] When the number-of-faces notifying means 22 determines that the face-notifying means 22 is in the presidency state, the browsing person recording means 25 determines the image to be determined when it is determined to be the presidency state (captured by the imaging unit 1) (Image) is stored in the browsing human storage unit 71.

[0081] Since the browsing person recording means 25 stores the image taken by the imaging unit 1 as it is in the browsing person storage unit 71, the browsing person recording unit 25 extracts only the portion where the face is reflected from the image, and extracts the extracted partial image. You may make it memorize | store in the browsing person memory | storage part 71. FIG. In this case, the browsing person recording unit 25 extracts an area including the face detected by the face detection unit 21 from the image captured by the imaging unit 1. Then, the browsing person recording unit 25 stores the extracted image of the area in the browsing person storage unit 71.

[0082] Further, in the present embodiment, the browsing person recording means 25 executes a canceling process for eliminating the pretending look state when it is determined that the face number notifying means 22 is in the look state. Do It functions as a modest state cancellation means.

Next, the operation will be described. FIG. 7 is a flowchart showing an example of a process for determining whether or not a display screen is viewed by another person other than the user and recording the person who looks at the display. . In the present embodiment, the imaging unit 1 captures an image according to the same processing as in step S11 shown in the first embodiment, and the face detection means 21 performs step S12 shown in the first embodiment. The face is detected from the photographed image according to the same processing as. Then, the face number notifying means 22 determines whether or not n or more faces are included in the photographed image according to the same processing as that in step S13 shown in the first embodiment.

[0084] In the first embodiment, when it is determined that the number-of-faces notification means 22 is in the sighted state, the warning output unit 3 only outputs a warning, I wasn't going to save the image. In the present embodiment, when the number-of-faces notification means 22 determines that it is in the presidency state, the browsing person recording means 25 displays the image to be determined when it is determined that it is in the viewing state. The data is stored in the storage unit 71 (step S34), and the process ends. That is, the browsing person recording means 25 stores the captured image captured by the imaging unit 1 in step S 11 in the browsing person storage unit 71. If the data processing device 6 determines that it is not in the presidency state, the information processing device ends the processing without storing the photographed image in the browsing person storage unit 71.

As described above, according to the present embodiment, when the information processing apparatus detects the presidency state, the information processing apparatus records a photographed image when it is determined to be the presidency state. Therefore, a user or administrator of the information processing apparatus can confirm the face of the person who performed the pretending action afterwards. Therefore, in the event of cheating by mischief, it is possible to easily identify the person who performed the cheating. Further, according to the present embodiment, it is possible to expect the effect of suppressing the look-ahead action by recording the look-ahead state image, and it is possible to prevent the look-ahead action in advance.

[0086] Embodiment 4.

Next, a fourth embodiment of the present invention will be described with reference to the drawings. FIG. 8 is a block diagram showing still another configuration example of the information processing apparatus having the pretending to look-in prevention function. As shown in FIG. 8, in the present embodiment, the data processing device 8 is connected to the data processing device shown in FIG. The second embodiment is different from the first embodiment in that the display switching means 24 is included in addition to the components of the device 2. Further, the information processing apparatus is different from the first embodiment in that it includes an image output unit 5 in addition to the components shown in FIG. The functions of the imaging unit 1, the face detection unit 21, the face number notification unit 22, the warning generation unit 23, and the warning output unit 3 are the same as those described in the first embodiment.

[0087] Similar to the image output unit 5 described in the second embodiment, the image output unit 5 is specifically realized by an image display device such as a monitor. Further, the image output unit 5 has a function of displaying a screen for displaying various types of information in accordance with an instruction from the data processing device 8 by an operation of the information processing device user (user).

[0088] The display switching means 24 is the same as the display switching means 24 shown in the second embodiment. If it is determined that there is, the image output unit 5 has a function of switching the image to be displayed to a predesignated image.

Next, the operation will be described. FIG. 9 is a flowchart showing an example of a process for determining whether or not a display screen is being viewed by someone other than the user and for causing the user to switch the display screen of the warning blueprint processing device. In the present embodiment, the imaging unit 1 captures an image in accordance with the same processing as in step S11 described in the first embodiment, and the face detection unit 21 performs step S12 illustrated in the first embodiment. The face is detected from the captured image according to the same process as above. Then, the warning generation unit 23 determines whether or not n or more faces are included in the photographed image, according to the same process as Step S13 shown in the first embodiment.

[0090] In the first embodiment, when the face number notification means 22 determines that the face is in a sighted state, the warning output unit 3 simply outputs a warning, and the image display means attached to the information processing apparatus The displayed image was output (displayed) as it was. Further, in the second embodiment, when the face number notification means 22 determines that it is in the presidency state, the image output unit 5 simply switches the image output (displayed) to a preset image, The user was not explicitly informed of the pretending status.

In the present embodiment, when the face number notifying means 22 determines that it is in the presidency, a warning is issued. The raw means 23 warns the user through the warning output unit 3 that the user is in a pretentious state (step S44). Note that the method in which the warning output unit 3 warns the user of the visual status is the same as the method shown in the first embodiment (for example, notification using sound, warning sound, image, character information, etc.). It is.

Furthermore, the display switching unit 24 switches the image output (displayed) by the image output unit 5 to a predetermined image (step S45), and ends the process. Note that the order of the processing of step S44 executed by the warning generation means 23 and the processing of step S45 executed by the display switching means 24 is in no particular order. For example, if it is determined that the face number notifying unit 22 is in the presidency state, the display switching unit 24 executes a process of causing the image output unit 5 to switch the display image, and then the warning generating unit 23 sets the warning output unit. The process of warning through 3 may be executed. Further, when it is determined that the face number notifying unit 22 is in the presidency state, the warning generating unit 23 and the display switching unit 24 may execute processing in parallel.

[0093] As described above, according to the present embodiment, it is possible to prevent a pretending act by others as in the first embodiment and the second embodiment. For example, as in the first embodiment, it is possible to notify the user of a warning that the person is in the presidency, so even if another person sees the screen from almost the same viewpoint as the user, You can prevent others from seeing the screen. In addition, it is possible to prevent others from seeing the screen without degrading the quality of the display screen of the image display device attached to the information processing device. Furthermore, it is possible to prevent the screen from being viewed with a simple method without registering the face data or attribute data of a regular user.

[0094] Also, according to the present embodiment, since the display image is controlled to be switched when the pretending look state is detected, the user of the information processing device can perform a specific look to eliminate the look by others. Even if no operation is performed, it is possible to prevent other people from seeing you.

[0095] Embodiment 5.

Next, a fifth embodiment of the present invention will be described with reference to the drawings. FIG. 10 is a block diagram showing still another configuration example of the information processing apparatus provided with the pretending prevention function. As shown in FIG. 10, in the present embodiment, the data processing device 9 replaces the face number notification means 22 of the data processing device 2 shown in FIG. 1 with gaze estimation means 26 and screen gaze determination means 27. It is different from the first embodiment in that it is included. Further, in the present embodiment, the function of the warning generating means 23A of the data processing device 9 is different from the function of the warning generating means 23 shown in the first embodiment. Note that the functions of the imaging unit 1, the face detection means 21, and the warning output unit 3 are the same as those shown in the first embodiment.

The line-of-sight estimation unit 26 has a function of estimating the line-of-sight direction of the face included in the captured image based on the position of the face eye in the image detected by the face detection unit 21. In the present embodiment, the line-of-sight estimation unit 26 estimates the direction of the line of sight of the eyes included in the face detected by the face detection unit 21.

The screen gaze judging means 27 is based on the position of the face in the image detected by the face detecting means 21 and the estimation result of the eye gaze estimating means 26 (gaze direction corresponding to the face in the image). A function is provided that determines whether or not the face person detected by the detection means 21 is in a state of gazing at the display screen of the information processing apparatus (also referred to as gaze determination). Further, the screen gaze determination means 27 has a function of determining whether or not there are n or more persons watching the screen. Further, the screen gaze determination means 27 determines that it is in the presidency when it is determined that the number of human beings determined to gaze at the screen displayed by the information processing apparatus is not less than several.

[0098] In the present embodiment, the warning generating means 23A, when the screen gaze determining means 27 determines that there are n or more persons watching the screen, notifies the warning output unit 3 to the user. A function is provided to output a warning to notify that the user is in the viewing state.

Next, the operation will be described. FIG. 11 is a flowchart showing another example of a process for determining whether or not a display screen is viewed by another person other than the user and warning the user. In the present embodiment, the imaging unit 1 captures an image in accordance with the same processing as in step S11 described in the first embodiment, and the face detection means 21 performs step S12 described in the first embodiment. In accordance with the same processing as in step 1, the photographed image force also detects the face.

[0100] In the first embodiment, after the face detection means 21 detects the face in step S12, the face number notification means 22 counts the number of detected faces, and n or more faces are detected. In the case, it was judged to be in charge. In the present embodiment, after the face detection means 22 detects a face in the captured image in step S12, the line-of-sight estimation means 26 estimates the line-of-sight direction of each detected face (step S 53). ). [0101] FIG. 12 is an explanatory diagram showing an example of an estimation method for estimating the gaze direction of a face in an image. In step S53, the line-of-sight estimation means 26 detects, for example, the position of the black eye included in the detected eye position, and the position of the corner of the eye or the head of the eye based on the edge information as shown in FIG. . Then, the line-of-sight estimation means 26 estimates the line-of-sight direction of the face person in the image based on the positions of the left and right black eyes and the distance between the corners of the eyes and the eyes.

[0102] For example, in the example shown in FIG. 12, the information processing apparatus stores in advance a correspondence table that describes the correspondence between the angles and the distances a, b, c, and d between the eye portions in a storage device or the like. ing. In the example shown in Fig. 12, a is the distance between the left eye corner and the black eye, b is the distance between the left eye eye and the eye, c is the distance between the right eye and the black eye, and d is the right eye. The distance between the black eyes and the corners of the eyes. For example, when the detected distance b between the left eye's black eye and the eye's eye is the same as the distance between the right eye's eye and the black eye (b = c), the gaze estimation means 26 indicates that the face person in the image is facing the front. It is determined that. If the detected distance b between the left eye's black eye and the eye's eye is greater than the distance c between the right eye's eye and the black eye (b> c), the line-of-sight estimation means 26 looks at the right side of the face person in the image. It is determined that

[0103] The line-of-sight estimation means 26 learns the correspondence between the distances a, b, c, d between each part of the eye and the line-of-sight direction using a learning model such as a neural network, and uses the learning result as a result. You can estimate the gaze direction based on this.

[0104] After estimating the gaze direction of each face detected by the gaze estimation unit 26, the screen gaze determination unit 27 gazes the screen based on the estimation result of the gaze estimation unit 26. It is determined whether or not (step S54).

FIG. 13 is an explanatory diagram showing an example of a determination method for determining whether or not the face in the image is gazing at the display screen of the information processing apparatus. As shown in FIG. 13, the screen gaze determination means 27 is in a screen gaze state in which a face person in the image gazes at the screen based on the position where the face is detected and the gaze direction of the face. Determine whether or not. In the example shown in FIG. 13, when it is estimated that the face person detected in the center of the image is looking in front, the screen gaze determination unit 27 determines that the screen is in the screen gaze state. Further, when it is estimated that the face detected in the center of the image is looking in a direction other than the front, the screen gaze determination unit 27 determines that the screen is not in the screen gaze state. Similarly, if the face person detected on the left side of the image looks at the right side When estimated, the screen gaze determination means 27 determines that the screen is in a gaze state. Further, when it is estimated that the face person detected on the left side of the image is looking in a direction other than the right side, the screen gaze determination unit 27 determines that the screen is not in the screen gaze state.

[0106] Further, the screen gaze determination means 27 determines whether or not the number of faces of the person who has determined that the screen is being watched is n (n≥1) or more (step S55). If it is determined that the number of faces of the person determined to be in a gaze state is less than several, the information processing apparatus ends the process. When it is determined that the power of the face of the person determined to be in the gaze state is ¾ or more, the warning generation means 23 outputs a signal for warning the gaze state to the warning output unit 3 to the user. A warning is output to the warning output unit 3 (step S56), and the process is terminated.

[0107] In the present embodiment, a case will be described in which a warning is output to the user when the information processing device detects a viewing state. However, the information processing device detects a presidency state. In this case, another process may be executed. For example, the information processing apparatus includes a display switching unit 24 and an image output unit 5 in addition to the components shown in FIG. May be switched. Further, for example, the information processing apparatus includes a browsing person recording unit 25 and a browsing person storage unit 71 in addition to the components shown in FIG. It is also possible to store the photographed image in which the person is photographed in the viewing person storage section 71.

[0108] As described above, according to the present embodiment, the information processing apparatus detects a point at which a person whose face has been detected merely by detecting a face in the image is gazing, thereby achieving higher accuracy. It is possible to make a scrutiny determination. Therefore, compared with the information processing apparatus shown in the first embodiment to the fourth embodiment, it is possible to more accurately prevent the act of pretending by others.

[0109] For example, when determining whether or not the person is in the preoccupation state based only on the number of faces in the photographed image, the face of a bona fide person who is not willing to see is accidentally captured. Even if it has occurred, there is a risk that it is determined to be in sight. According to the present embodiment, by estimating the direction of the line of sight of the face in the image, it is determined whether or not the face human being in the image is gazing at the display screen of the information processing device. . Based on the number of faces in the image that are gazing at the display screen, a determination is made as to whether or not the person is looking To do. Therefore, it is possible to determine whether or not the person is in a presiding state with higher accuracy than in the case where the determination is based only on the number of faces in the image.

Example

[0110] Next, the operation of the information processing apparatus having the pretending to see-through function will be described using a specific embodiment. FIG. 14 is an explanatory view showing a specific example of an information processing apparatus having a pretending to look prevention function. Note that the information processing apparatus shown in FIG. 14 corresponds to the information processing apparatus shown in the fourth embodiment. Further, as shown in FIG. 14, in the present embodiment, a case where the information processing apparatus is a foldable mobile phone will be described as an example.

As shown in FIG. 14, the mobile phone includes, as the imaging unit 1, a camera device (for example, a CCD camera or the like) for taking an image of a user looking at a liquid crystal monitor. Further, as shown in FIG. 14, the camera device is attached to a surface that becomes the inner side when the cellular phone is folded. The camera device is attached around the liquid crystal monitor.

[0112] In the mobile phone, the data processing device 8 is a CPU of the mobile phone that operates according to the program. In the mobile phone, the warning output unit 3 is realized by a speaker, a vibration unit, or the like. In the mobile phone, the image output unit 5 is a liquid crystal monitor.

The mobile phone has a central processing unit (CPU) that functions as face detection means 21, face number notification means 22, warning generation means 23, and display switching means 24.

[0114] In this example, a case will be described in which a mobile phone user displays important information that the user does not want to be seen by others, such as a phone book or mail. In this case, the mobile phone displays these important information, and automatically activates the camera device and the face detection means 21 to display images including the user viewing the liquid crystal monitor of the mobile phone and the surroundings of the user. Use to shoot. The mobile phone performs face detection processing on the photographed image. For example, when the number of detected faces is two or more, the mobile phone determines that a look is occurring. The mobile phone alerts the user by vibration using the vibrator function.

[0115] In addition, at the same time that the mobile phone issues a warning by vibration, the display screen of the liquid crystal monitor unit is switched from the important information screen to a screen that displays the face detection result. For example, as shown in Fig. 15, the mobile phone detects the user's own face and the faces of other people who are watching the screen. Switch to the screen that displays the detected results. In this way, by switching the important information to another image, it is possible to eliminate the presidency by others other than the user and notify the user that the presidency is in the presidency. Industrial applicability

The present invention can be applied to an information processing apparatus that displays important information such as a mobile phone and a personal computer, and can be applied to an application for preventing or preventing viewing by others other than the user. For example, the present invention can be applied to the use of an information search apparatus that searches for a synonym from a database and the use of a program for realizing the information search apparatus using a computer. In addition, for example, the present invention is used for a personal computer or a word processor dedicated machine equipped with a kana-kanji conversion device equipped with a function for searching for synonyms, or for a program for realizing a powerful kanji conversion function using a computer. Is also applicable.

Claims

The scope of the claims

[1] An imaging unit that captures an area for a person to view the screen;

Face detection means for detecting a face included in an image captured by the imaging unit;

Face number notifying means for notifying the number of faces detected by the face detecting means;

An information processing apparatus comprising:

[2] An imaging unit that captures an area for a person to view the screen;

A line-of-sight estimation means for estimating the direction of the line of sight of eyes contained in the face detected by the face detection means;

Screen gaze determining means for determining whether or not the person of the face detected by the face detecting means is gazing at the screen based on the estimation result of the line-of-sight estimation means;

Face number notifying means for notifying the number of faces gazing at the screen based on the determination result of the screen gaze determining means;

An information processing apparatus comprising:

[3] The information processing apparatus according to claim 1 or 2, wherein the face number notifying means notifies by at least one of display of an image, output of sound, vibration of the information processing apparatus, or light emission.

4. The information processing apparatus according to claim 1, wherein the face number notification means is an image display switching means for switching information displayed by the image display apparatus.

5. The information processing apparatus according to claim 4, wherein the image display switching means switches a screen displayed by the image display apparatus to a preset image.

6. The information processing apparatus according to claim 5, wherein the preset image is an image set by a user.

[7] A browsing person image storage unit that stores an image including a person browsing the screen, wherein the number-of-faces notification unit stores an image captured by the imaging unit in the browsing person image storage unit. The information processing apparatus according to claim 1, further comprising: means.

[8] The browsing person recording unit is configured so that the face detection unit uses the image captured by the imaging unit. 8. The information processing apparatus according to claim 7, wherein an area including the detected face is extracted, and an image of the extracted area is stored in the browsing person image storage unit.

9. The number-of-faces notification means determines whether or not to perform notification regarding the number of faces based on the number of faces detected by the face detection means. either

The information processing apparatus according to item 1.

[10] An image display device for displaying information is provided,

The imaging unit according to claim 1 to ₉ in Les characterized in that it is installed around the image display apparatus, an information processing apparatus according to item 1 Zureka.

[11] An information processing apparatus including an image capturing unit that captures an area for a person to browse a screen detects a face included in an image captured by the image capturing unit, and detects by the face detecting step An information processing method comprising: executing a face number notification step of notifying the number of faces.

[12] An information processing apparatus including an image capturing unit that captures an area for a person to browse a screen detects a face included in an image captured by the image capturing unit, and detects by the face detecting step A line-of-sight estimation step for estimating the direction of the line of sight of the eyes contained in the face;

Based on the estimation result of the gaze estimation step, based on the determination result of the screen gaze determination step and the screen gaze determination step for determining whether or not the face person detected by the face detection step is gazing at the screen. An information processing method comprising: performing a face number notifying step of notifying the number of faces while paying attention to the screen.

[13] The information processing apparatus displays an image and outputs a sound in the face number notification step.

13. The information processing method according to claim 11 or 12, wherein notification is made by at least one of vibration of the information processing device or light emission.

14. The information processing method according to claim 11, wherein the information processing device executes an image display switching step of switching information displayed by the image display device in the face number notification step. .

15. The information processing device according to claim 14, wherein, in the image display switching step, the information processing device switches a screen displayed by the image display device to a preset image. Information processing method.

16. The information processing method according to claim 15, wherein the preset image is an image set by a user.

[17] Further, in the information processing device including a browsing person image storage unit that stores an image including a person browsing the screen, in the face number notification step, the image captured by the imaging unit is used as the browsing person image. The information processing method according to any one of claims 11 to 16, wherein a browsing person recording step to be stored in the storage unit is executed.

[18] In the browsing person recording step, the information processing apparatus extracts an area including the face detected by the face detection step from an image photographed by the imaging unit, and the image of the extracted area is browsed 8. The information processing method according to claim 7, wherein the information is stored in a person image storage unit.

[19] The information processing apparatus, in the face number notification step, determines whether or not to perform notification regarding the number of faces based on the number of faces detected in the face detection step. The information processing method according to any one of items 11 to 18.

[20] A program that causes a computer to function as the information processing apparatus according to any one of claims 1 to 9.