JP2003264877A - Mobile terminal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a mobile terminal capable of more effectively preventing displayed information from being viewed by other person. <P>SOLUTION: When a generating menu for generating a sentence to be sent as electronic mail is displayed on an LCD (liquid crystal display) 35, a CCD (charge coupled device) camera 33 photographs a prescribed range which a front of the mobile phone 1 faces. Iris data denoting a pattern of an iris of a person included in a photographed image are extracted and discriminated whether or not the iris data are identical to iris data of a user of the mobile telephone 1 registered in advance in a memory card 43. When the extracted iris data are not identical to the iris data registered in advance, the mobile telephone 1 displays a warning display on the LCD 35 for warning the risk of other person than the user of the mobile telephone 1 peeping into the display of the LCD 35. This invention is applicable to a mobile terminal such as a mobile telephone. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to a mobile terminal,
In particular, the present invention relates to a mobile terminal that can more effectively prevent others from viewing information displayed on the mobile terminal.

[0002]

2. Description of the Related Art In recent years, it has become common to send and receive e-mail using mobile terminals such as mobile phones. Users can easily create texts to be sent as e-mails without being restricted by location or time.
It can be sent to the other terminal.

[0003]

However, for example, in a place where people are crowded, such as at a platform of a station, while writing a sentence to be sent as an email, or when writing a sentence of a received email, While reading,
There is a problem that the contents displayed on the mobile phone may be read by a person behind himself.

In the future, for example, important information about work and important personal information will be sent to mobile phones and PDAs (Personal D
It is expected that confirmation will be performed on a daily basis using mobile terminals such as digital assistants), and in that case, this problem may become more serious.

The present invention has been made in view of such circumstances, and it is possible to more effectively prevent others from seeing information displayed on a mobile terminal such as a mobile phone or PDA. It is a thing.

[0006]

A portable terminal of the present invention detects a characteristic of a person existing in the vicinity and outputs characteristic information corresponding to the characteristic, and based on the characteristic information output by the detecting means. A determining unit that determines whether or not the display of the mobile terminal is viewed by a person different from the user of the mobile terminal; and the determination unit determines that the display is viewed by a person different from the user of the mobile terminal. In this case, a warning means for outputting a warning is provided.

[0007] Further comprising a photographing means for photographing the subject,
The detection means may detect the feature based on the image captured by the image capturing means.

[0008] The mobile terminal further comprises storage means for storing the first characteristic information of the user, and the detecting means detects the characteristic of the person photographed by the photographing means, and the corresponding second characteristic is detected.
Of the mobile terminal based on whether the first feature information stored in the storage means and the second feature information detected by the detection means are the same. It is possible to determine whether or not the display is viewed by a person different from the user.

The first characteristic information is information on the iris pattern of the user of the mobile terminal, and the second characteristic information is
The information can be information about an iris pattern of a person photographed as a subject.

As a feature, the detecting means detects the size of the eyes of a person on the image photographed by the photographing means, and the judging means causes the detecting means to detect the eyes of a predetermined size or more for a plurality of persons. It is possible to determine whether or not the display is visually observed by a person different from the user of the mobile terminal based on whether or not the display is detected.

As a characteristic, the detecting means detects the distance to the eyes of the person based on the image photographed by the photographing means, and the judging means causes the detecting means to detect the eyes of the plurality of persons within a predetermined distance. It is possible to determine whether or not the display is visually observed by a person different from the user of the mobile terminal based on whether or not the presence is detected.

As a characteristic, the detecting means detects the direction of the eyes of the person seen from the mobile terminal based on the image photographed by the photographing means, and the judging means causes the detecting means to detect the area of the user of the mobile terminal. It is possible to determine whether or not the display is visually observed by a person different from the user of the mobile terminal, based on whether or not it is detected that a person's eyes are present in the direction of a predetermined adjacent area. it can.

The warning means may display a screen of a predetermined color as a warning.

The warning means may further include a storage means for instructing to interrupt the display as a warning and for saving the content of the display when the display is interrupted in response to the instruction from the warning means.

The portable terminal is formed to be foldable, and the warning means may instruct folding of the portable terminal as a warning for instructing interruption of display.

In the portable terminal of the present invention, the characteristics of a person existing in the vicinity are detected, and the characteristic information corresponding to them is output. Further, based on the output characteristic information, it is determined whether or not the display of the mobile terminal is viewed by a person different from the user of the mobile terminal, and the display is viewed by a person different from the user of the mobile terminal. If it is determined that, a warning is output.

[0017]

FIG. 1 is a perspective view showing an example of the external configuration of a mobile phone 1 to which the present invention is applied.

As shown in FIG. 1, the mobile telephone 1 is
Basically, it is composed of a display portion 22 and a main body 23, and is formed so as to be foldable by a hinge portion 21 at the center.

An antenna 31 is provided on the left side of the upper surface of the display unit 22.
Are provided, and radio waves are transmitted and received to and from the base station via the antenna 31.

At the upper end of the display unit 22, there is provided a camera unit 32 which is rotatable about a horizontal direction (horizontal direction of the housing), and a CCD (Charge C) provided on the camera unit 32.
The captured image is taken by the camera 33. When the camera unit 32 is rotated by approximately 180 degrees by the user, as shown in FIG. 2, a speaker provided near the center of the rear side of the camera unit 32 (the rear side with the surface shown in FIG. 1 as the front). 34 is located on the front side.

Further, the CCD camera 33 is provided on the member 32A of the camera unit 32, and by driving the member 32A, a line connecting the center of the upper surface and the center of the lower surface of the member 32A is used as an axis to form a predetermined angle in the left-right direction. It is supposed to be rotatable within the range. For example, while the text to be transmitted as an electronic mail is being created by the user, the CCD camera 33 captures an image of the space in front of the mobile phone 1 within a predetermined range.

As will be described in detail later, based on the image taken by the CCD camera 33, in addition to the user of the mobile phone 1 (hereinafter, simply referred to as a user), an LCD (Liquid Cry) is displayed.
stal Display) 35 It is determined whether or not there is a person looking at the display (for example, the text being created), and when it is determined that there is a person looking at the display of LCD 35, the screen is seen by another person. A predetermined warning screen is displayed on the LCD 35 to warn the user that it is being (visually viewed). Specifically, it is determined whether the person photographed by the CCD camera 33 is a user or another person by comparing the iris data (iris code) extracted from the photographed image,
The determination is made based on the size of the eyes, the distance to the person, the direction in which the eyes are detected, or the like.

An LCD 35 is provided substantially in the center of the display section 22. On the LCD 35, the reception status of electric waves, the remaining battery level, the name and telephone number registered as a telephone directory, the outgoing call history, and the contents of e-mail are displayed according to the instruction from the user.

On the other hand, the main body 23 is provided with operation keys 36 such as numeral keys "0" to "9", a calling key, a power key, and the like, and a memo button 37 and a microphone 38 are provided below the operation keys 36. It is provided. A jog dial 39, which is rotatable about a horizontal direction (the left-right direction of the housing) as an axis, is provided on the upper front surface of the operation key 36 of the main body 23 in a state of slightly protruding from the surface of the main body 23.
The mobile phone 1 executes various processes such as a scrolling operation of the electronic mail displayed on the LCD 35 in response to a rotating operation on the jog dial 39. A left direction button 40 and a right direction button 41 are provided on the left and right of the jog dial 39, respectively.

A memory card 43 is provided on the left side of the main body 23.
There is provided a memory card slot 42 for mounting a memory card. The memory card 43 has a CCD as well as various data.
The iris data of the user and the like extracted from the image captured by the camera 33 are stored. The iris data stored in the memory card 43 is used, for example, to identify whether the person photographed by the CCD camera 33 is a user.

FIG. 3 is a block diagram showing an example of the internal structure of the mobile phone 1.

The CPU (Central Processing Unit) 51 is an RO
The control program stored in the M (Read Only Memory) 52 is expanded in the RAM (Random Access Memory) 53, and the entire operation of the mobile phone 1 is controlled via the bus 57.

For example, the CPU 51, when instructed by the user to create an electronic mail, the display control unit 55.
To display a text creation screen on the LCD 35 and control the camera control unit 56 to photograph the surroundings at a predetermined cycle. Further, the CPU 51 determines whether or not there is a person other than the user looking at the display of the LCD 35 on the main control unit 71 (see FIG. 4) of the camera control unit 56 based on the image captured by the CCD camera 33. Then, when it is determined that there is a person other than the user who is viewing the display on the LCD 35, a predetermined warning screen is displayed on the LCD to warn the user.
35 (display).

The communication controller 54 transmits and receives radio waves to and from the base station via the antenna 31, and, for example, in the voice call mode, RF (Radio Frequency) received by the antenna 31.
The quency) signal is amplified and subjected to predetermined processing such as frequency conversion processing, analog-digital conversion processing, and spectrum despreading processing, and the obtained audio data is output to the audio codec 59. In addition, the communication control unit 54 uses the audio codec 5
When the audio data is supplied from 9, the audio data obtained is subjected to predetermined processing such as digital-analog conversion processing, frequency conversion processing, and spread spectrum processing, and the obtained audio signal is transmitted from the antenna 31.

The display control unit 55 controls the display of the LCD 35 and creates a text creation screen to be sent as an electronic mail or a CCD.
The image and the like taken by the camera 33 are displayed on the LCD 35.

The camera control unit 56 controls the driving of the CCD camera 33, and also performs various kinds of processing on the image taken by the CCD camera 33 and obtained. For example,
When registering the iris data of the user, the camera control unit 56 acquires the iris data from the image captured by the CCD camera 33 and outputs it to the memory card interface 60 via the bus 57. The output iris data is stored in the memory card 43 by the memory card interface 60.

The folding detection unit 58 is the hinge unit 2 of FIG.
1 is provided in the vicinity of 1 to monitor the open / closed state of the mobile phone 1, and when it is detected that the mobile phone 1 is folded from the opened state or when it is opened from the folded state, the CPU 51 To notify. As will be described later, for example, when the mobile phone 1 is folded during the creation of a sentence to be sent as an electronic mail, the folding detection unit 58 notifies the CPU 51 that the text has been folded, and in response to the notification, it is being created. Is saved.

The voice codec 59 converts the voice data supplied from the communication controller 54 into a voice signal and outputs the corresponding voice signal from the speaker 34. The voice codec 59 also converts the voice of the user collected by the microphone 38 into voice data, and outputs the voice data to the communication control unit 54.

The memory card interface 60 controls the reading and writing of data with respect to the memory card 43 mounted in the memory card slot 42, and provides the data stored in the memory card 43 as appropriate in response to a request from the CPU 51. .

A drive 61 is connected to the bus 57 as needed.
Is required, and a magnetic disk 62, an optical disk 63, a magneto-optical disk 64, or the like is mounted as appropriate, and a computer program read from it or a computer program read from the memory card 43 by the memory card interface 60 is required. Is installed on the mobile phone 1. Although illustration is omitted, the bus 57 also receives an instruction from the user using the jog dial 39, an instruction from the user using the memo button 37, and the like.
Is output to the CPU 51 via.

FIG. 4 is a block diagram showing a configuration example of the camera control unit 56 of FIG.

The main control unit 71 controls the angle setting unit 72, the photographing control unit 73, and the image processing unit 74 based on an instruction from the CPU 51 of FIG. 3, and, for example, data acquired by the photographing control unit 73. Is supplied, the supplied data is output to the CPU 51 and the like via the bus 57.

The angle setting unit 72, for example, when the text creation screen or the received electronic mail is displayed on the LCD 35, the angle setting unit 72 controls the CCD based on the control by the main control unit 71.
The shooting range of the camera 33 is controlled within the range shown in FIGS. 5 and 6.

As shown in FIG. 5, the angle setting section 72
Indicates the CC camera 33 so that, for example, a range of 45 degrees to the left and right can be photographed centering on the front of the CCD camera 33 (dashed line in the figure).
The D camera 33 (member 32A) is driven. For example, the shooting range is sequentially switched from the left side to the right side in FIG. Note that, in FIG. 5, the photographing range of the mobile phone 1 is shown from slightly above the front of the mobile phone 1.

Further, as shown in FIG. 6, the angle setting section 72 has an upper side of 30 degrees and a lower side of 15 degrees with respect to the horizontal.
The camera unit 32 (CCD camera 33) is driven so that the range of degrees can be photographed. For example, the shooting range is sequentially switched from the upper side to the lower side of FIG.

Returning to the explanation of FIG. 4, the photographing control section 73
When the main controller 71 controls shooting by the D camera 33 and the main controller 71 instructs shooting, the image captured by the CCD camera 33 is output to the image processor 74 at that moment.

The image processing unit 74 extracts, for example, image data of the eyes of the subject and iris data from the image supplied from the photographing control unit 74, and outputs the extracted information to the main control unit 71.

FIG. 7 is a block diagram showing a detailed configuration example of the image processing section 74 of FIG.

The image acquisition unit 81 acquires the image captured by the photographing control unit 73 and outputs it to the face area detection unit 82.

The face area detecting section 82 detects a person's face area based on a plurality of images acquired by the image acquiring section 81, for example, from image shifts due to movement of a subject (person), or A face area of a person is detected based on a template prepared in advance, and the image data of the detected face area is output to the face part detection unit 83.

The face part detecting section 83 extracts the feature amount of each part of the face from the image data of the face area supplied from the face area detecting section 82, and the eyes, nose,
The positions of face parts such as mouth and ears are detected. The image data of the face parts detected by the face part detection unit 83 is output to the eye image acquisition unit 84.

The eye image acquisition unit 84 acquires only the image data of the eyes of the person based on the image data of the face parts detected by the face part detection unit 83, and a person other than the user can use the LCD.
When the main control unit 71 instructs to determine whether or not the display 35 is viewed based on the iris data (to identify the person), the image data of the detected eyes of the person is extracted as iris data. Output to the unit 85. In addition, when it is instructed to determine whether or not a person other than the user is looking at the display of the LCD 35, for example, based on the distance from the mobile phone 1 (CCD camera 33) to the eyes of the person,
Alternatively, when it is instructed to make a determination based on the size of the person's eyes on the captured image, the eye image acquisition unit 84 outputs the eye image data to the main control unit 71.

The iris data extracting section 85 includes an eye image acquiring section 8
The iris data of the photographed person is extracted on the basis of the image data of the eyes supplied from No. 4. The iris appears radially from the outside of the pupil of the human eye to the boundary between the black and white eyes, with the pupil as the center, and each person has a unique pattern. Specifically, the iris data extraction unit 85 uses, for example, 256 bytes from data indicating the shading of the iris pattern for each of a plurality of regions divided in the radial direction and the circumferential direction in polar coordinates with the center of the eye as the origin. Iris data (iris code) for one eye is extracted. The iris data extracted by the iris data extraction unit 85 is output to the main control unit 71. For example, the main control unit 71
Thus, it is determined (checked) whether the extracted iris data is the same as the iris data registered in the memory card 43 by the user in advance.

Next, the operation of the mobile phone 1 having the above configuration will be described.

First, the registration process of the mobile phone 1 for registering the iris data of the user will be described with reference to the flowchart of FIG.

When the user performs a predetermined operation and gives an instruction to register the iris data, the CPU 51 controls the camera control unit 56 in step S1 to perform photographing. That is, the main controller 7 of the camera controller 56
1 is based on an instruction from the CPU 51, and the photographing control unit 73
The image processing unit 74 is caused to output the image captured by the CCD camera 33.

When the image data is supplied from the photographing control unit 73, the image acquisition unit 81 of the image processing unit 74 acquires it in step S2. The image data acquired by the image acquisition unit 81 is output to the face area detection unit 82.

In step S3, the face area detecting unit 82
For example, based on the plurality of images acquired by the image acquisition unit 81, the area of the face of the person is detected from the image shift due to the movement of the person, and the image data of the detected face area is detected by the face component detection unit 83. Output to.

In step S4, the face part detecting section 83 determines the positions of the face parts such as the eyes, nose, mouth, and ears of the person based on the image data of the face area supplied from the face area detecting section 82. Detect and acquire image data of each part.
The image data of each face part acquired by the face part detection unit 83, its position information, and the like are output to the eye image acquisition unit 84.

In step S5, the eye image acquisition unit 84
Acquires the image data of the eyes of the person based on the image data of the face parts supplied from the face part detection unit 83,
It is output to the iris data extraction unit 85.

In step S6, as described above, the iris data extraction unit 85 indicates the shading of the iris pattern for each of a plurality of regions divided in the radial direction and the circumferential direction in polar coordinates with the center of the eye as the origin. From the data, for example, 2
The 56-byte iris data for one eye or the 512-byte iris data for both eyes are extracted and output to the main control unit 71.

When the iris data is supplied from the iris data extraction unit 85, the main control unit 71 registers the iris data of the user in the memory card 43 via the memory card interface 60 in step S7.

By the above processing, the iris data of the user is stored in the memory card 43, and thereafter, based on the registered data, the person looking at the display of the LCD 35 is collated (the photographed person is the user. It is possible to perform (determination processing of whether or not).

Next, referring to the flowchart of FIG.
Mobile phone 1 for detecting a person looking at the display of LCD 35
The detection process of will be described.

This processing is performed, for example, when the LCD 35 displays a creation screen for creating a text to be sent as an electronic mail, or when the text of the received electronic mail is displayed on the LCD 3.
When it is displayed at 5, it is performed in a predetermined cycle. Of course, it may be performed when the simple home page is displayed or when the data registered in the address book is displayed.

In step S21, the CPU 51 causes the LCD
It is determined whether or not it is a predetermined timing for detecting the person watching the display 35, and the process waits until it is determined that the predetermined timing is reached.

In step S21, the CPU 51 displays the LCD
If it is determined that the predetermined timing for detecting the person watching the display 35 is reached, the process proceeds to step S22, the camera control unit 56 is controlled, and, for example, within the range shown in FIG. 5 and FIG. It is made to photograph by 33.

In step S23, the CPU 51 performs a determination process for determining whether or not the person looking at the display on the LCD 35 is the user, based on the image data captured and obtained in step S22. To run.

In step S23, for example, the iris data extracted based on the photographed image, the size of the eye on the photographed image, the distance to the photographed eye, or the direction of the photographed eye. Based on LCD35
It is determined whether or not the person who is viewing the display is a user. For the determination process performed based on the iris data of the person performed in step S23, refer to the flowchart of FIG. 11, and for the determination process performed based on the size of the eyes on the captured image, the determination process of FIG. Referring to the flowchart, the determination process performed based on the distance to the captured eye is referred to the flowchart of FIG. 13, and the determination process performed based on the captured eye direction is the flowchart of FIG. Will be described later.

In step S24, the CPU 51 determines whether or not the line of sight of a person other than the user of the mobile phone 1 is detected based on the notification from the camera controller 56, that is, the person other than the user sees the display on the LCD 35. It is determined whether or not it is detected.

When the CPU 51 determines in step S24 that the line of sight of a person other than the user has not been detected, it skips steps S25 to S29, which will be described later, and proceeds to step S30. On the other hand, CPU
If it is determined that the line of sight by a person other than the user of the mobile phone 1 is detected in step S24,
In step S25, in order to notify the user that the display on the LCD 35 may be seen by another person,
A predetermined warning screen is displayed on the LCD 35.

FIG. 10 is a diagram showing an example of a warning screen displayed on the LCD 35 when a line of sight of a person other than the user is detected during the creation of a sentence.

In the example of FIG. 10, a message "There is a possibility that another person is looking in. Are you sure you want to stop writing the text?" Is displayed, and the user operates the operation key 36 in response to the instruction. Operate the jog dial 39, etc. to display the cursor 35 displayed below the message.
By A, it is possible to select whether or not to interrupt the creation of the sentence. Further, in order to more surely warn the user, the screen of the LCD 35 shown in FIG. 10 is a screen of a predetermined color such as red.

When the warning screen as shown in FIG. 10 is displayed, for example, when "YES" is selected by the cursor 35A, the CPU 51 interrupts the creation of the sentence (display the sentence creation screen. Interrupted), while "NO"
If is selected, the text creation continues (the text creation screen continues to be displayed).

The text of the received e-mail is displayed on the LCD.
When the line of sight of a person other than the user is detected in the state displayed in 35, a message such as "There is a possibility that another person is looking in. Do you want to interrupt the text display?" It is displayed instead of the displayed message.

In step S26, the CPU 51 determines whether or not it is instructed to continue to display the sentence creation screen or the content of the received mail based on the user's input to the operation keys 36, the jog dial 39, or the like. If it is determined that it is instructed to suspend the display, the process proceeds to step S27 and the content of the display is saved. In addition, the CPU 51 determines in step S28 that the LCD 3
Switch the display of 5.

For example, when the line of sight of a person other than the user is detected and an instruction to interrupt the display is given while the text creation screen is displayed (when "YES" in FIG. 10 is selected), The text being created is saved (step S27).
Processing), the display is switched to a standby screen or a menu screen in the initial state (processing of step S28). Also,
When the line of sight of a person other than the user is detected and an instruction to interrupt the display is given while the content of the received mail is being displayed, the information regarding the displayed received mail is stored (processing of step S27). The display is switched to the standby screen, the menu screen, or the like (processing of step S28).

In addition, for example, when a message such as "There is a possibility that another person is looking into. Please fold." Is displayed, the user folds the mobile phone 1 according to the instruction to display the message. It can be interrupted. When the mobile phone 1 is folded, the folding detection unit 58 detects it and notifies the CPU 51 of it. At this time, the CPU 51 determines in step S26 that it is instructed to suspend the display, proceeds to step S27, performs the above-described saving process, and erases the screen display (process of step S28). Then, the above detection process is ended.

On the other hand, when it is determined in step S26 that the instruction to continue the display of the sentence creation screen or the like is instructed (when "NO" in FIG. 10 is selected), the CPU 51 proceeds to step S29 and displays the warning screen. The display of the LCD 35 is returned to the original state of the screen displayed immediately before the display.
For example, when the contents displayed on the LCD 35 are contents that may be seen by others, the user selects "NO" with the cursor 35A on the warning screen as shown in FIG. LCD3 on the screen that was displayed
Return the display of 5.

In step S30, the CPU 51 determines whether or not the user has instructed to end the display of the sentence creation screen or the end of display of the received mail, and when it is determined that the end has not been instructed, Returning to step S21, the subsequent processing is repeatedly executed. In addition, CPU51
If it is determined in step S30 that the end of displaying the text creation screen has been instructed, the process ends.

As described above, the characteristic information such as iris data of the person existing in the vicinity is detected, and the LCD 3 is detected by the person different from the user of the portable terminal based on the detected characteristic information.
It is determined whether or not the display of No. 5 is visually observed, and a warning such as a predetermined warning screen is output according to the judgment result, so that the information displayed on the LCD 35 can be seen by others. , Can be more effectively prevented.

In addition, when the user gives an instruction to suspend the display of the screen that has been displayed on the warning screen, the contents of the displayed screen are saved.
For example, even if the display of the creation screen is suddenly interrupted during the creation of a sentence, it is possible to prevent the halfway created sentence from being erased. In addition to displaying the warning screen, the warning may be output by various methods such as outputting a predetermined warning sound or driving a vibration function.

Next, referring to the flowchart of FIG. 11, the portable telephone 1 which determines whether or not a person other than the user is viewing the display based on the iris data, which is executed in step S23 of FIG. The determination process will be described.

Steps S41 to S4 in FIG.
The process of 5 is the same as the process of steps S2 to S6 of FIG.

That is, in step S41, the image acquisition section 81 uses the photographing control section 73 in step S22 of FIG.
The image data captured by is acquired and output to the face area detection unit 82. The face area detection unit 82 performs step S
At 42, the area of the face of the person included in the captured image is detected, and the image data of the face is output to the face part detection unit 83. Therefore, for example, when a person other than the user is near the user, the face area of the person is also detected together with the face area of the user, and the image data of each face is output to the face part detection unit 83.

In step S43, the face part detecting section 8
3 detects the positions of face parts such as eyes and noses that form the face of each person, and outputs the image data of the face parts and the position information to the eye image acquisition section 84. The eye image acquisition unit 84
In step S44, eye image data is acquired based on the data supplied from the facial part detection unit 83,
It is output to the iris data extraction unit 85.

The iris data extraction section 85 determines in step S45.
At, the iris data of the photographed person is extracted based on the eye image data supplied from the eye image acquisition unit 84. For example, when only the user is photographed as a subject, the iris data of the user is extracted by the iris data extraction unit 85, and when the user and a person other than the user facing the mobile phone 1 are photographed, The iris data and the iris data of the person are extracted. Also, depending on the shooting angle, only the iris data of a person other than the user is extracted by the iris data extraction unit 85. The iris data extracted by the iris data extraction unit 85 is stored in the main control unit 71.
Is output to.

In step S46, the main control unit 71 stores the iris data extracted by the iris data extraction unit 85 and the iris data pre-registered in the memory card 43 (registered by the processing of FIG. 8). Collation is performed, and the process proceeds to step S47 to determine whether the extracted iris data and the iris data registered in the memory card 43 are the same.

For example, when only the iris data of the user is extracted by the iris data extraction unit 85, the main control unit 71.
Determines in step S47 that the extracted iris data and the iris data registered in the memory card 43 are the same, and proceeds to step S48.

In step S48, the main controller 71
Is that the iris data different from that of the user has not been detected, so the gaze of anyone other than the user has not been detected.
Notify U51. Then, step S2 of FIG.
The processing after 4 is executed, and the CPU 51 determines that the line of sight other than the user has not been detected.

On the other hand, when the iris data extracting unit 85 extracts the iris data of the user and the iris data of the person other than the user who is facing the mobile phone 1, or only the iris data of the person other than the user. If is extracted, the main control unit 71 determines in step S47 that the extracted iris data and the iris data registered in the memory card 43 are not the same (including iris data that are not the same). , And proceeds to step S49.

In step S49, the main controller 71
Notifies the CPU 51 that a line of sight other than the user has been detected. Then, in step S24 of FIG. 9, it is determined that the line of sight of the user other than the user is detected, and the warning screen for the user is displayed as described above.

Next, referring to the flowchart of FIG. 12, a person other than the user looks at the display based on the size of the person's eyes on the photographed image, which is executed in step S23 of FIG. A determination process of the mobile phone 1 for determining whether or not there is will be described.

For example, when the eyes of a person other than the user are photographed (largely) over a wide area, the display of the LCD 35 may be seen by a person other than the user, and the size of the photographed eye may be different. Whether or not a person other than the user is watching the display can be determined based on whether or not the size is equal to or larger than a predetermined size.

The processing of steps S61 to S64 is the same as the processing of steps S41 to S44 of FIG. That is, step S22 of FIG.
The image captured by the image acquisition unit 81 is acquired by the process (step S61), and the face area detection unit 82 detects the face area of the person based on the acquired data (step S62). Further, the face part detection unit 83 detects each part from the face image data (process of step S63), and the eye image acquisition unit 84 acquires eye image data (process of step S64). The image data of the eyes acquired by the eye image acquisition unit 84 is output to the main control unit 71.

In step S65, the main control section 71 calculates the size of a person's eyes on the photographed image based on the eye image data acquired and supplied by the eye image acquiring section 84. .

In step S66, the main controller 71
Is the result of calculating the size of the eyes of a person in the captured image, and whether or not eyes of a size equal to or larger than a predetermined threshold are detected for a plurality of people, that is, other than the user of a size equal to or larger than the predetermined size. It is determined whether the person's eyes have been detected.

When the main control unit 71 determines in step S66 that the eyes of the size equal to or larger than the predetermined threshold have not been detected for a plurality of persons, the process proceeds to step S67, and the line of sight other than the user is not detected. To the CPU 51.

On the other hand, if the main control section 71 determines in step S66 that a plurality of eyes having a size equal to or larger than the predetermined threshold value have been detected, then the process proceeds to step S68, in which it is determined that a line of sight other than the user has been detected. Notify the CPU 51.

After that, the processes from step S24 onward in FIG. 9 are executed, the CPU 51 determines that the line of sight of the user other than the user is detected, and the warning screen is displayed.

Next, with reference to the flowchart in FIG. 13, it is determined whether or not a person other than the user is viewing the display based on the distance to the person's eyes, which is executed in step S23 in FIG. The processing of the mobile phone 1 will be described.

For example, when the eyes of a person other than the user are photographed and the distance to the photographed eye is short, the display on the LCD 35 may be viewed by that person, and the distance to the photographed eye is high. However, it can be determined whether or not a person other than the user is viewing the display based on whether or not the distance is equal to or less than the predetermined distance.

The processing of steps S81 to S84 is the same as the processing of steps S61 to S64 of FIG. 12, and duplicate explanations are omitted as appropriate. That is, the image data obtained by the CCD camera 33 is acquired by the image acquisition unit 81 (step S8).
1 process), based on the acquired image data, the face area detection unit 82, the face part detection unit 83, and the eye image acquisition unit 8
4 for various processing (steps S82, S83, S84
The image data of the person's eyes acquired by the eye image acquisition unit 84 is output to the main control unit 71.

In step S85, the main controller 71
Calculates the distance from the mobile phone 1 to the photographed person's eyes. For example, when a plurality of persons are photographed, the distance to each person's eyes is calculated.

In step S86, the main controller 71
Determines whether an eye other than that of the user has been detected within a predetermined distance. For example, as shown in FIG. 14, when the distance to the eye E1 of the user holding the mobile phone 1 is X, the main control unit 71 causes the distance 2X.
Until then, it is determined whether eyes other than the user have been detected. Therefore, in the example of FIG. 14, the eye E11 of a person other than the user is within the imaging range of the mobile phone 1 in the left-right direction and within the radial range from the distance X to the user's eye E1 to the distance 2X. Is detected by the person
It is determined that the display on the LCD 35 is being viewed. That is, when the eyes of a person other than the user are detected in the vicinity of the mobile phone 1 as in the case of determining the presence / absence of another person based on the size of the eye as described above, the LCD is displayed by the person.
The display of 35 may be seen.

When the eye E12 is detected at a position 2X or more away from the mobile phone 1, it is determined that the display on the LCD 35 is not viewed by a person other than the user because the distance is too large. .

If the main control unit 71 determines in step S86 that no eyes other than the user's eyes have been detected within the predetermined distance, the process proceeds to step S87, in which no line of sight of the user is detected. Notify the CPU 51.

On the other hand, if the main control unit 71 determines in step S86 that an eye other than that of the user has been detected within the predetermined distance, the process proceeds to step S88, and the CPU 51 confirms that the line of sight of the user other than the user has been detected. Notify. After that, the processes after step S24 in FIG. 9 are executed, the CPU 51 determines that the line of sight of the user other than the user is detected, and the warning screen is displayed.

Next, referring to the flowchart of FIG. 15, it is determined whether or not a person other than the user is looking at the display based on the eye direction of the photographed person, which is executed in step S23 of FIG. The process of the mobile phone 1 for determination will be described.

For example, in the shooting range as shown in FIG. 5, when the eyes of a person are detected in the direction of the range close to the range where the user originally exists (the range in front of the mobile phone 1), the user other than the user is detected. There is a possibility that the display of the LCD 35 is being viewed by a person other than the user, and the person other than the user displays the image based on whether the direction of the eyes of the person other than the user is the direction of the area close to the area of the user. You can determine whether you are watching.

The processes of steps S101 to S104 are the same as the processes of steps S61 to S64 of FIG. 12, and the redundant description will be omitted as appropriate. That is, the image data captured and acquired by the CCD camera 33 is acquired by the image acquisition unit 81 (processing of step S101), and the face area detection unit 82, the face part detection unit 83, based on the acquired image data. And various processing by the eye image acquisition unit 84 (steps S102, S
103, S104), and the eye image acquisition unit 84
The image data of the eyes of the person acquired by the main control unit 71
Is output to.

In step S105, the main controller 71
Detects the direction of the eyes of the photographed person, with the front of the mobile phone 1 as a reference. For example, when a plurality of persons are photographed, the eye direction of each person is detected.

In step S106, the main control section 71 determines whether or not the eyes of a person are detected in the direction of the area close to the user area set as the area of the user holding the mobile phone 1.

For example, as shown in FIG. 16, as the area of the user who holds the mobile phone 1, the front side of the mobile phone 1 (a predetermined left and right angle around the one-dot chain line shown in the vertical direction in the figure) Area), the user area A1 is set, and areas within the shooting range of the CCD camera 33 that sandwich the user area A1 (areas close to the user area A1) are set to the left and right, area A2 and area A3, respectively. If so, the main control unit 71 determines whether or not there is a human eye in the direction of the area A2 and the area A3. For example, when the main control unit 71 detects the eye E22 of a person other than the user in the area A2 shown in FIG. 16, the main control unit 71 recognizes that the line of sight of the user other than the user has been detected.

When the main control unit 71 determines in step S106 that the eyes of the person are not detected in the direction of the area close to the user area, the process proceeds to step S107, and the line of sight of a user other than the user is not detected. That CPU5
Notify 1.

On the other hand, the main control unit 71 causes the step S106.
When it is determined that the eyes of the person are detected in the direction of the area close to the user area, the process proceeds to step S108, and the CPU 51 is notified that a line of sight other than the user has been detected. After that, the processing after step S24 in FIG. 9 is executed, and the warning screen is displayed.

As described above, by various methods, it is possible to determine whether the photographed person is the user,
In addition to the above, for example, whether or not the display is viewed by a person other than the user by performing face recognition and comparing it with the face feature data of the user or the feature data of the face parts that are registered in advance. May be determined. Similar to the above-mentioned iris data, these characteristic data are registered in advance in the memory card 43, for example.

In the above description, the presence of another person is detected by the mobile phone 1 and a warning screen is displayed if necessary. However, the above processing is performed by, for example, a PDA (Personal Digital It may be executed in a mobile terminal such as Assistants) or a mobile personal computer.

The series of processes described above can be executed not only by hardware but also by software. When a series of processes is executed by software, a program that constitutes the software can execute various functions by installing a computer in which dedicated hardware is installed or various programs. Possible, for example, installed in a general-purpose personal computer from a network or a recording medium.

This recording medium, as shown in FIG.
A magnetic disk 62 (including a floppy disk) on which a program is recorded, which is distributed to provide a program to a user separately from the apparatus main body, and an optical disk 63.
(CD-ROM (Compact Disk-Read Only Memory), DVD (Digit
al Versatile Disk), magneto-optical disk 64 (MD
(Registered trademark (including Mini-Disk)), or a package medium including a memory card 43, etc. ROM
Or a hard disk included in the storage unit.

In the present specification, the steps for describing the program recorded on the recording medium are not limited to the processing performed in time series according to the order described, but are not necessarily performed in time series, but may be performed in parallel. Alternatively, it also includes processes that are individually executed.

[0117]

As described above, according to the present invention, the presence of a person different from the user of the mobile phone can be detected.

Further, according to the present invention, it is possible to more effectively prevent others from seeing the information displayed on the portable terminal.

Further, according to the present invention, the state of the immediately preceding screen can be saved even when the display of the screen is interrupted.

[Brief description of drawings]

FIG. 1 is a perspective view showing an example of the external configuration of a mobile phone to which the present invention has been applied.

FIG. 2 is a perspective view showing a configuration example of the back side of the camera unit in FIG.

FIG. 3 is a block diagram showing an internal configuration example of the mobile phone of FIG.

FIG. 4 is a block diagram showing a configuration example of a camera control unit in FIG.

5 is a diagram showing an example of a shooting range of the mobile phone of FIG.

FIG. 6 is a diagram showing another example of a shooting range of the mobile phone of FIG.

7 is a block diagram showing a configuration example of an image processing unit in FIG.

8 is a flowchart illustrating a registration process of the mobile phone of FIG.

9 is a flowchart illustrating a detection process of the mobile phone of FIG.

FIG. 10 is a diagram showing a display example of an LCD.

11 is a flowchart illustrating a determination process of the mobile phone of FIG.

12 is a flowchart illustrating another determination process of the mobile phone of FIG.

13 is a flowchart illustrating still another determination process of the mobile phone of FIG.

FIG. 14 is a diagram showing an example of the positions of the eyes of a person photographed by the mobile phone of FIG.

15 is a flowchart illustrating a determination process of the mobile phone of FIG.

16 is a diagram showing another example of the positions of the eyes of a person photographed by the mobile phone of FIG.

[Explanation of symbols]

1 mobile phone, 33 CCD camera, 35 LCD,
42 memory card, 51 CPU, 55 display control unit, 56 camera control unit, 58 fold detection unit,
71 main control unit, 74 image processing unit, 84 eye image detecting unit, 85 iris data extracting unit

   ─────────────────────────────────────────────────── ─── Continued front page    F term (reference) 5B057 AA19 BA02 CA12 CA16 DA06                       DB02 DC01 DC36                 5K027 AA11 BB01 FF01 FF22 HH29                 5K067 AA30 BB04 EE02 FF23 KK17                 5L096 BA02 CA02 DA03 FA00 JA11

Claims (10)

[Claims] 1. A detecting means for detecting a characteristic of a person existing in the vicinity and outputting characteristic information corresponding to the characteristic, and a person different from a user of a mobile terminal based on the characteristic information output by the detecting means. When the display of the mobile terminal is determined to be viewed by a person different from the user of the mobile terminal, a warning is issued when the display of the mobile terminal is viewed. A mobile terminal, comprising: a warning unit for outputting. 2. The mobile terminal according to claim 1, further comprising: photographing means for photographing a subject, wherein the detecting means detects the characteristic based on an image photographed by the photographing means. . 3. A storage unit for storing first characteristic information of a user of the mobile terminal, wherein the detection unit detects a characteristic of a person photographed by the photographing unit, and a second characteristic corresponding thereto is detected. Outputting the characteristic information, and the determining means is based on whether the first characteristic information stored by the storage means and the second characteristic information output by the detecting means are the same. 3. It is determined whether or not the display is viewed by a person different from the user of the mobile terminal.
The mobile terminal described in. 4. The first feature information is information about an iris pattern of a user of the mobile terminal, and the second feature information is information about an iris pattern of a person photographed as the subject. The mobile terminal according to claim 3, wherein the mobile terminal is a mobile terminal. 5. The detecting means, as the characteristic information,
The size of a person's eyes on the image captured by the image capturing unit is detected, and the determination unit determines whether or not the detection unit detects eyes of a predetermined size or more for a plurality of persons. The mobile terminal according to claim 2, wherein it is determined whether or not the display is viewed by a person different from the user of the mobile terminal. 6. The detecting means detects the distance to a person's eyes based on the image photographed by the photographing means as the characteristic, and the judging means causes the detecting means to detect a distance within a predetermined distance. 7. It is determined whether or not the display is visually observed by a person different from the user of the mobile terminal based on whether or not the presence of a plurality of persons' eyes is detected. The mobile terminal according to 2. 7. The detecting means detects the direction of a person's eyes viewed from the mobile terminal based on an image captured by the capturing means, and the determining means uses the detecting means. , The display is visually observed by a person different from the user of the mobile terminal based on whether or not a person's eyes are detected in the direction of a predetermined area close to the area of the user of the mobile terminal. The mobile terminal according to claim 2, wherein it is determined whether or not the mobile terminal is present. 8. The mobile terminal according to claim 1, wherein the warning unit displays a screen of a predetermined color as the warning. 9. The warning means further comprises storage means for, as the warning, instructing interruption of the display, and saving the content of the display when the display is interrupted in response to the instruction from the warning means. The mobile terminal according to claim 1, wherein the mobile terminal is a mobile terminal. 10. The portable terminal is formed so as to be foldable, and the warning means instructs folding of the portable terminal as a warning for instructing interruption of the display. Mobile terminal.