JP2013161331A - Terminal device, method for controlling screen, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To adjust the direction of a screen display according to a use situation of a user's terminal device.SOLUTION: The terminal device includes: display means; a sensor for detecting inclination of a self device; and a control unit that acquires inclination information of the self device from the sensor, acquires the direction of the user's face operating the self device from a camera, when the direction of the self device is perpendicular to the gravitational force direction and the direction of the face is parallel to the gravitational force direction, causes the display means to display a display item in such a direction that the downward direction of the gravitational force direction becomes the downward direction of the self device and, when the direction of the self device and the direction of the face are perpendicular to the gravitational force direction, causes the display means to display the display item in such a direction as orienting in the direction parallel to the perpendicular direction.

Description

  The present invention relates to a terminal device, a screen control method, and a program.

  A terminal device such as a mobile phone, a smartphone, or a tablet may have a screen display adjustment function that automatically rotates the screen display horizontally and vertically according to the vertical and horizontal inclinations of the terminal device. The screen display is adjusted by detecting the direction of gravity by an acceleration sensor built in the terminal device. Thereby, even if the user tilts the terminal device, the screen display direction is adjusted accordingly, which is convenient for the user.

  When this function is used, as shown in the right diagram of FIG. 1, when the user lies down with the terminal device lying down, the mobile terminal faces sideways, as shown by the solid line portion in the right diagram. The screen display is adjusted. However, in this case, the user himself / herself is also facing sideways, so that the user wants the display object to be displayed on the screen as it is as shown by the dotted line in the right figure, but the solid line portion As described above, since the screen display adjustment is performed and the orientation of the display object is changed, there is a problem that it is inconvenient for the user.

  In this field, Patent Document 1 discloses a terminal device that activates an application in accordance with a use state.

JP 2011-216042 A

  However, since the technique described in the literature does not adjust the screen display direction, the above-described problem cannot be solved. Therefore, an object of the present invention is to provide a terminal device, a screen control method, and a program that adjust the screen display orientation according to the usage state of the user's terminal device.

  In order to achieve such an object, an embodiment of the present invention is a terminal device, which is a display unit that displays a display object, a sensor that detects the tilt of the own device, and information on the tilt of the own device is acquired from the sensor. The direction of the face of the user holding the device is obtained from the camera, the line connecting the upper and lower portions of the device is perpendicular to the direction of gravity, and the upper and lower portions of the face are If the connecting line is parallel to the direction of gravity, the display means displays the direction of the display object such that the downward direction of gravity is the downward direction of the display object, and the upper and lower parts of the device itself And the line connecting the upper part and the lower part of the face are perpendicular to the direction of gravity, the direction of the display object is set so that the vertical direction of the display object is parallel to the vertical direction. A control unit for displaying on the display means.

  In addition, the present invention provides a terminal device that includes a display unit that displays a display object and a sensor that detects the tilt of the own device, and obtains information about the tilt of the own device from the sensor and holds the own device. The step of acquiring the orientation of the person's face from the camera, the line connecting the upper and lower parts of the device is perpendicular to the direction of gravity, and the line connecting the upper and lower parts of the face is the direction of gravity The direction of the display object is displayed on the display means such that the downward direction of gravity is the downward direction of the display object, and a line connecting the upper part and the lower part of the own device, When the line connecting the upper part and the lower part of the face is perpendicular to the direction of gravity, the direction of the display object is set so that the vertical direction of the display object is parallel to the vertical direction. A control step for displaying on the display means. To provide a control program.

  In addition, the present invention provides a terminal device including a display unit that displays a display object and a sensor that detects the tilt of the own device, acquires information on the tilt of the own device from the sensor, and holds the own device. The direction of the person's face is obtained from the camera, the line connecting the upper and lower parts of the device is perpendicular to the direction of gravity, and the line connecting the upper and lower parts of the face is parallel to the direction of gravity. If it is a direction, the direction of the display object is displayed on the display means such that the downward direction of gravity is the downward direction of the display object, and the line connecting the upper part and the lower part of the device itself, and the When the line connecting the upper part and the lower part of the face is perpendicular to the direction of gravity, the display means is arranged so that the direction of the display object is parallel to the vertical direction. A screen control method to be displayed on the screen is provided.

  The present invention provides a terminal device, a screen control method, and a program for adjusting the orientation of screen display according to the usage state of the terminal device user.

It is a figure explaining a terminal device. It is a figure explaining a terminal device. It is a figure which shows an example of a structure of a terminal device. It is a figure which shows an example of a setting screen. It is a figure which shows an example of operation | movement of 1st Embodiment. It is a figure explaining a terminal device It is a figure which shows an example of a setting screen. It is a figure explaining a terminal device. It is a figure which shows an example of a memory | storage part. It is a figure which shows an example of operation | movement of 2nd Embodiment. It is a figure which shows an example of a structure of a terminal device.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same components are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  In addition, each part which comprises the apparatus of each embodiment is comprised by hardware, such as a logic circuit. Each unit includes a computer control unit, a memory, a program loaded in the memory, a storage unit such as a hard disk for storing the program, a network connection interface, and the like, and may be realized by any combination of hardware and software. Good. And unless there is particular notice, the realization method and apparatus are not limited.

  The control unit is composed of a CPU (Central Processing Unit) and the like, and operates an OS (Operating System) to control the entire apparatus and the like. For example, a program and data are loaded into a memory from a recording medium mounted on a drive device or the like. Read and execute various processes according to this. The recording medium is, for example, an optical disk, a flexible disk, a magnetic optical disk, an external hard disk, a semiconductor memory, or the like, and records a computer program so that the computer can read it. The computer program may be downloaded from an external computer (not shown) connected to the communication network.

<Embodiment 1>
FIG. 2 shows an example of the appearance of the terminal device 10. FIG. 3 shows an example of the configuration of the terminal device 10.

  As illustrated in FIGS. 2 and 3, the terminal device 10 includes a control unit 100, a display unit 110, an operation unit 120, a camera 130, a sensor 140, and a storage unit 150.

  The display unit 110 is a screen such as an LCD (Liquid Crystal Display) for displaying various information (display objects) such as characters and images. The operation unit 120 is a button or the like that gives an instruction for the user to operate the terminal device 10. The display unit 110 may be a touch panel or the like, and may be integrated with the operation unit 120.

  The camera 130 is an existing camera that captures still images such as photographs and moving images, and is provided on the same surface as the display unit 110, for example. The storage unit 150 is a memory or the like that stores various data. The sensor 140 is an existing acceleration sensor or the like that detects the tilt direction of the terminal device 10.

  The control unit 100 controls each unit described above. For example, the control unit 100 causes the sensor 140 to detect the inclination of the terminal device 10 as described later. Moreover, the control part 100 will start the camera 130, if the inclination of the terminal device 10 is detected, and will make the camera 130 image | photograph a user's face. The detailed operation of the control unit 100 will be described later.

  In the following, as shown in FIG. 1, when the user holds the terminal device 10 during normal operation of the terminal device 10, the upper part of the terminal device 10 is the upper part (or the head), and the lower part is the lower part (or the foot part). Call. The vertical direction of the terminal device 10, that is, the head and feet, for example, are stored in advance in the storage unit 150 or the like. Therefore, the sensor 140 can detect the inclination of the terminal device 10 and can grasp which direction the head or foot of the terminal device 10 is facing.

  The case where the line connecting the head and the foot is parallel to the direction of gravity is referred to as the terminal device 10 being upright. The left diagram of FIG. 1 illustrates a case where the terminal device 10 is upright. On the other hand, as shown in the right diagram of FIG. 1, when the line connecting the head and the foot is perpendicular to the direction of gravity, the direction of the terminal device 10 is referred to as the horizontal direction.

  The terminal device 10 may have a setting function capable of setting whether to enable or disable the screen display adjustment function when the terminal device 10 is in the horizontal direction (see FIG. 4). The screen display adjustment function is a function that adjusts the vertical direction of the display object displayed on the display unit 110 according to the direction of gravity detected by the sensor 140. When the screen display adjustment function is valid, the control unit 100 adjusts the vertical direction of the display object in accordance with the vertical direction of gravity detected by the sensor 140 as shown by the solid line portion in the right diagram of FIG. 110. That is, even when the terminal device 10 is tilted, the control unit 100 rotates the display object so that the downward direction of gravity is the downward direction of the display object and causes the display unit 110 to display the display object.

  On the other hand, when the screen display adjustment function is invalid, the display object is displayed on the display unit 110 as shown by the dotted line in the right diagram of FIG. 1 even when the terminal device 10 is in the horizontal direction. That is, when the screen display adjustment function is invalid, the control unit 100 maintains the orientation of the display object displayed on the display unit 110 when the terminal device 10 is upright, while maintaining the orientation of the display object. Do not adjust.

  Further, the terminal device 10 may have a setting function capable of setting whether to enable or disable the screen display adjustment function when the user and the terminal device 10 are in the horizontal direction (FIG. 4). See, Goro sleep mode). When the “user and terminal device 10” is in the horizontal direction and the screen display adjustment function is invalid, the sleep mode is valid. Here, when the user is in the horizontal direction, as shown in the right diagram of FIG. 1, when the terminal device 10 is in the horizontal direction, the direction of the line connecting the head and chin of the user's face is the terminal device. This is a case where the head is parallel to a straight line connecting the 10 heads and the feet. The control unit 100 can determine whether or not the user's orientation is in the horizontal direction by photographing the user's face image with the camera 130 and using the existing face orientation determination function. Note that the user can check the screen on the setting screen displayed on the display unit 110, for example, as in the setting screen in FIG. 4, and the screen display adjustment function when the terminal device 10 is in the horizontal direction. You can set whether to enable or disable. This setting is stored in the storage unit 150.

  Next, the operation of the present embodiment will be described with reference to FIG.

  First, the sensor 140 detects a change in the orientation of the terminal device 10 (tilt of upright or lateral direction) (step S1).

  Based on the detection result of the sensor 140, the control unit 100 determines the current orientation of the terminal device 10 and determines whether the terminal device 10 is in the horizontal direction (step S2).

  As a result of determining the orientation of the terminal device 10 by the control unit 100, if the terminal device 10 is in the horizontal direction (step S2, Yes), the control unit 100 has a screen display adjustment function when the terminal device 10 is in the horizontal direction. Whether the setting is valid is checked with reference to the storage unit 150 (step S3). When step S3 is No, the control unit 100 ends the processing while leaving the setting of the screen display adjustment function invalid.

  When step S3 is Yes, the control part 100 confirms whether the go-sleep mode is effective with reference to the memory | storage part 150 (step S4). If the Goro sleep mode is invalid (No at Step S4), the control unit 100 performs screen display adjustment and ends the process.

  On the other hand, when step S4 is Yes, the terminal device 10 activates the camera 130, detects the face of the user, and detects the face direction (step S5). For the detection of the face and the face direction, a face detection function and a pupil detection function that are generally installed in mobile phones and digital cameras are used.

  The designer or the like may provide a measure for preventing the camera 130 from erroneously detecting the face of another person who is not the user operating the terminal device 10. As a preventive measure, for example, the control unit 100 may determine whether the detected face is the face of the user operating the terminal device 10 based on the size of the face detected by the camera 130. . Specifically, the detection unit 100 may determine that the user is not a user when the detected face size is equal to or smaller than a threshold value. This is because the user of the terminal device 10 is in a position where the terminal device 10 can be held, and therefore the size of the face captured by the camera 130 is considered not to be so small.

  Next, the control unit 100 determines whether or not the face direction is the horizontal direction (step S6). That is, as shown in FIG. 6, when the longitudinal direction of the terminal device 10 (the direction of the line connecting the head and the foot) and the face direction (the direction of the line connecting the head and the jaw) are parallel, the control is performed. The unit 100 determines that the user is operating the terminal device 10 with the user lying down and lying down, and invalidates the screen display adjustment function according to the orientation of the terminal device 10 (step S7).

  Next, the control unit 100 detects when the camera 130 no longer detects the user's face from this state, when the orientation of the user's face is changed, or when the orientation of the terminal device 10 is changed (step). (S8, Yes) determines that the user is not operating the terminal device 10 while lying down and lies down, and returns the screen display adjustment function to be valid (step S9).

  Note that step S3 and step S4 can be omitted if it is assumed that the screen display adjustment function and the sleep mode are set in advance.

  In addition, when the orientation of the terminal device 10 changes, the control unit 100 displays a selection screen for selecting whether to enable or disable the screen display adjustment function on the display unit 110, and allows the user to select the screen display function. Setting may be performed. This makes it possible for the user to set the screen display adjustment function when the direction of the terminal device 10 is horizontal.

  Also, as shown in FIG. 7, when a specific application is activated when the Goro sleep mode is set, the screen display adjustment function may be enabled. As shown in FIG. 7, the user can set which application is to activate the screen display adjustment function when the application is started, and the setting is stored in the storage unit 150. In the example shown in FIG. 7, for example, the go-sleep mode is valid when using mail and using the telephone, and the go-sleep mode is disabled when using 1Seg. That is, the control unit 100 may invalidate the screen display adjustment function only when a predetermined application such as a mail application is operating. In this way, the screen display adjustment function may be effective when the user is watching TV using one-segment broadcasting, etc., but when using e-mail that requires character input etc. The user can make flexible adjustments such as disabling the screen display adjustment function.

  According to the present embodiment, the screen display can be adjusted according to the usage state of the user of the terminal device 10. The reason is that even if the terminal device has a screen display adjustment function, the screen display adjustment function is invalidated when the terminal device is in the horizontal direction and the user is also in the horizontal direction.

<Embodiment 2>
Next, the terminal device 10 in the 3rd Embodiment of this invention is demonstrated.

  The terminal device 10 in the present embodiment has the same basic operation as that of the first embodiment. In the first embodiment, the control unit 100 detects the user's face taken by the camera 130 as it is and determines whether or not the user is in the horizontal direction. On the other hand, in this embodiment, the first is that the user's face photograph and the orientation of the terminal device 10 when the user takes the face photograph are stored in advance in the storage unit 150 in association with each other. Different from the embodiment.

  For example, as shown in FIG. 6, the user takes a picture of his / her face with the camera 130 when the terminal device 10 and the user are in the horizontal direction. At this time, the control unit 100 acquires the orientation of the terminal device 10 from the sensor 140, and stores the face photograph of the user and the orientation of the terminal device 10 in association with each other in the storage unit 150 (see the first line in FIG. 9).

  In the example illustrated in FIG. 6, the head of the terminal device 10 is facing right and the foot is facing left. Such a state is that the terminal device 10 is in the right direction (the opposite case is called the left direction). The orientation of the user's face image is said to be the front direction when the user's head is located on the head side of the terminal device 10 and the user's jaw is located on the foot side. That is, in the case of the left figure of FIG. 1 or the case of FIG. 6, the user (face) is in the front direction. Further, as shown in the left diagram of FIG. 8, when the face is facing left with respect to the direction connecting the head and the foot of the terminal device 10, the user (or face) is in the left direction, and When the direction is 180 degrees opposite (the right diagram in FIG. 8), the direction is the right direction. As described above, the orientation of the terminal device 10 is specified by the relationship with respect to gravity, while the user orientation is specified by the relationship with respect to the orientation of the terminal device 10. Note that the method of specifying such a direction can be changed as appropriate.

  FIG. 9 shows a state of the storage unit 150 that stores the face photograph of the user and the orientation of the terminal device 10 in association with each other. The first line of FIG. 9 indicates that the user's face is in the front direction and the terminal device 10 is in the right direction. This means that the user took a picture of his / her face with the camera 130 with the terminal device 10 in the right direction and the user lying down in the right direction as shown in FIG. 1 and FIG. In this way, by associating the direction of the user's face photograph with the orientation of the terminal device 10, the storage unit 150 can store in what state the user has photographed his / her face.

  Next, the operation of the terminal apparatus 10 in the present embodiment will be described using FIG. In the following, it is assumed that both the screen display adjustment function and the go-sleep mode of the terminal device 10 are effective.

  First, the sensor 140 detects that the terminal device 10 is in the horizontal direction (step A1). Upon receiving the detection result from the sensor 140, the control unit 100 activates the camera 130 and takes a picture of the user's face (step A2).

  Next, the control unit 100 compares the photographed user's face with the user's face stored in the storage unit 150 (step A3). For example, when the current orientation of the terminal device 10 is the right direction, the control unit 100 extracts from the storage unit 150 a face in which the orientation of the terminal device 10 is associated with the right direction. Next, the control unit 100 compares the extracted face with the photographed user's face, and determines whether or not they are the same person and whether or not the face orientations match.

  As a result, when the face is the same person and the face direction is the front (referred to as face matching) (step A4, Yes), the control unit 100 causes the terminal device to lie down in the horizontal direction. 10 is determined to be operated, and the screen display adjustment function is disabled (step A5).

  Thereafter, the control unit 100 may instruct the camera 130 to periodically take a photo, and periodically compare the taken photo with the photo stored in the storage unit 150. As a result of comparison, when the user's face is no longer detected, when the orientation of the user's face is changed, or when the control unit 100 determines that the orientation of the terminal device 10 has been changed (step A6, Yes), It is determined that the user is not lying down and lying down, and the screen display adjustment function is enabled again (step A7).

  When the terminal device 10 is in the horizontal direction, if the orientation of the user's face image captured by the camera 130 is the same direction as the registered face image or a direction rotated by 180 degrees, the control unit 100 may invalidate the image display adjustment function. This is because such a state means that the current direction (lateral direction) of the terminal device 10 and the user's face are parallel, and both the user and the terminal device 10 are in the horizontal direction. This is because it is considered that the terminal device 10 is operated in the state where the user lies down.

  In FIG. 9, the storage device 150 stores the photograph and the orientation of the terminal device 10 in association with each other. In addition, the storage device 150 enables or disables the screen display adjustment function. Such information may be stored in association with each other. Thus, the control unit 100 can control whether to enable or disable the screen display function according to the current orientation of the terminal device 10 and the orientation of the user's face.

  According to the present embodiment, the screen display can be adjusted according to the usage state of the user of the terminal device 10. The reason is that even if the terminal device has a screen display adjustment function, the screen display adjustment function is invalidated when the terminal device is in the horizontal direction and the user is also in the horizontal direction.

<Embodiment 3>
Next, a third embodiment of the present invention will be described.

  The terminal device 10 in this Embodiment is provided with the display part 110 which displays a display thing, the sensor 140 which detects the inclination of an own apparatus, and the control part 100. FIG.

  The terminal device 10 has the above-described screen display adjustment function, and the control unit 100 rotates the display object so that the downward direction of gravity is the downward direction of the display object even when the terminal device 10 is tilted, and the display unit 110. To display.

  The control unit 100 acquires information about the tilt of the terminal device 10 from the sensor 140 and acquires the orientation of the user's face from the camera 130.

  When the terminal device 10 is in the horizontal direction and the face is not tilted, the control unit 100 keeps the screen display adjustment function valid, and the direction of the display object is such that the downward direction of gravity is downward. Is displayed on the display unit 110.

  On the other hand, when the orientation of the terminal device 10 and the orientation of the face are both perpendicular to the direction of gravity, the control unit 100 disables the screen display adjustment function and makes the orientation of the display object parallel to the vertical direction. Display on the display unit 110 in any direction.

  According to the present embodiment, the screen display can be adjusted according to the usage state of the user of the terminal device 10. The reason is that even if the terminal device has a screen display adjustment function, the screen display adjustment function is invalidated when the terminal device is in the horizontal direction and the user is also in the horizontal direction.

  While the present invention has been described with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

DESCRIPTION OF SYMBOLS 10 Terminal device 100 Control part 110 Display part 120 Operation part 130 Camera 140 Sensor 150 Storage part

Claims (9)

Display means for displaying a display object;
A sensor that detects the tilt of the device;
Obtain information about the tilt of the device from the sensor, obtain the orientation of the face of the user holding the device from the camera,
If the line connecting the upper and lower parts of the device is perpendicular to the direction of gravity and the line connecting the upper and lower parts of the face is parallel to the direction of gravity, The direction is displayed on the display means such that the downward direction of gravity is the downward direction of the display object,
When the line connecting the upper part and the lower part of the own device and the line connecting the upper part and the lower part of the face are perpendicular to the direction of gravity, the direction of the display object is determined by the vertical direction of the display object. A control unit for displaying on the display means so as to be parallel to the vertical direction;
A terminal device comprising: Storage means for storing a user's face image in which the vertical direction of the own device matches the vertical direction of the face;
The control unit activates the camera to capture a face image of the user when a line connecting an upper part and a lower part of the own apparatus is inclined in a direction perpendicular to a gravitational direction,
When the face image stored in the storage means and the captured face image are face images of the same person and the face orientations match, a line connecting the upper part and the lower part of the own device and the The line connecting the upper part and the lower part of the face is determined to be perpendicular to the direction of gravity, and the display object is placed on the display means so that the vertical direction of the display object is parallel to the vertical direction. The terminal device according to claim 1 to be displayed.   The control unit displays the display object only when a line connecting the upper part and the lower part of the own apparatus is perpendicular to the direction of gravity and a predetermined application is running in the own apparatus. The terminal device according to claim 1, wherein the display unit displays the orientation of the display object such that a vertical direction of the display object is parallel to the vertical direction. Display means for displaying the display object, a sensor for detecting the tilt of the device itself,
In a terminal device comprising
An acquisition step of acquiring information about the tilt of the own device from the sensor, and acquiring the orientation of the face of the user holding the own device from the camera;
If the line connecting the upper and lower parts of the device is perpendicular to the direction of gravity and the line connecting the upper and lower parts of the face is parallel to the direction of gravity, The direction is displayed on the display means such that the downward direction of gravity is the downward direction of the display object,
When the line connecting the upper part and the lower part of the device itself and the line connecting the upper part and the lower part of the face are perpendicular to the direction of gravity, the direction of the display object is A control step of causing the display means to display the direction so that the direction is parallel to the vertical direction;
A screen control program that executes The terminal device further comprising storage means for storing a user's face image in which the vertical direction of the own device matches the vertical direction of the face,
An imaging step of activating the camera and capturing the face image of the user when a line connecting the upper part and the lower part of the own apparatus is inclined in a direction perpendicular to the direction of gravity;
When the face image stored in the storage means and the captured face image are face images of the same person and the face orientations match, a line connecting the upper part and the lower part of the own device and the The line connecting the upper part and the lower part of the face is determined to be perpendicular to the direction of gravity, and the display object is placed on the display means so that the vertical direction of the display object is parallel to the vertical direction. The control step of displaying;
5. The screen control program according to claim 4, wherein:   Only when the line connecting the upper part and the lower part of the own device is perpendicular to the direction of gravity and a predetermined application is running in the own device, the direction of the display object is 6. The screen control program according to claim 4, wherein the terminal device is caused to execute the control step of causing the display means to display the display object so that the vertical direction of the display object is parallel to the vertical direction. Display means for displaying the display object, a sensor for detecting the tilt of the device itself,
A terminal device comprising:
Obtain information about the tilt of the device from the sensor, obtain the orientation of the face of the user holding the device from the camera,
If the line connecting the upper and lower parts of the device is perpendicular to the direction of gravity and the line connecting the upper and lower parts of the face is parallel to the direction of gravity, The direction is displayed on the display means such that the downward direction of gravity is the downward direction of the display object,
When the line connecting the upper part and the lower part of the device itself and the line connecting the upper part and the lower part of the face are perpendicular to the direction of gravity, the direction of the display object is A screen control method for displaying on the display means such that the direction is parallel to the vertical direction. The terminal device further comprising storage means for storing a user's face image in which the vertical direction of the own device matches the vertical direction of the face,
When the line connecting the upper part and the lower part of the device is tilted in the direction perpendicular to the direction of gravity, the camera is activated to capture the user's face image,
When the face image stored in the storage means and the captured face image are face images of the same person and the face orientations match, a line connecting the upper part and the lower part of the own device and the The line connecting the upper part and the lower part of the face is determined to be perpendicular to the direction of gravity, and the display object is placed on the display means so that the vertical direction of the display object is parallel to the vertical direction. The screen control method according to claim 7 to be displayed.   Only when the line connecting the upper part and the lower part of the own device is perpendicular to the direction of gravity and a predetermined application is running in the own device, the direction of the display object is The screen control method according to claim 7, wherein the display unit displays the display object so that a vertical direction of the display object is parallel to the vertical direction.

Images