JP6227234B2 - Terminal device - Google Patents

Description

  The present invention relates to a terminal device, and more particularly to a portable terminal device that displays an image.

  In recent years, portable terminal devices such as smartphones and tablet PCs are widely used. Such a terminal device has a function of rotating the image display so that the lower direction of the image display always coincides with the direction of gravity. That is, it is possible to prevent the image display direction from changing regardless of whether the terminal device is held vertically or horizontally (see, for example, Patent Document 1).

JP 2005-229490 A

  However, the above-described conventional technology has a problem that it is difficult to display an image suitable for the user's posture. 16 and 17 are diagrams for explaining such a conventional problem.

  As shown in FIG. 16, when the user changes his / her posture from a posture where the body is raised (posture A) to a lying posture (posture B), the image display rotation function described above causes the image display to move downward. The orientation of the image display is changed so that is coincident with the direction of gravity.

  However, when the orientation of the image display is changed, there is a problem that the image display becomes horizontal for the user who has laid down, and the image becomes difficult to see. It is desirable for the lying user to maintain the image display direction.

  Also, as shown in FIG. 17, the user may view the image of the terminal device in various postures other than the posture B. Therefore, if the image display suitable for each posture can be performed, the user can see the terminal device. Will be easier to use. However, the above-described prior art does not achieve such a purpose.

  In view of the above situation, the present invention provides a terminal device that enables a user to display an image in an orientation suitable for the posture when the user views an image displayed on the terminal device in various postures. Objective.

In order to solve the above problems, a first technical means of the present invention includes a display unit, a gravity direction detection unit that detects the direction of gravity in a predetermined coordinate system of the terminal device, and a plane parallel to the display unit. In the case of the xy plane, a change detection unit that detects a change in the direction of detection of gravity parallel to the xy plane, an imaging unit that captures an image of the user's face, and an image of the user's face captured by the imaging unit. Based on the direction determination unit that determines the orientation of the user's face based on the terminal device, the direction of gravity detected in the detection result by the change detection unit, and the determination result by the direction determination unit, the display And a display control unit that controls the display direction of the image displayed on the unit.
According to a second technical means of the present invention, in the first technical means, when the determination result by the orientation determination unit determines that the orientation of the user's face has changed with reference to the terminal device, The image display direction is changed.
According to a third technical means of the present invention, in the first technical means, when the determination result by the orientation determination unit determines that the orientation of the user's face has not changed with reference to the terminal device, The image display direction is not changed.

According to a fourth technical means of the present invention, in the first technical means, the operation of the imaging unit is stopped when the orientation determination unit does not succeed in recognizing the user's face .
According to a fifth technical means of the present invention, in the first technical means, when the orientation determination unit does not succeed in recognizing the user's face, the display control unit detects the detection result by the change detection unit and the The display direction of the image based on the determination result by the direction determination unit is not controlled.
The sixth technical means of the present invention is characterized in that, in the first technical means, the imaging unit does not capture an image of the user's face when the gravity detection direction is not changed.
The seventh technical means of the present invention is characterized in that, in the first technical means, the direction of display of the image is not changed when the gravity detection direction is not changed.

The eighth technical means of the present invention is characterized in that, in the first technical means, the orientation determining unit determines the orientation of the face based on the positions of both eyes of the user.

According to a ninth technical means of the present invention, in the first technical means, the orientation determination unit determines the orientation of the face based on the positions of the user's white eyes and black eyes.

According to a tenth technical means of the present invention, in the first technical means, the orientation determination unit determines the orientation of the face based on a shape of glasses worn by the user.

An eleventh technical means of the present invention is characterized in that, in the first technical means, the orientation determination unit recognizes the face of the user and determines the orientation of the face based on a result of the face recognition. To do.
According to a twelfth technical means of the present invention, when a gravity direction detecting step for detecting a direction of gravity in a predetermined coordinate system of the terminal device and a plane parallel to the display unit of the terminal device is an xy plane, the xy plane A change detection step for detecting a change in the direction of detection of gravity parallel to the image, an imaging step for capturing an image of the user's face, and the terminal device based on the image of the user's face captured in the imaging step An orientation determination step for determining the orientation of the user's face, and the display of the image displayed on the display unit based on the detection direction of the gravity in the detection result of the change detection step and the determination result of the orientation determination step. And a display control step for controlling the orientation.
The thirteenth technical means of the present invention is a computer program characterized by causing a computer to execute the twelfth technical means.

  According to the present invention, since the orientation of the user's face is determined and the display direction of the image is controlled based on the determination result, the user views the images displayed on the terminal device in various postures. In this case, the image can be displayed in a direction suitable for the posture.

It is a figure which shows an example of a structure of the terminal device which concerns on Embodiment 1 of this invention. It is a figure explaining determination of the direction of a face. It is a figure explaining the process which determines the direction of a face from the positional relationship of both eyes. It is a figure explaining the process which determines the direction of a face from the position of a white eye and a black eye. It is a figure explaining the process which determines direction of a face from the shape of glasses. It is a figure explaining control of the display direction of an image. It is a flowchart which shows an example of the process sequence of the image display control process which concerns on Embodiment 1 of this invention. It is a flowchart which shows an example of the process sequence of the face direction determination process shown to step S104 of FIG. It is a flowchart which shows another example of the process sequence of the face direction determination process shown to step S104 of FIG. It is a flowchart which shows another example of the process sequence of the face direction determination process shown to step S104 of FIG. It is a flowchart which shows another example of the process sequence of the face direction determination process shown to step S104 of FIG. It is a figure which shows an example of a structure of the terminal device which concerns on Embodiment 2 of this invention. It is a figure explaining control of the display direction of an image. It is a flowchart which shows an example of the process sequence of the image display control process which concerns on Embodiment 2 of this invention. It is a flowchart which shows an example of the process sequence of the image display control process which concerns on Embodiment 2 of this invention. It is a figure explaining the conventional problem. It is a figure explaining the conventional problem.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
FIG. 1 is a diagram illustrating an example of a configuration of a terminal device 10 according to Embodiment 1 of the present invention. The terminal device 10 is a portable terminal device such as a smartphone or a tablet PC. The terminal device 10 includes an imaging unit 11, a touch panel unit 12, a communication processing unit 13, a storage unit 14, and a control unit 15.

  The imaging unit 11 includes a camera and is a processing unit that captures an image with the camera. The touch panel unit 12 is a touch panel including an input unit that receives information input from a user and a display unit that displays various information. The communication processing unit 13 is a processing unit that performs wireless communication or wired communication with another device via a wireless network or a wired network.

  The storage unit 14 is a storage device such as a memory or a hard disk device. The storage unit 14 stores image data 14a, face orientation data 14b, and the like. The image data 14 a is image data displayed by the touch panel unit 12. The face orientation data 14b is data of the user's face orientation determined by an orientation determination unit 15a described later.

  The control unit 15 is a control device such as a CPU (Central Processing Unit) that controls each unit of the terminal device 10. The control unit 15 includes an orientation determination unit 15a and a display control unit 15b.

  The orientation determination unit 15a is a processing unit that determines the orientation of the user's face. Specifically, the orientation determination unit 15a performs a recognition process for recognizing the entire face using the video imaged by the imaging unit 11, and determines the orientation of the user's face based on the result. Thus, by recognizing the entire face, the orientation of the user's face can be accurately determined.

  FIG. 2 is a diagram illustrating determination of the face orientation. As a result of the face recognition processing, the direction determination unit 15a recognizes whether the user's face is in a vertical orientation as shown in FIG. 2A or as shown in FIG. Next, it is determined whether it is sideways. At that time, the direction determination unit 15a determines whether the terminal device 10 is in the vertical direction or the horizontal direction.

  Here, the imaging unit 11 captures a user's face image at a short predetermined interval. Then, the orientation determination unit 15a sequentially performs face recognition processing on each captured face image, and determines whether or not the face has changed to portrait orientation or landscape orientation. The direction determination unit 15a can track the change to determine whether the user's face is upside down or whether the user's face is tilted to the right or the left.

  2A shows an example in which the user's face is not upside down, and FIG. 2B shows an example in which the user's face is tilted to the left side of the user. Yes.

  In addition, the orientation determination unit 15a may detect both eyes by face recognition processing on the captured video and determine the orientation of the face from the positional relationship between the eyes. Thereby, the orientation of the face can be determined efficiently. FIG. 3 is a diagram illustrating a process for determining the face orientation from the positional relationship between both eyes.

  As shown in FIG. 3A, when both eyes are arranged side by side, the orientation determination unit 15a determines that the user's face is in the portrait orientation. Further, as shown in FIG. 3B, when both eyes are aligned vertically, the orientation determination unit 15a determines that the user's face is sideways. At the time of this determination, the direction determination unit 15a determines whether the terminal device 10 is in portrait orientation or landscape orientation.

  Also in this case, the imaging unit 11 captures the user's face image at a short predetermined interval. Then, the orientation determination unit 15a sequentially performs face recognition processing on each captured face image to determine whether the positional relationship between both eyes has changed. The direction determination unit 15a can track the change to determine whether the user's face is upside down or whether the user's face is tilted to the right or the left.

  FIG. 3A shows an example in which the orientation of the user's face is not upside down, and FIG. 3B shows an example in which the user's face is inclined to the left side of the user. Yes.

  Furthermore, the orientation detection unit 16a may determine the orientation of the user's face from the position of the user's white and black eyes. As a result, even when only one eye is recognized in the video of the user's face, the face orientation can be determined. FIG. 4 is a diagram for explaining processing for determining the face direction from the position of white eyes / black eyes.

  As shown in FIG. 4A, when white eyes and black eyes are arranged side by side, the orientation determination unit 15a determines that the user's face is in the portrait orientation. Further, as shown in FIG. 4B, when white eyes and black eyes are arranged vertically, the orientation determination unit 15a determines that the user's face is in landscape orientation. At the time of this determination, the direction determination unit 15a determines whether the terminal device 10 is in portrait orientation or landscape orientation.

  Also in this case, the imaging unit 11 captures the user's face image at a short predetermined interval. Then, the orientation determination unit 15a sequentially performs face recognition processing on each captured face image, and determines whether the position of white eyes / black eyes has changed. The direction determination unit 15a can track the change to determine whether the user's face is upside down or whether the user's face is tilted to the right or the left.

  FIG. 4A shows an example in which the orientation of the user's face is not upside down, and FIG. 4B shows an example in which the user's face is tilted to the left side of the user. Yes.

  In addition, when the user wears glasses such as sunglasses, the glasses may be detected by image recognition processing of the captured video, and the face orientation may be determined from the shape of the glasses. Thereby, even when the user wears glasses such as sunglasses, the orientation of the user's face can be determined. FIG. 5 is a diagram for explaining the process of determining the face direction from the shape of the glasses.

  As shown in FIG. 5A, when the two lenses are arranged side by side, the orientation determination unit 15a determines that the user's face is in the portrait orientation. Also, as shown in FIG. 5B, when the two lenses are arranged vertically, the orientation determination unit 15a determines that the user's face is sideways. At the time of this determination, the direction determination unit 15a determines whether the terminal device 10 is in portrait orientation or landscape orientation.

  Also in this case, the imaging unit 11 captures the user's face image at a short predetermined interval. Then, the orientation determination unit 15a sequentially performs face recognition processing on each captured face image to determine whether or not the lens positional relationship has changed. The direction determination unit 15a can track the change to determine whether the user's face is upside down or whether the user's face is tilted to the right or the left.

  FIG. 5A shows an example in which the orientation of the user's face is not upside down, and FIG. 5B shows an example in which the user's face is tilted to the left side of the user. Yes.

  Returning to the description of FIG. 1, the display control unit 15b is a processing unit that controls the display direction of an image based on the determination result by the direction determination unit 15a. FIG. 6 is a diagram for explaining the control of the image display direction.

  The display control unit 15b causes the touch panel unit 12 to display the image 20 sideways when the orientation determination unit 15a determines that the user's face is sideways. At that time, the display control unit 15b prevents the image 20 from being displayed upside down for the user depending on whether the user's face is tilted to the right side or the left side of the user.

  Further, the display control unit 15b causes the touch panel unit 12 to display the image 20 in the portrait orientation when the orientation determination unit 15a determines that the user's face is in the portrait orientation. At that time, the display control unit 15b prevents the image 20 from being displayed upside down for the user, depending on whether the orientation of the user's face is upside down.

  Next, a processing procedure of image display control processing according to the first embodiment of the present invention will be described. FIG. 7 is a flowchart showing an example of the processing procedure of the image display control process according to the first embodiment of the present invention.

  First, the imaging unit 11 of the terminal device 10 illustrated in FIG. 1 activates the camera (step S101) and captures the user's video (step S102). Then, the orientation determination unit 15a performs a process for determining the orientation of the user's face from the user's video as described with reference to FIGS. 2 to 5 (step S103). The processing procedure of this processing will be described in detail later.

  Subsequently, the orientation determination unit 15a determines whether or not the face orientation determination process in step S103 is the first determination process (step S104). If the face orientation determination process is the first determination process (YES in step S104), the storage unit 14 stores the face orientation information obtained as a result of the determination as face orientation data 14b (step S104). S105). Thereafter, the processing after step S102 is executed again.

  When it is determined in step S104 that the face orientation determination process is not the first determination process (NO in step S104), the orientation determination unit 15a stores the face stored in the storage unit 14 as the face orientation data 14b. The orientation information and the face orientation information obtained as a result of the determination in step S103 are compared to determine whether or not the face orientation has changed (step S106).

  If the face orientation has changed (YES in step S106), the display control unit 15b changes the image display orientation as described with reference to FIG. 6 according to the face orientation (step S107). ). The storage unit 14 stores the face orientation information determined in step S103 as face orientation data 14b (step S108). This information is referred to when the determination process in step S106 is performed next.

  After the process of step S108 or when the face orientation has not changed in step S106 (NO in step S106), the orientation determination unit 15a has received an instruction to end this image display control process from the user. It is determined whether or not (step S109). This end instruction is, for example, an instruction not to change the orientation of the image.

  In step S109, when an instruction to end the image display control process is received from the user (YES in step S109), the imaging unit 11 stops the camera (step S110). Then, this image display control process ends. When an instruction to end the image display control process has not been received from the user (NO in step S109), the processes after step S102 are executed again.

  Next, the face orientation determination process shown in step S103 of FIG. 7 will be described. FIG. 8 is a flowchart showing an example of the processing procedure of the face orientation determination process shown in step S103 of FIG. Specifically, FIG. 8 is a flowchart in the case of determining the face orientation by the face recognition process described in FIG.

  First, the orientation determination unit 15a of the terminal device 10 performs face recognition processing (step S201). Then, the orientation determination unit 15a determines whether the face recognition is successful (step S202). When the face recognition is successful (YES in step S202), the direction determination unit 15a determines the face direction with reference to the terminal device 10 as described with reference to FIG. 2 based on the face recognition result. Determination is made (step S203).

  Specifically, the orientation determination unit 15a determines whether the orientation of the face is portrait or landscape. Furthermore, when the orientation of the user's face is vertical, the orientation determination unit 15a determines whether it is upside down. In addition, when the orientation of the user's face is landscape, the orientation determination unit 15a determines whether the user's face is tilted to the right or left of the user.

  If the face recognition is not successful in step S202 (NO in step S202), the processing from step S109 shown in FIG. 7 is executed.

  Next, as described with reference to FIG. 3, a case where the face orientation is determined by detecting both eyes of the user's face will be described. FIG. 9 is a flowchart showing another example of the processing procedure of the face orientation determination process shown in step S103 of FIG.

  First, the orientation determination unit 15a of the terminal device 10 executes recognition processing for both eyes (step S301). Then, the orientation determination unit 15a determines whether the recognition of both eyes has been successful (step S302). When the recognition of both eyes is successful (YES in step S302), the orientation determination unit 15a determines the orientation of the face based on the recognition result of both eyes, as described in FIG. Determination is made (step S303).

  Specifically, the orientation determination unit 15a determines whether the orientation of the face is portrait or landscape. Furthermore, when the orientation of the user's face is vertical, the orientation determination unit 15a determines whether it is upside down. In addition, when the orientation of the user's face is landscape, the orientation determination unit 15a determines whether the user's face is tilted to the right or left of the user.

  If the recognition of both eyes is not successful in step S302 (in the case of NO in step S302), the processing after step S109 shown in FIG. 7 is executed.

  Next, as described with reference to FIG. 4, a case where the face orientation is determined by detecting the user's white eyes and black eyes will be described. FIG. 10 is a flowchart showing another example of the procedure of the face orientation determination process shown in step S103 of FIG.

  First, the orientation determination unit 15a of the terminal device 10 performs white eye and black eye recognition processing (step S401). Then, the orientation determination unit 15a determines whether the recognition of white eyes and black eyes has succeeded (step S402). When white eye and black eye recognition succeeds (in the case of YES in step S402), the direction determination unit 15a uses the terminal device 10 as a reference based on the white eye and black eye recognition results as described in FIG. The direction of the face is determined (step S303).

  Specifically, the orientation determination unit 15a determines whether the orientation of the face is portrait or landscape. Furthermore, when the orientation of the user's face is vertical, the orientation determination unit 15a determines whether it is upside down. In addition, when the orientation of the user's face is landscape, the orientation determination unit 15a determines whether the user's face is tilted to the right or left of the user.

  If recognition of white eyes and black eyes is not successful in step S402 (in the case of NO in step S402), the processing after step S109 shown in FIG. 7 is executed.

  Next, as described with reference to FIG. 5, a case where the face orientation is determined by detecting the shape of the glasses worn by the user will be described. FIG. 11 is a flowchart showing another example of the processing procedure of the face orientation determination process shown in step S103 of FIG.

  First, the orientation determination unit 15a of the terminal device 10 executes recognition processing for two lenses of glasses (step S501). Then, the orientation determination unit 15a determines whether or not the two lenses have been successfully recognized (step S502). When the recognition of the two lenses is successful (in the case of YES in step S502), the direction determination unit 15a uses the terminal device 10 as a reference as described in FIG. 5 based on the recognition result of the two lenses. The direction of the face is determined (step S503).

  Specifically, the orientation determination unit 15a determines whether the orientation of the face is portrait or landscape. Furthermore, when the orientation of the user's face is vertical, the orientation determination unit 15a determines whether it is upside down. In addition, when the orientation of the user's face is landscape, the orientation determination unit 15a determines whether the user's face is tilted to the right or left of the user.

  In step S502, when the recognition of the two lenses is not successful (NO in step S502), the processing from step S109 shown in FIG. 7 is executed.

(Embodiment 2)
In the first embodiment, the display direction of the image is controlled based on the determination result of the face direction. However, the display direction of the image may be controlled based on the direction of gravity. In the second embodiment, such a case will be described.

  FIG. 12 is a diagram illustrating an example of a configuration of the terminal device 30 according to the second embodiment of the present invention. The terminal device 30 is a portable terminal device such as a smartphone or a tablet PC. The terminal device 30 includes a gravity direction detection unit 31, an imaging unit 32, a touch panel unit 33, a communication processing unit 34, a storage unit 35, and a control unit 36.

  The gravity direction detection unit 31 is a sensor that detects the direction of gravity in a predetermined coordinate system set in the terminal device 30. The imaging unit 32 includes a camera and is a processing unit that captures an image with the camera. The imaging unit 32 captures a user's video when a change detection unit 36a described later detects a change in the direction of gravity detection in the predetermined coordinate system. The captured video is used to determine the orientation of the user's face.

  If a change in the direction of gravity detection is detected, it is highly likely that the orientation of the user's face has changed. Therefore, when a change in the detection direction of gravity is detected, the user's face direction can be efficiently determined by photographing the user's video with the camera and determining the user's face direction.

  Returning to the description of FIG. 12, the touch panel unit 33 and the communication processing unit 34 are the same as the touch panel unit 12 and the communication processing unit 13 described in FIG. 1.

  The storage unit 35 is a storage device such as a memory or a hard disk device. The storage unit 35 stores image data 35a, face orientation data 35b, gravity direction data 35c, and the like. The image data 35a and the face orientation data 35b are the same as the image data 14a and the face orientation data 14b described in FIG. The gravity direction data 35 c is data on the direction of gravity detected by the gravity direction detection unit 31.

  The control unit 36 is a control device such as a CPU that controls each unit of the terminal device 30. The control unit 36 includes a change detection unit 36a, a direction determination unit 36b, and a display control unit 36c.

  The change detection unit 36a is a processing unit that detects a change in the gravity detection direction. The direction determination unit 36b is the same as the direction determination unit 15a described in FIG. The display control unit 36c is a processing unit that controls the display direction of the image based on the detection result by the change detection unit 36a and the determination result by the direction determination unit 36b. FIG. 13 is a diagram illustrating the control of the image display direction.

  FIG. 13A shows a case where the direction of gravity does not change but the direction of gravity changes. When the direction of gravity changes from the x direction to the y direction, that is, when the terminal device 10 that has been held vertically is held horizontally, the display control unit 36c rotates the screen 20 to the left by 90 degrees. At that time, the display control unit 36c prevents the image 20 from being displayed upside down for the user, depending on whether the orientation of the user's face is upside down.

  When the direction of gravity changes from the y direction to the x direction, that is, when the terminal device 10 that has been held horizontally is held vertically, the display control unit 36c rotates the screen 20 90 degrees to the right. Let Also in this case, the display control unit 36c prevents the image 20 from being displayed upside down for the user depending on whether the orientation of the user's face is upside down.

  On the other hand, FIG. 13B shows a case where both the face direction and the gravity direction change. When the orientation of the face changes from portrait to landscape and the direction of gravity changes from x to y, that is, when the terminal device 10 held in portrait orientation is held in landscape orientation, the display control unit 36 c maintains the display direction of the screen 20.

  Further, when the orientation of the face changes from landscape to portrait and the direction of gravity changes from y to x, that is, when the terminal device 10 held in landscape orientation is held in portrait orientation, The display control unit 36 c maintains the display direction of the screen 20.

  Next, a processing procedure of image display control processing according to the second embodiment of the present invention will be described. 14 and 15 are flowcharts showing an example of the processing procedure of the image display control process according to the second embodiment of the present invention.

  First, the imaging unit 32 of the terminal device 30 illustrated in FIG. 12 activates the camera (step S601) and captures the user's video (step S602). Then, the orientation determining unit 15a performs a process for determining the orientation of the user's face from the user's video as described with reference to FIGS. 2 to 5 (step S603). The processing procedure of this processing is the same as that described with reference to FIGS.

  And the memory | storage part 35 memorize | stores the information of the face direction determined by step S603 as the face direction data 35b (step S604). This information is referred to when the determination process in step S612 is performed.

  Thereafter, the gravity direction detection unit 31 detects the direction of gravity (step S605). Then, the change detection unit 36a determines whether or not the gravity detection process in step S605 is the first detection process (step S606).

  If the gravity direction detection process is the first detection process (YES in step S606), the storage unit 35 stores the gravity direction information obtained as a result of the detection as gravity direction data 35bc (step S606). S607). Thereafter, the processing after step S605 is executed again.

  When the gravity direction detection process is not the first detection process (NO in step S606), the change detection unit 36a includes the gravity direction information stored in the storage unit 35 as the gravity direction data 35c, and in step S605. The gravity direction information obtained as a result of the detection is compared, and it is determined whether or not the gravity detection direction has changed (step S608).

  In step S608, if the gravity detection direction has not changed (NO in step S608), the processes in and after step S605 are executed again. In step S608, when the gravity detection direction changes (YES in step S608), the storage unit 35 stores the gravity direction detected in step S605 as gravity direction data 35c (step S609).

  Thereafter, as illustrated in FIG. 15, the imaging unit 32 captures a video of the user (step S610). Subsequently, the orientation determination unit 36b performs processing for determining the orientation of the user's face from the user's video as described with reference to FIGS. 2 to 5 (step S611). The procedure of this determination process is the same as that described with reference to FIGS.

  Then, the orientation determination unit 36b compares the facial orientation information stored in the storage unit 35 as the facial orientation data 35b with the facial orientation information obtained as a result of the determination in step S611, and determines the terminal device 30. As a reference, it is determined whether or not the face orientation has changed (step S612).

  When the face orientation has changed (YES in step S612), the display control unit 36c changes the image display orientation according to the direction of gravity, as described with reference to FIG. (Step S613). Then, the storage unit 35 stores the face orientation information determined in step S611 as face orientation data 35b (step S614). This information is referred to when the determination process in step S612 is performed next.

  On the other hand, if the face orientation has not changed in step S612 (NO in step S612), the display control unit 36c maintains the image display orientation as described with reference to FIG. (Step S615).

  After step S614, after step S615, or when NO is determined in step S202 of FIG. 8, step S302 of FIG. 9, step S402 of FIG. 10, or step S502 of FIG. The determination unit 36b determines whether or not an instruction to end the image display control process has been received from the user (step S616). This end instruction is, for example, an instruction not to change the orientation of the image.

  When an instruction to end the image display control process is received from the user (YES in step S616), the imaging unit 32 stops the camera (step S617). Thereafter, the image display control process ends.

  In step S616, if an instruction to end the image display control process has not been received from the user (NO in step S616), the process from step S605 shown in FIG. 14 is executed again.

  The present invention is not limited to the above-described embodiments, and various modifications and corrections can be made without departing from the scope of the present invention.

DESCRIPTION OF SYMBOLS 10 ... Terminal device, 11 ... Imaging part, 12 ... Touch panel part, 13 ... Communication processing part, 14 ... Memory | storage part, 14a ... Image data, 14b ... Face direction data, 15 ... Control part, 15a ... Direction determination part, 15b ... Display control unit, 20 ... image, 30 ... terminal device, 31 ... gravity direction detection unit, 32 ... imaging unit, 33 ... touch panel unit, 34 ... communication processing unit, 35 ... storage unit, 35a ... image data, 35b ... face orientation Data, 35c ... Gravity direction data, 36 ... Control unit, 36a ... Change detection unit, 36b ... Direction determination unit, 36c ... Display control unit.

Claims (13)

A display unit;
A gravity direction detector for detecting the direction of gravity in a predetermined coordinate system of the terminal device;
When a plane parallel to the display unit is an xy plane, a change detection unit that detects a change in the detection direction of gravity parallel to the xy plane;
An imaging unit that captures an image of the user's face;
An orientation determination unit that determines the orientation of the user's face with reference to the terminal device, based on the video of the user's face captured by the imaging unit;
A display control unit for controlling the display direction of the image displayed on the display unit based on the detection direction of the gravity in the detection result by the change detection unit and the determination result by the orientation determination unit;
A terminal device comprising:   The display direction of the image is changed when the determination result by the orientation determination unit determines that the orientation of the user's face is changing with reference to the terminal device. The terminal device described.   The display direction of the image is not changed when the determination result by the orientation determination unit determines that the orientation of the user's face has not changed with reference to the terminal device. The terminal device described.   The terminal device according to claim 1, wherein when the orientation determination unit does not successfully recognize the user's face, the operation of the imaging unit is stopped.   When the orientation determination unit does not succeed in recognizing the user's face, the display control unit determines the display direction of the image based on the detection result by the change detection unit and the determination result by the orientation determination unit. The terminal device according to claim 1, wherein the terminal device is not controlled.   The terminal device according to claim 1, wherein the imaging unit does not capture an image of the user's face when the gravity detection direction has not changed.   The terminal device according to claim 1, wherein the direction of display of the image is not changed when the gravity detection direction is not changed.   The terminal device according to claim 1, wherein the orientation determination unit determines the orientation of the face based on the positions of both eyes of the user.   The terminal device according to claim 1, wherein the orientation determination unit determines the orientation of the face based on positions of white eyes and black eyes of the user.   The terminal device according to claim 1, wherein the orientation determination unit determines the orientation of the face based on a shape of glasses worn by the user.   The terminal device according to claim 1, wherein the orientation determination unit performs face recognition of the user and determines the orientation of the face based on a result of the face recognition. A gravity direction detecting step for detecting a direction of gravity in a predetermined coordinate system of the terminal device;
When a plane parallel to the display unit of the terminal device is an xy plane, a change detection unit that detects a change in the detection direction of gravity parallel to the xy plane;
A change detection step for detecting a change in the detection direction of gravity parallel to the display unit;
An imaging step for capturing an image of the user's face;
A direction determination step for determining the orientation of the user's face with reference to the terminal device, based on the video of the user's face captured in the imaging step;
A display control step of controlling the display direction of the image displayed on the display unit based on the detection direction of the gravity in the detection result of the change detection step and the determination result of the orientation determination step;
A display control method comprising:   A computer program for causing a computer to execute the display control method according to claim 12.

Images