JP6221526B2 - Portable electronic device and display control program - Google Patents

Description

  The present invention relates to a portable electronic device and a display control program.

  In general, for example, a portable electronic device such as a smartphone may be provided with a function of automatically switching between a portrait screen display and a landscape screen display. That is, the portable electronic device detects the rotation of the device itself using, for example, an acceleration sensor, and rotates the screen display in accordance with the rotation of the device when the rotation is detected. For this reason, the user can switch between the display of the vertically long screen and the display of the horizontally long screen only by rotating the portable electronic device according to the display content of the screen.

  In recent years, a function has been realized in which the rotation of the screen display is suppressed when the portable electronic device rotates with a change in the user's posture, such as when the user lies down while holding the portable electronic device. Specifically, when the portable electronic device detects the rotation of the device itself, the portable electronic device activates a camera provided on the same plane as the display that displays the screen to capture the user's face, and the image obtained by capturing the image The direction of the user's face is detected. When the orientation of the user's face matches the orientation of the device itself, the portable electronic device maintains the displayed screen without rotating the screen display. Thereby, even when the portable electronic device is rotated, if the user does not intend, the screen display is not unnecessarily rotated, and the convenience of the portable electronic device is improved.

JP 2011-41067 A

  However, the above-described technology that suppresses unnecessary screen display using a camera has a problem in that power consumption increases because the camera is activated each time the portable electronic device rotates. That is, in the above technique, when the rotation of the portable electronic device is detected, the camera is activated every time and the user's face is imaged, thereby detecting the orientation of the user's face. Therefore, every time rotation of the portable electronic device is detected, the camera consumes power for imaging.

  The disclosed technology has been made in view of the above points, and an object thereof is to provide a portable electronic device and a display control program capable of suppressing an increase in power consumption.

In one aspect, the portable electronic device disclosed in the present application is orthogonal to a display device that displays a screen and an imaging device that is provided on the same plane as the display device and that captures the face of the user facing the display device. An acceleration sensor that acquires acceleration data related to the three axes, and a processor that is connected to the display device, the imaging device, and the acceleration sensor and acquires acceleration data acquired by the acceleration sensor. It is detected that the device has rotated around one axis perpendicular to the display device among the three axes, and the variation amount of acceleration data regarding the one axis at the detected rotation is less than a predetermined threshold value. whether judges, when the amount of change of the acceleration data for the 1-axis is determined to be less than a predetermined threshold value, prior to Rotate the screen being displayed on the display device, when the amount of change of the acceleration data for the 1-axis is determined to be equal to or greater than a predetermined threshold value, the activate the imaging device, is captured by the imaging device started to give It is determined whether or not to rotate the screen being displayed on the display device using the obtained image.

  According to one aspect of the portable electronic device and the display control program disclosed in the present application, there is an effect that an increase in power consumption can be suppressed.

FIG. 1 is a diagram illustrating an appearance of a portable electronic device according to an embodiment. FIG. 2 is a block diagram illustrating a configuration of the portable electronic device according to the embodiment. FIG. 3 is a block diagram illustrating functions of the processor according to the embodiment. FIG. 4 is a flowchart showing display control processing according to the embodiment. FIG. 5 is a diagram illustrating a specific example of acceleration data according to an embodiment. FIG. 6 is a diagram illustrating a specific example of the direction of the eyes and the screen according to the embodiment.

  Hereinafter, an embodiment of a portable electronic device and a display control program disclosed in the present application will be described in detail with reference to the drawings. In addition, this invention is not limited by this embodiment.

  FIG. 1 is a diagram illustrating an appearance of a portable electronic device 100 according to an embodiment. This portable electronic device is an electronic device capable of displaying a screen, such as a smartphone or a tablet terminal. A portable electronic device 100 illustrated in FIG. 1 includes a display 110 and a camera 120.

  The display 110 is a substantially rectangular display device including a liquid crystal panel, for example. The display 110 displays various screens when the mobile electronic device 100 is powered on. Specifically, the display 110 displays a vertically long screen whose short direction matches the user's left and right direction, or displays a horizontally long screen whose long direction matches the user's left and right direction. Therefore, for example, the user can display a vertically long screen during normal operation, for example, and can display a horizontally long screen when viewing a moving image, for example.

  The camera 120 is an imaging device provided on the same plane as the display 110. Since the camera 120 is provided on the same plane as the display 110, when the user is viewing the display 110, the user's face facing the display 110 can be captured.

  In addition, the mobile electronic device 100 illustrated in FIG. 1 includes an acceleration sensor that detects the movement of the mobile electronic device 100. This acceleration sensor acquires triaxial acceleration data orthogonal to each other. In other words, the acceleration sensor obtains acceleration data for the three axes of the x axis parallel to the short direction of the display 110, the y axis parallel to the longitudinal direction of the display 110, and the z axis perpendicular to the display 110, respectively. get. Then, as will be described later, the mobile electronic device 100 rotates the screen displayed on the display 110 based on the acceleration data acquired by the acceleration sensor and the image captured by the camera 120.

  FIG. 2 is a block diagram showing a configuration of the portable electronic device 100 according to the embodiment. A portable electronic device 100 illustrated in FIG. 2 includes a display 110, a camera 120, an acceleration sensor 130, a wireless interface (hereinafter abbreviated as “wireless I / F”) 140, a processor 150, and a memory 160.

  The display 110 displays screen information generated by the processor 150. Further, the display 110 rotates the screen information displayed on the vertically long screen into a horizontally long screen according to the instruction of the processor 150, or rotates the screen information displayed on the horizontally long screen into a vertically long screen.

  The camera 120 executes imaging processing according to application processing while a predetermined application is in operation. Further, the camera 120 is stopped when not used by an application and is in a state of not consuming power. When instructed by the processor 150, the camera 120 is activated regardless of whether the application is running or not, and images the face of the user facing the display 110.

  The acceleration sensor 130 detects acceleration of three axes orthogonal to each other, acquires acceleration data, and outputs the acquired acceleration data to the processor 150. Since the acceleration sensor 130 detects acceleration for each of the three axes, for example, when the portable electronic device 100 moves in the x-axis direction, acceleration data related to the x-axis varies. For example, when the portable electronic device 100 rotates around the z axis, acceleration data related to the x axis and the y axis varies.

  The wireless I / F 140 performs wireless communication with another wireless communication device via an antenna. That is, the wireless I / F 140 transmits a transmission signal output from the processor 150 via an antenna, and outputs a reception signal received via the antenna to the processor 150.

  The processor 150 includes, for example, a central processing unit (CPU) or a field programmable gate array (FPGA), and controls the operation of the portable electronic device 100. Specifically, the processor 150 acquires acceleration data from the acceleration sensor 130, and detects that the portable electronic device 100 has rotated based on the acceleration data. Then, when the processor 150 detects the rotation of the portable electronic device 100, the processor 150 determines whether or not the acceleration data related to the z-axis perpendicular to the display 110 has changed during this rotation. A screen rotation instruction is output to 110. Thereby, the display 110 rotates the screen display as the portable electronic device 100 rotates.

  On the other hand, the processor 150 outputs an activation instruction to the camera 120 when the acceleration data regarding the z-axis fluctuates when the portable electronic device 100 rotates, and the user's eyes are based on the image obtained by the camera 120. It is determined whether or not the direction matches the screen direction. Then, the processor 150 outputs a screen rotation instruction to the display 110 when the user's eye direction and the screen direction do not match. Thereby, the display 110 rotates the screen display as the portable electronic device 100 rotates. Further, the processor 150 does not rotate the screen display on the display 110 even when the portable electronic device 100 rotates when the user's eye direction matches the screen direction. Note that specific processing of the processor 150 will be described in detail later.

  The memory 160 includes a volatile memory such as a RAM (Random Access Memory), and temporarily stores data used for processing by the processor 150.

  FIG. 3 is a block diagram illustrating functions of the processor 150 according to the embodiment. The processor 150 illustrated in FIG. 3 includes a rotation detection unit 151, a threshold comparison unit 152, a camera control unit 153, an image processing unit 154, and a display control unit 155.

  The rotation detection unit 151 acquires triaxial acceleration data orthogonal to each other from the acceleration sensor 130. Then, the rotation detection unit 151 detects that the portable electronic device 100 has rotated by comparing the acquired acceleration data with acceleration data of a predetermined time. Specifically, the rotation detection unit 151 causes the mobile electronic device 100 to be displayed on the display 110 when acceleration data about the x-axis and the y-axis fluctuates by a predetermined threshold or more compared to acceleration data for a predetermined time. It detects that it rotated around the z axis perpendicular to it.

  Here, when changing the screen viewed by the user from the portrait screen to the landscape screen or from the landscape screen to the portrait screen, the user rotates the portable electronic device 100 around the z axis. That is, the user who views the vertically long screen holds the portable electronic device 100 so that the short side direction of the display 110 and the left and right direction of the user coincide with each other. The electronic device 100 is rotated approximately 90 degrees around the z axis. As a result, the longitudinal direction of the display 110 coincides with the left and right direction of the user, and the user views the landscape screen. As described above, when the mobile electronic device 100 rotates around the z-axis, there is a possibility that the screen viewed by the user may be changed, so the rotation detection unit 151 mainly includes the x-axis and the parallel to the display 110. Rotation about the z-axis is detected using acceleration data about the y-axis.

  When the rotation detection unit 151 detects that the portable electronic device 100 has rotated around the z axis, the threshold comparison unit 152 compares the amount of change in acceleration data related to the z axis with a predetermined threshold. That is, when the rotation of the mobile electronic device 100 is detected, the threshold comparison unit 152 determines whether or not the acceleration data regarding the x axis and the y axis has changed and at the same time the acceleration data regarding the z axis has also changed.

  As a result, when the fluctuation amount of the acceleration data regarding the z-axis is less than the predetermined threshold value, the threshold value comparison unit 152 instructs the display control unit 155 to rotate the display screen of the display 110. On the other hand, when the fluctuation amount of the acceleration data related to the z-axis is equal to or greater than a predetermined threshold, the threshold comparison unit 152 instructs the camera control unit 153 to activate the camera 120.

  Here, when the portable electronic device 100 is rotated with the intention of switching the screen viewed by the user between the portrait screen and the landscape screen, it is considered that the mobile electronic device 100 hardly moves in the z-axis direction. . That is, when the user rotates the portable electronic device 100, the variation amount of the acceleration data regarding the z-axis is small. On the other hand, when the rotation of the portable electronic device 100 is detected with a change in the user's posture, for example, when the user lies down while holding the portable electronic device 100, the movement of the portable electronic device 100 becomes irregular. , And is considered to move relatively large in the z-axis direction. That is, when the portable electronic device 100 rotates with a change in the user's posture, the amount of change in acceleration data related to the z axis is large.

  As described above, when the rotation of the mobile electronic device 100 is detected, it is determined whether the user has rotated the mobile electronic device 100 with the intention of rotating the screen display, based on the amount of change in the acceleration data related to the z-axis. Is possible. Therefore, the threshold value comparison unit 152 compares the variation amount of the acceleration data related to the z axis with a predetermined threshold value, and if the variation amount of the acceleration data related to the z axis is small, the display control unit 152 does not start the camera 120. 155 is instructed to immediately rotate the screen. As a result, when the user rotates the portable electronic device 100 with the intention of rotating the screen display, the activation of the camera 120 is omitted, and an increase in power consumption can be suppressed. In addition, since processing such as imaging by the camera 120 is not performed between the time when the rotation of the portable electronic device 100 is detected and the time when the screen display is actually rotated, the delay time until the screen display is rotated. Can be shortened.

  The camera control unit 153 receives an instruction from the threshold comparison unit 152 and activates the camera 120. The camera control unit 153 instructs the camera 120 to capture an image of the face of the user who is viewing the display 110.

  The image processing unit 154 acquires an image obtained by being captured by the camera 120, performs image processing, and detects the position of the user's eyes. Specifically, the image processing unit 154 acquires an image of the user's face output from the camera 120, and detects the position of the user's eyes from the image. Further, the image processing unit 154 determines the positions of the eyes and the corners of the eyes where the positions are detected in the image, and determines the direction of the user's eyes. That is, the image processing unit 154 determines the direction connecting the user's eyes and the corners of the eyes as the eye direction. Therefore, the determined eye direction is equivalent to the left-right direction of the user. Then, the image processing unit 154 notifies the display control unit 155 of the determined eye direction.

  When the display control unit 155 receives an instruction from the threshold comparison unit 152, the display control unit 155 instructs the display 110 to rotate the screen immediately and rotates the screen display on the display 110. On the other hand, when the eye direction is notified from the image processing unit 154, the display control unit 155 determines whether the notified eye direction matches the screen direction currently displayed on the display 110. Determine. The display control unit 155 maintains the direction of the currently displayed screen when the directions match, and causes the screen direction to match the eye direction when the directions do not match. The screen display of the display 110 is rotated.

  Next, the display control process by the portable electronic device 100 configured as described above will be specifically described with reference to the flowchart shown in FIG. FIG. 4 is a flowchart mainly showing the operation of the processor 150.

  In the rotation detection unit 151 of the processor 150, acceleration data for the three axes is always acquired from the acceleration sensor 130 (step S101), and the acceleration data for the x-axis and the y-axis is the acceleration data for a predetermined time ago. To be compared. Then, based on the comparison result, it is determined whether or not the portable electronic device 100 has rotated around the z axis (step S102). That is, it is determined whether or not the acceleration data about the x-axis and the y-axis has fluctuated more than a predetermined threshold value compared with a predetermined time ago.

  As a result of the determination, if the acceleration data regarding the x-axis and the y-axis does not fluctuate more than a predetermined threshold value, it is determined that the portable electronic device 100 is not rotating (No in step S102), and the acceleration sensor 130 continues. Acceleration data acquisition and rotation detection are performed. On the other hand, when the acceleration data regarding the x-axis and the y-axis fluctuates by a predetermined threshold value or more, it is determined that the mobile electronic device 100 has rotated (Yes in step S102), and the fact is notified to the threshold value comparison unit 152. .

  Then, whether or not the amount of change in the acceleration data with respect to the z-axis is less than the predetermined threshold during the period when the acceleration comparison data with respect to the x-axis and the y-axis has fluctuated by a predetermined threshold value or more is synchronized with the threshold value comparison unit 152. Determination is made (step S103).

  As described above, when the user rotates the mobile electronic device 100, the amount of change in the acceleration data related to the z-axis is small, while the rotation of the mobile electronic device 100 is detected as the user's posture changes. Has a large fluctuation amount of acceleration data with respect to the z-axis. As a specific example, in the upper diagram of FIG. 5, the acceleration data related to the x-axis indicated by the broken line and the acceleration data related to the y-axis indicated by the alternate long and short dash line vary greatly during the period T, whereas the solid line The acceleration data related to the z-axis indicated by (2) has a small amount of fluctuation during the period T. This is because the upper diagram in FIG. 5 shows the change in acceleration data when the user rotates the portable electronic device 100 with the intention of rotating the screen.

  On the other hand, in the lower diagram of FIG. 5, the acceleration data related to the x-axis indicated by the broken line and the acceleration data related to the y-axis indicated by the alternate long and short dash line greatly fluctuate during the period T, and the acceleration data related to the z-axis indicated by the solid line also varies. There is a relatively large fluctuation during T. This is because the lower diagram in FIG. 5 shows the change in the acceleration data when the user changes the posture while holding the portable electronic device 100.

  As described above, when the acceleration data regarding the x-axis and the y-axis largely fluctuates, if the variation amount of the acceleration data regarding the z-axis is small, the user rotates the portable electronic device 100 with the intention of rotating the screen. Judgment can be made. Therefore, if the threshold value comparison unit 152 determines that the variation amount of the acceleration data with respect to the z axis is less than the predetermined threshold value (Yes in step S103), the display control unit 155 is immediately instructed to rotate the screen. Is issued. Then, when the display control unit 155 gives a screen rotation instruction to the display 110, the screen display of the display 110 is rotated (step S104).

  As a result, when the user rotates the portable electronic device 100 with the intention of rotating the screen display, the activation of the camera 120 is omitted, and an increase in power consumption can be suppressed. In addition, since processing such as imaging by the camera 120 is not performed between the time when the rotation of the portable electronic device 100 is detected and the time when the screen display is actually rotated, the delay time until the screen display is rotated. Can be shortened.

  On the other hand, if the acceleration data related to the x-axis and the y-axis change greatly, and the amount of change in the acceleration data related to the z-axis is large, the user's posture may change simultaneously with the rotation of the portable electronic device 100. Therefore, when the threshold value comparison unit 152 determines that the variation amount of the acceleration data with respect to the z axis is equal to or greater than the predetermined threshold value (No in step S103), the camera control unit 153 is instructed to activate the camera 120. Is issued. Then, the camera control unit 153 activates the camera 120 (step S105), and instructs to capture the face of the user who views the display 110.

  An image obtained by being captured by the camera 120 is acquired by the image processing unit 154, and a predetermined image process is executed to detect the position of the user's eyes in the image (step S106). Further, the image processing unit 154 determines the positions of the user's eyes and corners in the image, and determines the direction of the eyes connecting the user's eyes and corners. The determined user's eye direction is notified to the display control unit 155. As described above, the direction of the user's eyes is the same as the left-right direction of the user.

  Then, the display control unit 155 determines whether or not the user's eye direction notified from the image processing unit 154 matches the screen direction being displayed on the display 110 (step S107). Here, the screen direction is the left-right direction when an image is properly displayed. That is, when the direction of the user's eyes and the direction of the screen match, the image is displayed in an appropriate direction for the user, and the user can easily view the image.

  Specifically, as shown in the upper diagram of FIG. 6, when the user's eye direction 201 and the left-right direction 202 of the image coincide with each other, the image is displayed on the display 110 of the portable electronic device 100. The displayed image is displayed in a proper orientation for the user. On the other hand, as shown in the lower diagram of FIG. 6, when the user's eye direction 203 and the left-right direction 204 of the image do not match, the image displayed on the display 110 of the portable electronic device 100 is appropriate for the user. Will not be displayed in the correct orientation.

  Therefore, when the display control unit 155 determines that the user's eye direction matches the screen direction (Yes in step S107), the screen currently displayed on the display 110 does not rotate and the screen is not rotated. The direction of is maintained. If the display control unit 155 determines that the user's eye direction and the screen direction do not match (No in step S107), the display 110 is instructed to rotate the screen 110, and the display 110 The screen display is rotated (step S104).

  As described above, when the variation amount of the acceleration data regarding the z-axis is large, the camera 120 is activated, and it is determined whether to rotate the screen display by comparing the direction of the user's eyes with the direction of the screen. . For this reason, for example, when there is a possibility that the portable electronic device 100 is rotated in accordance with a change in the posture of the user, the screen display can be rotated in accordance with the direction of the user's eyes.

  As described above, according to the present embodiment, the rotation of the portable electronic device is detected based on the acceleration data related to two axes among the acceleration data related to three axes, and the fluctuation amount of the acceleration data related to the remaining one axis at the time of this rotation. If is small, the screen display is immediately rotated. Also, if the amount of variation in acceleration data for all three axes is large, the camera is activated and a decision is made as to whether or not to rotate the screen display by comparing the direction of the user's eyes with the direction of the displayed screen. To do. For this reason, when the user rotates the portable electronic device with the intention of rotating the screen display, the screen display can be quickly rotated without starting the camera, and an increase in power consumption can be suppressed. . In addition, the delay time from when the portable electronic device rotates to when the screen display rotates can be shortened.

  In the above-described embodiment, when the acceleration data regarding the x-axis and the y-axis greatly fluctuates, if the variation of the acceleration data regarding the z-axis is less than a predetermined threshold, the user intends to rotate the screen display. It was determined that the portable electronic device 100 was rotated. However, the determination as to whether or not the user has rotated the portable electronic device 100 with the intention of rotating the screen display may not be based only on the change in the acceleration data related to the z axis. That is, for example, when acceleration data about the x-axis and the y-axis greatly fluctuate, the user intends to rotate the screen display by using information on whether the fluctuation is abrupt or slow. It can also be determined whether or not the electronic device 100 has been rotated.

  Specifically, as shown in the upper diagram of FIG. 5, the acceleration data relating to the x-axis indicated by the broken line and the acceleration data relating to the y-axis indicated by the alternate long and short dash line change rapidly during the period T. On the other hand, as shown in the lower diagram of FIG. 5, the acceleration data related to the x-axis indicated by the broken line and the acceleration data related to the y-axis indicated by the alternate long and short dash line vary slowly during the period T. Thus, when the portable electronic device 100 rotates with a change in the user's posture, the acceleration data fluctuates relatively slowly. For this reason, in addition to the determination using the acceleration data regarding the z-axis, the user intends to rotate the screen display by determining whether or not the acceleration data regarding the x-axis and the y-axis has fluctuated more than a predetermined level. You may make it determine whether the portable electronic device 100 was rotated.

  In the above-described embodiment, the rotation of the portable electronic device 100 is detected by comparing current and past acceleration data regarding the x-axis and the y-axis. It may be used. That is, for example, the rotation of the portable electronic device 100 may be detected by comparing the average value of acceleration data at a plurality of past time points with the current acceleration data. By doing so, it is possible to prevent erroneous detection of the rotation of the portable electronic device 100 even when the acceleration data of a predetermined time ago instantaneously shows a peculiar value.

  Furthermore, the method for detecting the rotation of the portable electronic device 100 is not limited to the method using the acceleration data acquired by the acceleration sensor 130. That is, for example, when the portable electronic device 100 includes a gyro sensor that detects its own inclination, the rotation of the portable electronic device 100 may be detected using the gyro sensor. Even in this case, if acceleration data relating to the z-axis perpendicular to the display 110 greatly changes during the detected rotation, the screen of the display 110 is immediately rotated without starting the camera 120.

  Note that the display control processing of the portable electronic device 100 described in the above embodiment can be described as a display control program that can be executed by a computer. In this case, the display control program can be stored in a computer-readable recording medium and introduced into the computer. Examples of the computer-readable recording medium include a portable recording medium such as a CD-ROM, a DVD disk, and a USB memory, and a semiconductor memory such as a flash memory.

110 Display 120 Camera 130 Acceleration sensor 140 Wireless I / F
DESCRIPTION OF SYMBOLS 150 Processor 151 Rotation detection part 152 Threshold comparison part 153 Camera control part 154 Image processing part 155 Display control part 160 Memory

Claims (4)

A display device for displaying the screen;
An imaging device that is provided on the same plane as the display device and captures the face of the user facing the display device;
An acceleration sensor that acquires acceleration data about three axes orthogonal to each other;
A processor connected to the display device, the imaging device, and the acceleration sensor to acquire acceleration data acquired by the acceleration sensor;
The processor is
Detecting that the device rotates about one axis perpendicular to the display device among the three axes;
Determining whether or not the variation amount of the acceleration data related to the one axis at the time of the detected rotation is less than a predetermined threshold;
When it is determined that the fluctuation amount of the acceleration data related to the one axis is less than a predetermined threshold, the screen being displayed on the display device is rotated,
When it is determined that the fluctuation amount of the acceleration data related to the one axis is equal to or greater than a predetermined threshold, the imaging device is activated,
A portable electronic device that determines whether or not to rotate a screen that is being displayed on the display device, using an image obtained by imaging by an activated imaging device. The process to detect is
Compare current acceleration data for two axes other than the one axis with acceleration data for a predetermined time,
Determine whether the amount of change in acceleration data from a predetermined time before to the present is greater than or equal to a predetermined threshold,
If the amount of change of the acceleration data is determined to be equal to or greater than a predetermined threshold value, the portable electronic device of claim 1, wherein the local device around the 1 axis and detects that it has rotated. The processor is
It is further determined whether or not the variation amount of the acceleration data related to the two axes other than the one axis at the time of the detected rotation is a predetermined level or more,
The process of rotating is
When it is determined that the fluctuation amount of the acceleration data related to the one axis is less than a predetermined threshold, and it is determined that the acceleration data related to two axes other than the one axis is greater than or equal to a predetermined level, the display is displayed on the display device. portable electronic device according to claim 1, wherein the screen during rotation. A display device that displays a screen; an imaging device that is provided on the same plane as the display device and that captures the face of the user facing the display device; and an acceleration sensor that acquires acceleration data about three axes orthogonal to each other On the computer,
Detecting that the computer rotates about one of the three axes perpendicular to the display device;
Determining whether or not the variation amount of the acceleration data related to the one axis at the time of the detected rotation is less than a predetermined threshold;
When it is determined that the fluctuation amount of the acceleration data related to the one axis is less than a predetermined threshold, the screen being displayed on the display device is rotated,
When it is determined that the fluctuation amount of the acceleration data related to the one axis is equal to or greater than a predetermined threshold, the imaging device is activated,
A display control program for executing a process of determining whether or not to rotate a screen being displayed on the display device, using an image obtained by imaging by an activated imaging device.

Images