JP2014082648A - Electronic apparatus, image correction method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve visibility when display contents are confirmed during movement.SOLUTION: An electronic apparatus 10 includes: a display unit 14 for displaying an image; and a control unit 11 for calculating a motion vector of eyeballs on the basis of the positions of eyeballs of a user, which are to be acquired by imaging, when the movement of the user to browse an image is detected, and correcting the image to cancel the calculated motion vector. When an inclination exceeding a predetermined angle is detected, the control unit 11 calculates the positions of eyeballs of a user to be imaged by image recognition and calculates the motion vector of eyeballs on the basis of a previous calculation value and this time calculation value. When the calculated motion vector exceeds a specified value, the control unit corrects an image to be displayed in the display unit 14 so as to cancel the motion vector.

Description

  The present invention relates to an electronic device, an image correction method, and a program.

  Conventionally, a wristwatch (running watch) has been developed that allows the user to determine the current position and route of travel by continuously measuring the amount and direction of movement while walking and running. Has been. According to this wristwatch, the course can be set according to the desired calorie, the moving speed, the number of steps, etc. according to the setting of the distance and time desired by the user. Also, with the spread of smartphones, the same functions as the wristwatch described above can be realized by mounting a dedicated application program on the smartphone and carrying it.

  By the way, there are cases where the user wants to check the running time, lap, heart rate, and the like with the above-described electronic device such as a wristwatch or a smartphone while traveling. In this case, the display becomes difficult to see due to shaking of the head, hand, or arm, and visibility is deteriorated. For this reason, conventionally, for example, Patent Literature 1 proposes a line-of-sight control display device capable of controlling the display of a screen based on the line of sight measured with a camera image. Further, Patent Document 2 proposes a line-of-sight display device that analyzes and displays the movement of the line of sight of an observer placed on a moving object.

Japanese Patent Publication No. 5-83247 JP 2007-213469 A

  According to the technique disclosed in Patent Document 1, the motion detection unit detects the motion of a moving object such as a vehicle, and the display unit displays the motion of the detected moving object on the image captured by the photographing unit. Then, it is identified and displayed together with the movement of the line of sight of the subject obtained by analyzing the movement of the eyeball by the calculation means. Therefore, it is possible to distinguish and observe the movement of the line of sight based on the intention of the observer and the influence of the movement of the moving object. However, since the captured image is not directly corrected based on the movement of the line of sight, when the display content is confirmed during movement, the deterioration of visibility due to the influence of the head and hand shake cannot be denied.

  Further, according to the technique disclosed in Patent Document 2, the display area is selected and controlled depending on where the line of sight is moving out of the N display areas, so that the line-of-sight detection accuracy is low. However, accurate display control is possible. However, as in Patent Document 1, since an image captured based on the movement of the line of sight is not directly corrected, when confirming the display content during movement, the deterioration of visibility due to the influence of the head and camera shake cannot be denied.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide an electronic device, an image correction method, and a program that improve visibility when confirming display contents while moving. There is to do.

  In order to solve the above-described problems, an electronic device according to the present invention calculates a motion vector of the eyeball from a display unit that displays an image, and a position of the user's eyeball acquired by photographing, and the calculated motion vector And a control unit that corrects the image so as to cancel out the image.

  The present invention is an image correction method for an electronic device having at least a display unit, and the user's eyeball acquired by photographing when detecting an operation of a user viewing an image displayed on the display unit A step of calculating a motion vector of the eyeball from the position, and a step of correcting the image displayed on the display unit so as to cancel the calculated motion vector.

  The program of the present invention cancels the calculated motion vector, and a process of calculating a motion vector of the eyeball from a position of the user's eyeball acquired by photographing on a computer of an electronic device having at least a display unit And a process of correcting the image displayed on the display unit.

  ADVANTAGE OF THE INVENTION According to this invention, the electronic device, the image correction method, and program which aimed at the improvement of the visibility at the time of confirming a display content during a movement can be provided.

It is a block diagram which shows the internal structure of the electronic device which concerns on embodiment of this invention. It is a flowchart which shows operation | movement of the electronic device which concerns on embodiment of this invention. It is an image figure of the prescription | regulation angle used as the criterion for a gaze detection. It is a figure which shows an example of a captured image and a display image. It is a figure which shows the example of a screen display at the time of applying to a soft keyboard. It is a figure which shows the example of application of the electronic device which concerns on embodiment of this invention.

  Hereinafter, an embodiment for carrying out the present invention (hereinafter simply referred to as an embodiment) will be described in detail with reference to the accompanying drawings.

(Configuration of the embodiment)
The electronic device 10 according to the present embodiment is, for example, a portable display device, and includes a control unit 11, a storage unit 12, an operation unit 13, a display unit 14, an acceleration sensor 15, an imaging unit 16, and the like. And a communication unit 17. These blocks are commonly connected via a system bus 18 having a plurality of lines for address, data, and control.

  The control unit 11 is acquired by photographing when it detects an operation in which the user browses an image by sequentially reading and executing the program according to the present embodiment that is allocated and stored in the program area of the storage unit 12. A function for correcting the image so as to cancel the calculated motion vector is executed by calculating a motion vector of the eyeball from the position of the user's eyeball.

  Specifically, the control unit 11 calculates the position of the user's eyeball imaged by the imaging unit 16 by image recognition when the acceleration sensor 15 used as the inclination detection unit detects an inclination exceeding a predetermined angle. Then, the motion vector of the eyeball is calculated from the value calculated at the previous sampling and the value calculated at the current sampling, and the motion vector is canceled when the calculated motion vector exceeds a prescribed value (Prescribed value). The image displayed on the display unit 14 is corrected. Further, the control unit 11 corrects the image when the calculated motion vector exceeds the first specified value P1, and when the calculated motion vector exceeds the second specified value P2 larger than the first specified value P1, You may change the size of the characters included. That is, the first specified value P1 <the second specified value P2.

  A program area and a work area are allocated to the storage unit 12, and the program according to the present embodiment is assigned to the program area, and the work area is moved based on, for example, changes in the user's eyeball position and eyeball position. Various data generated in the course of program execution such as vectors are temporarily stored. The program according to the present embodiment is acquired by photographing when at least the computer (control unit 11) of the electronic device 10 including the display unit 14 detects an operation of the user viewing the image displayed on the display unit 14. A process of calculating the eyeball motion vector from the position of the user's eyeball and a process of correcting the image displayed on the display unit 14 so as to cancel the calculated motion vector are executed. For example, an SDRAM (Synchronous Dynamic Random Access Memory) is mounted as the storage element.

  The operation unit 13 includes, for example, a key switch, and is output as a command or data to the control unit 11 when operated by a user. The display unit 14 is a display monitor having a display element such as an LCD (Liquid Crystal Device) or an organic EL (Organic Electro-Luminescence), and displays an image corrected by the control unit 11 here. The operation unit 13 may be a touch panel formed integrally with the display unit 14.

  The acceleration sensor 15 is an electronic component to which, for example, a strain gauge is bonded as a detection element, and has a built-in bridge circuit that detects the posture (tilt) of the mounted device. The acceleration sensor 15 functions as an inclination detection unit that detects the inclination α of the electronic device 10 and outputs it to the control unit 11. The control unit 11 is in a state where the user is browsing the display screen of the electronic device 10 (display unit 14) when the acceleration sensor 15 detects an inclination exceeding a predetermined angle of, for example, 30 degrees to 50 degrees. The position of the user's eyeball imaged by the imaging unit 16 is calculated by image recognition, and an eyeball motion vector is calculated from the value calculated at the previous sampling and the value calculated at the current sampling. .

  The imaging unit 16 is a camera including a lens, an image sensor such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), a driver IC, and the like. The imaging unit 16 converts the subject light incident through the lens into an electrical signal by the image sensor, amplifies it by the driver IC, and outputs it to the control unit 11.

  The communication unit 17 obtains data such as a map by accessing a server via a network (not shown) with a protocol according to TCP / IP (Transmission Control Protocol / Internet Protocol), or with other electronic devices, for example. Exchange data between them. In addition, for example, it includes a communication adapter compliant with a short-range wireless communication standard such as Wi-Fi (registered trademark) or BlueTooth (registered trademark), and establishes a link with other electronic devices to perform short-range wireless communication. Exchange data.

(Operation of the embodiment)
Hereinafter, the operation of the electronic apparatus 10 according to the present embodiment shown in FIG. 1 will be described with reference to the flowchart of FIG.

  In the electronic device 10, first, the control unit 11 acquires data from the acceleration sensor 15, calculates the gravity direction (vertical direction), and calculates the inclination α of the housing with respect to the gravity direction (step S101). Next, the control unit 11 determines whether or not the calculated inclination is within a predetermined angle range (step S102). That is, here, when the inclination of the casing of the electronic device 10 is in the range of 30 degrees to 50 degrees, that is, when 30 ° ≦ α ≦ 50 °, the display of the electronic device 10 is held by the user's hand. It is determined that the screen (display unit 14) is being browsed.

  If it is determined that the user is viewing the display screen (step S102 “Yes”), the control unit 11 activates the imaging unit 16 and the eyeball of the user photographed by the imaging unit 16 is displayed. Image data is acquired every predetermined sampling period (step S103). If it is determined that the user is not viewing the display screen (“No” at step S102), the process returns to the inclination calculation process at step S101.

  Subsequently, the control unit 11 calculates the position of the user's eyeball from the acquired image by image analysis (step S104). Note that the image analysis method is disclosed in detail in, for example, Patent Document 1. The position of the eyeball may be one eye, but here, the positions of the left and right eyeballs are detected. The accuracy can be increased by using two eyeball positions. On the other hand, if the eyeball cannot be detected from the image (step S105 “No”), the user is not browsing the display screen, and the process returns to the tilt calculation process in step S101. Next, a motion vector between the eyeball position calculated at the previous sampling and the eye position calculated at the current sampling is calculated (step S106). A method of calculating a motion vector is also disclosed in detail in Patent Document 1.

  Subsequently, the control unit 11 compares the calculated motion vector with the first specified value P1 (step S107). Here, the first specified value P1 is a threshold that requires a shift process of the display image. The left-eye motion vector amount is VL, and the right-eye motion vector amount is VR. The amount of the motion vector is represented by, for example, the number of moved pixels. As an example, the first specified value P1 is 20 pixels, and the second specified value is P2 is 200 pixels. Then, a value obtained by adding the left-eye motion vector amount VL and the right-eye motion vector amount VR is compared with a specified value. When the motion vector is equal to or greater than the first specified value P1, that is, when (VL + VR) ≦ 20 (step S107 “Yes”), the calculated motion vector is compared with the second specified value P2 (step S107). Step S108). The specified value P2 refers to a threshold value that requires the character size to be increased if the display image has a large amount of shake and includes characters.

  Here, when the motion vector amount is equal to or larger than the second specified value P2, that is, when 200 ≦ (VL + VR) (step S108 “Yes”), the control unit 11 determines that the size of the character included in the display image is If it is smaller, a process for improving readability is performed by converting it to a prescribed large character size (step S109). The large character size may be two stages, for example, twice, but the character size may be changed in multiple stages according to the motion vector amount. Subsequently, the control unit 11 performs image correction by shift processing of the image displayed on the display unit 14 (step S110).

  If the motion vector is equal to or smaller than the first specified value P1 in the first specified value P1 determination process in step S107 (step S107 “No”), or the second specified value P2 in the specified value 2 determination process in step S108. In the following case (step S108 “No”), since it is determined that there is little blur between the position of the user's eyes and the display position of the image, the image shift processing is omitted, and the tilt calculation processing in step S101 is performed. Return.

  Images of image correction by shift processing are shown in FIGS. 4 (a), (b), (c), and (d). 4A is a head subject image captured one frame before by the imaging unit 16, and FIG. 4B is a head subject image showing a correction result by the image shift processing. As shown in FIG. 4B, as is apparent from the eye position of the head subject image A (shown with a dotted line) captured one frame before, the amount of motion vector (deviation amount) in the direction opposite to the motion vector. It can be seen that only B) is translated. In this case, the amount of motion vector is (VL + VR) / 2.

  FIG. 4C is a character image before correction displayed on the display unit 14, and FIG. 4D is a character image after correction displayed on the display unit 14 through image shift processing. As shown in FIG. 4D, the character image is paralleled by the amount of motion vector (shift amount E) from the display position C to the corrected display position D in the direction opposite to the eyeball position shift (in the direction of the arrow in the figure). You can see that it is moving.

  Moreover, according to the electronic device 10 according to the present embodiment, when the user performs an input operation using the operation unit 13 using a soft keyboard (SIP: Software Input Panel), for example, as illustrated in FIG. By simultaneously shifting the soft keyboard image to a partial display area of the display unit 14 and correcting the input position (image correction) accordingly, it is possible to provide a user-free operation interface. At this time, the operability can be improved by making the display area of the soft keyboard larger than usual.

(Effect of embodiment)
As described above, according to the electronic device 10 according to the present embodiment, the control unit 11 is viewed from the electronic device 10 when the acceleration sensor 15 detects the state in which the user is browsing the electronic device 10. The movement amount and direction of the user's eyeball are calculated as a motion vector, and when the movement amount is equal to or greater than a specified value, the image displayed on the display unit 14 is corrected in a direction to cancel the previously calculated motion vector. In this way, for example, even if the position of the user's eyes deviates from the display position of the image due to the vibration of the moving vehicle or due to hand or arm shake, the image is corrected and displayed accordingly. Therefore, an effect that visibility is improved is obtained. In this case, since the image is corrected by parallel movement (shift correction), the processing is easy, and the burden on the control unit 11 is reduced.

  Note that the electronic device 1 according to the present embodiment can be applied to a wristwatch (running watch) whose appearance is shown in FIG. The running watch stores measurement data such as measurement time, travel lap, travel distance, or heart rate, and is used for applications that support planned training. Of course, the present invention is not limited to this running watch, and the position of the user's eyes and the display position of the image may be shifted due to the influence of vibration of a moving vehicle such as a smartphone or a mobile phone, or due to hand or arm shake. It can be applied to all portable electronic devices that become hot.

  Further, in the electronic device 10 according to the present embodiment, the image correction has been described as being performed by the electronic device 10 itself. However, for example, as illustrated in FIG. The image correction apparatus 20 (server) to be corrected may correct the image and provide a service to the electronic device 10 such as the running watch 10a or the smartphone 10b. At this time, if the image correction device 20 (server) has an SNS (Social Networking Service) function, a terminal such as the smartphone 10b participating in the SNS connected via the network 30 such as an IP (Internet Protocol) network, It is also possible to write a comment for a person who is running or jogging with the running watch 10a.

  Note that the image correction method of the electronic device according to the present embodiment is an image correction method applied to the electronic device 10 having at least the display unit 14 as shown in FIG. Then, as shown in FIG. 2, for example, as shown in FIG. 2, the image correction method detects the user's eyeball acquired by photographing when an operation of browsing the image displayed on the display unit is detected. Calculating a motion vector of the eyeball from the position (S101 to S106), and correcting the image displayed on the display unit so as to cancel the calculated motion vector (S107 to S110).

  According to the image correction method of the electronic device according to the present embodiment, for example, even when the position of the eye is shifted from the display position due to the influence of vibration or the like when the wristwatch or the smartphone is viewed during running or jogging, it is adjusted accordingly. Since the image is corrected, an effect that the display is easy to see is obtained.

  Further, for example, as shown in FIG. 1, the program according to the present embodiment is applied to the computer (control unit 11) of the electronic device 10 having at least the display unit 14. Then, for example, as shown in FIG. 2, the program detects the movement of the user viewing the image displayed on the display unit 14 from the position of the user's eyeball acquired by photographing. Processing for calculating an eyeball motion vector (S101 to S106) and processing for correcting the image displayed on the display unit so as to cancel the calculated motion vector (S107 to S110) are executed.

  According to the program according to the present embodiment, the control unit 11 detects that the user has browsed the display content of the front electronic device 10 by sequentially reading and executing the program stored in the program area of the storage unit 12. Sometimes, the amount of movement of the user's eyes (eyeball) viewed from the electronic device 10 is calculated as a motion vector, and when the motion vector is equal to or greater than a specified value, the display content is shifted in a direction to cancel the motion vector. By performing the process of performing the above, it is possible to provide the electronic device 10 that is improved in visibility and operability.

  As mentioned above, although preferred embodiment of this invention was explained in full detail, it cannot be overemphasized that the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiments. Further, it is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

The claims appended at the time of filing this application will be appended below.
[Appendix]
[Claim 1]
A display for displaying an image;
A control unit that calculates a motion vector of the eyeball from a position of the eyeball of the user acquired by photographing, and corrects the image so as to cancel the calculated motion vector;
An electronic device comprising:
[Claim 2]
An inclination detection unit and an imaging unit;
The controller is
When an inclination exceeding a predetermined angle is detected by the inclination detection unit, the position of the user's eyeball imaged by the imaging unit is calculated by image recognition, and from the previously calculated value and the currently calculated value, The motion vector of the eyeball is calculated, and when the calculated motion vector exceeds a predetermined value, an image displayed on the display unit is corrected so as to cancel the motion vector. Electronic equipment.
[Claim 3]
The controller is
When the calculated motion vector exceeds a first specified value, the image is corrected. When the calculated motion vector exceeds a second specified value larger than the first specified value, the size of characters included in the image is set. The electronic device according to claim 1, wherein the electronic device is changed.
[Claim 4]
An operation unit for inputting data by operating a software keyboard displayed in at least a partial area of the display unit;
The controller is
The electronic apparatus according to claim 1, wherein when an operation by the operation unit is detected, a display area of the software keyboard is expanded.
[Claim 5]
An image correction method for an electronic device having at least a display unit,
Calculating a motion vector of the eyeball from a position of the user's eyeball acquired by photographing when detecting an operation of a user viewing an image displayed on the display unit;
And a step of correcting the image displayed on the display unit so as to cancel the calculated motion vector.
[Claim 6]
In an electronic device computer having at least a display unit,
A process of calculating a motion vector of the eyeball from the position of the eyeball of the user acquired by photographing;
Processing for correcting the image displayed on the display unit so as to cancel the calculated motion vector;
A program characterized by having executed.

DESCRIPTION OF SYMBOLS 10 ... Electronic device, 10a wristwatch (running watch), 10b ... Smartphone, 11 ... Control part, 12 ... Memory | storage part, 13 ... Operation part, 14 ... Display part, 15 ... Accelerometer, 16 ... Imaging part, 17 ... Communication part , 18 ... system bus, 20 ... image correction device (server), 30 ... network.

Claims (6)

A display for displaying an image;
A control unit that calculates a motion vector of the eyeball from a position of the eyeball of the user acquired by photographing, and corrects the image so as to cancel the calculated motion vector;
An electronic device comprising: An inclination detection unit and an imaging unit;
The controller is
When an inclination exceeding a predetermined angle is detected by the inclination detection unit, the position of the user's eyeball imaged by the imaging unit is calculated by image recognition, and from the previously calculated value and the currently calculated value, The motion vector of the eyeball is calculated, and when the calculated motion vector exceeds a predetermined value, an image displayed on the display unit is corrected so as to cancel the motion vector. Electronic equipment. The controller is
When the calculated motion vector exceeds a first specified value, the image is corrected. When the calculated motion vector exceeds a second specified value larger than the first specified value, the size of characters included in the image is set. The electronic device according to claim 1, wherein the electronic device is changed. An operation unit for inputting data by operating a software keyboard displayed in at least a partial area of the display unit;
The controller is
The electronic apparatus according to claim 1, wherein when an operation by the operation unit is detected, a display area of the software keyboard is expanded. An image correction method for an electronic device having at least a display unit,
Calculating a motion vector of the eyeball from a position of the user's eyeball acquired by photographing when detecting an operation of a user viewing an image displayed on the display unit;
And a step of correcting the image displayed on the display unit so as to cancel the calculated motion vector. In an electronic device computer having at least a display unit,
A process of calculating a motion vector of the eyeball from the position of the eyeball of the user acquired by photographing;
Processing for correcting the image displayed on the display unit so as to cancel the calculated motion vector;
A program characterized by having executed.

Images