JP2021078073A - Display apparatus - Google Patents

Abstract

To provide a technology that can suppress an unintended change of an observation range, and also can give feeling close to that at the time of observation with the naked eye (feeling without or with little discomfort) to a user.SOLUTION: A display apparatus according to the present invention includes: imaging means; display means that displays an image captured by the imaging means; attitude detection means that detects an attitude of the display apparatus; line-of-sight detection means that detects a line of sight of a user toward the display means; and control means that does not control an imaging direction of the imaging means on the basis of the line of sight, which has been detected by the line-of-sight detection means, when an amount of change in the attitude detected by the attitude detection means is smaller than a predetermined amount, but controls the imaging direction on the basis of the line of sight, when the amount of change in the attitude is larger than the predetermined amount.SELECTED DRAWING: Figure 1

Description

The present invention relates to a display device, particularly to electronic binoculars.

Electronic binoculars are display devices that include a camera and a display that is placed in front of the user's eyes when in use, and display images taken by the camera on the display in real time. By viewing the image (telephoto image) displayed on the display of the electronic binoculars, the user can observe the distance as if looking through the binoculars. Some electronic binoculars are wearable and can be attached to and detached from the head, such as a head-mounted display.

Patent Document 1 discloses a technique of detecting a user's line-of-sight position and controlling the shooting range so that the line-of-sight position becomes the center position of the shooting range.

Japanese Unexamined Patent Publication No. 2015-149552

When using electronic binoculars, it is desirable to provide the user with a field of view similar to visual observation. In the case of observation with the naked eye, the field of view expands in the direction of the line of sight, regardless of the movement of the head or the direction (direction) of the face. However, in the case of general electronic binoculars, the field of view (the subject range (angle of view) of the image displayed on the display; the observation range) changes depending on the movement of the head regardless of the direction of the line of sight. Since the observation range changes due to unintended movement of the head, the user feels uncomfortable and it becomes difficult to immerse himself in the observation. Especially when displaying an image with a high telephoto magnification, the observation range changes greatly due to a slight shake of the head.

Further, when the technique disclosed in Patent Document 1 is used so that the imaging range is the observation range, the line-of-sight position is the center position of the imaging range, so that the entire observation range can be set by the user. Even when you want to look around, the observation range changes depending on your line of sight.

As described above, in the prior art, an unintended change in the observation range occurs, or a sensation (uncomfortable feeling) far from the observation with the naked eye is given to the user. Such problems arise with both wearable electronic binoculars and non-wearable electronic binoculars.

An object of the present invention is to provide a technique capable of suppressing an unintended change in the observation range and giving the user a sensation close to that of observation with the naked eye (a sensation of no discomfort or less sensation). ..

The first aspect of the present invention is
It ’s a display device,
Imaging means and
A display means for displaying an image captured by the image pickup means, and a display means.
A posture detecting means for detecting the posture of the display device and
A line-of-sight detecting means for detecting the line of sight of the user with respect to the display means,
When the amount of change in the posture detected by the posture detecting means is smaller than a predetermined amount, the imaging direction of the imaging means is not controlled based on the line of sight detected by the line-of-sight detecting means. A control means for controlling the imaging direction based on the line of sight when the amount of change in posture is larger than the predetermined amount.
It is a display device characterized by having.

A second aspect of the present invention is
A method for controlling a display device having an image pickup means and a display means for displaying an image captured by the image pickup means.
A posture detection step for detecting the posture of the display device, and
A line-of-sight detection step for detecting the user's line of sight with respect to the display means,
When the amount of change in the posture detected in the posture detection step is smaller than a predetermined amount, the imaging direction of the imaging means is not controlled based on the line of sight detected in the line-of-sight detection step. A control step that controls the imaging direction based on the line of sight when the amount of change in posture is larger than the predetermined amount.
It is a control method characterized by having.

A third aspect of the present invention is a program for making a computer function as each means of the above-mentioned display device.

According to the present invention, it is possible to suppress an unintended change in the observation range and to give the user a sensation close to that of observation with the naked eye (a sensation of no discomfort or less).

It is a flowchart which shows the processing flow which concerns on Example 1. FIG. It is external drawing of the electronic binoculars which concerns on Examples 1 and 2. It is a block diagram of the electronic binoculars which concerns on Examples 1 and 2. It is a schematic diagram which shows the state of the display device and the user which concerns on Examples 1 and 2. It is a schematic diagram which shows the relationship between the subject distance and the rotation amount which concerns on Examples 1 and 2. It is a flowchart which shows the processing flow which concerns on Example 2.

<Example 1>
Hereinafter, Example 1 of the present invention will be described. An example of applying the present invention to electronic binoculars that can be attached to and detached from the head (wearable) such as a head-mounted display will be described, but the display device to which the present invention can be applied is not limited to wearable electronic binoculars. .. For example, the present invention may be applied to non-wearable electronic binoculars, or to other wearable devices that can be attached to and detached from the head (for example, head-mounted displays such as VR goggles, AR glasses, MR glasses, and smart glasses). The present invention may be applied. The present invention may be applied to a display device (such as VR goggles) that covers both eyes so that the user cannot see the surroundings with the naked eye (the user can see the image with both eyes). The present invention may be applied to a display device that covers only one eye so that the user can see the image with one eye and the surroundings with the naked eye with the other eye. Although details will be described later, according to the present invention, suitable observation is possible even in a situation where the user cannot see the surroundings with the naked eye. Therefore, the present invention can be suitably applied to a display device covering both eyes.

2 (A) and 2 (B) are external views showing the hardware configuration of the electronic binoculars 10 according to the first embodiment. FIG. 2A is a front perspective view of the electronic binoculars 10 viewed from the front side, and FIG. 2B is a rear perspective view of the electronic binoculars 10 viewed from the rear direction. The electronic binoculars 10 are eyeglass-type electronic binoculars and are removable from the head. Specifically, the electronic binoculars 10 are fixed (mounted) to the head by sandwiching the head between the left and right temples 100 of the electronic binoculars 10. In addition to the temple 100, the electronic binoculars 10 include a camera 101, displays 102, 103, a panning unit 104, a tilt unit 105, a gyro sensor 106, a line-of-sight detection unit 107, and an operating member 108.

The camera 101 is an imaging unit, and as shown by an arrow in FIG. 2A, it can rotate individually in the pan direction and the tilt direction. That is, the imaging direction of the camera 101 can be individually changed to the pan direction and the tilt direction. The panning unit 104 rotates the camera 101 in the pan direction (the direction in which the camera 101 is tilted to the left or right with respect to the electronic binoculars 10) by driving an actuator built in the panning unit 104. The tilt unit 105 rotates the camera 101 in the tilt direction (the direction in which the camera 101 is tilted up and down with respect to the electronic binoculars 10) by driving an actuator built in the tilt unit 105. The changing direction and mechanism for changing the imaging direction are not particularly limited.

Further, the camera 101 is configured so that its focal length can be changed. In the first embodiment, the focal length of the camera 101 is set in two stages of 100 mm and 400 mm (both are focal lengths converted to 35 mm full size) according to the operation (user operation) performed by the user on the electronic binoculars 10. Suppose it can be switched. The operation member 108 is a member (button, switch, etc.) that accepts user operations, and accepts user operations that instruct, for example, to change (switch) the focal length of the camera 101, or to turn on / off the power of the electronic binoculars 10. The number and range of focal lengths that can be set are not particularly limited. The focal length may be seamlessly changeable within a predetermined range.

Further, the camera 101 has an autofocus function and is configured to automatically focus on a subject included in the imaging range. The distance to the subject that is in focus (focused) is uniquely determined by the stop position of the focusing lens (not shown) that is driven in focus adjustment (autofocus). Therefore, by storing information (table or function) indicating the relationship between the stopping position of the focusing lens and the subject distance in the electronic binoculars 10 in advance, the electronic binoculars 10 can use the information from the stopping position of the focusing lens. , The subject distance can be detected. The camera 101 also has a function of detecting the subject distance by such a method. The method of detecting the subject distance is not particularly limited.

The displays 102 and 103 are display units that display images captured by the camera 101. When the user wears the electronic binoculars 10, the display 102 is arranged in front of the user's right eye, and the display 103 is arranged in front of the user's left eye. Therefore, the user sees the image displayed on the display 102 with the right eye and the image displayed on the display 103 with the left eye.

The gyro sensor 106 is a posture detection unit that detects the posture of the electronic binoculars 10, and can also detect a change in the posture of the electronic binoculars 10 (presence or absence of change, direction, size, etc.). When the user wears the electronic binoculars 10, the posture of the electronic binoculars 10 corresponds to the posture of the user's head. Therefore, the gyro sensor 106 can detect the posture and movement (swing) of the head.

The line-of-sight detection unit (line-of-sight detection sensor) 107 detects the user's (relative) line of sight with respect to the displays 102 and 103. The line-of-sight detection unit 107 can also detect a change in the line of sight (presence or absence of change, direction, magnitude, etc.).

FIG. 3 is a block diagram showing the configuration of the electronic binoculars 10. The CPU 201 controls each part of the electronic binoculars 10. The CPU 201 includes a camera 101, displays 102, 103,
It is connected to a gyro sensor 106, a line-of-sight detection unit 107, an operating member 108, a camera rotating unit 202, and the like. The CPU 201 processes information from each part of the electronic binoculars 10 and controls the operation of each part according to the processing result. The camera rotating unit 202 includes a panning unit 104 and a tilt unit 105, and rotates the camera 101 in the pan direction and the tilt direction in response to an instruction from the CPU 201.

FIG. 1 is a flowchart showing a processing flow (processing flow of the electronic binoculars 10) according to the first embodiment. The processing flow of FIG. 1 is realized, for example, by the CPU 201 expanding a program stored in a ROM (not shown) into a RAM (not shown) and executing the program. When a user operation instructing the power of the electronic binoculars 10 to be turned on is performed, the electronic binoculars 10 are activated, and a process of displaying the image captured by the camera 101 on the displays (displays 102 and 103) in real time is started. As a result, the user can start observing the subject by looking at the image captured by the camera 101 and displayed on the display. Then, the processing flow of FIG. 1 starts. The initial value of the focal length of the camera 101 (the focal length immediately after the power is turned on) is not particularly limited, but it is preferably a wide-angle focal length so that the user can easily find the observation target. In the first embodiment, it is assumed that the focal length is controlled to 100 mm immediately after the power is turned on. Although not shown in FIG. 1, the focal length is switched from one of 100 mm and 400 mm to the other each time a user operation for instructing a change (switch) of the focal length is performed during the processing flow of FIG.

FIG. 4A shows the initial direction of the image pickup direction of the camera 101 (the image pickup direction immediately after the power is turned on; the reference direction). As shown in FIG. 4A, the reference direction is a direction in which the optical axis of the camera 101 is parallel to the front direction of the electronic binoculars 10 (the direction in which the face of the user wearing the electronic binoculars 10 is facing). FIG. 4A is drawn from the viewpoint of looking overhead of the user so that the pan direction component in the imaging direction can be seen, but the tilt direction component in the imaging direction is also the same. Although only the control for changing the imaging direction in the pan direction will be described below, the imaging direction is also changed in the tilt direction by the same control method as the control method for changing the imaging direction in the pan direction.

In S101 of FIG. 3, the camera 101 performs autofocus (AF) control and automatic exposure (AE) control based on the captured image.

In S102, the camera 101 detects (acquires) the subject distance L from the result of the AF control of S101, and the CPU 201 determines (calculates) the rotation amount A1 of the camera 101 based on the detected subject distance L.

FIG. 5 shows the relationship between the subject distance L and the rotation amount A1. The asterisk in FIG. 5 indicates an observation object existing in front of the user. Normally, in visual observation, the user turns his face to the subject and captures the subject in the center of the field of view. Here, consider the case where the imaging direction of the camera 101 is the reference direction. In this case, depending on the mounting position of the camera 101, the observation target captured in the center of the user's field of view with the naked eye may not be displayed in the center of the display, and the user may feel uncomfortable. The rotation amount A1 determined in S102 is a rotation amount for reducing such a feeling of strangeness. Here, the direction of rotation toward the left is the positive direction, and the direction of rotation toward the right is the negative direction. In FIG. 5, the camera 101 is attached at a position deviated from the center of the head by a distance a to the right. Therefore, by rotating the camera 101 with the rotation amount A1 = φ1 = arctan (a / L1), the observation target (observation target existing in front of the user) existing at the subject distance L1 can be displayed in the center of the display. .. Similarly, by rotating the camera 101 with the rotation amount A1 = φ2 = arctan (a / L2), the observation object (observation object existing in front of the user) existing at the subject distance L2 can be displayed in the center of the display. .. As described above, in S102, the larger the rotation amount A1 as the subject distance is shorter, is determined from the subject distance L based on the relational expression “A1 = arctan (a / L)”. According to this relational expression, when the subject distance L is relatively long, the rotation amount A1 becomes approximately 0 (zero). Therefore, the amount of rotation A1 = 0 may be set when it is assumed that only a long-distance subject is observed or when the subject distance L is longer than a predetermined distance.

In S103 of FIG. 1, the CPU 201 determines whether or not the focal length of the camera 101 is longer than the threshold value (predetermined distance). If the focal length is longer than the threshold value, the process proceeds to S104, and if the focal length is equal to or less than the threshold value, the process proceeds to S109. If the focal length is equal to or greater than the threshold value, the process may proceed to S104, and if the focal length is less than the threshold value, the process may proceed to S109. In the first embodiment, since the focal lengths that can be set are 100 mm and 400 mm, in S103, it is determined whether or not the focal length is 400 mm, and if it is 400 mm, the process proceeds to S104, and if it is 100 mm, the process proceeds to S109. Is advanced.

In S104, the CPU 201 detects the posture of the electronic binoculars 10 (head) using the gyro sensor 106, and whether or not the posture of the electronic binoculars 10 (head) changes by a change amount larger than the threshold value (predetermined amount). To judge. If the posture changes with a change amount larger than the threshold value, the process proceeds to S105, and if the posture change amount is less than the threshold value, the process proceeds to S109. The process may be advanced to S105 when the posture is changed by the amount of change equal to or more than the threshold value, and may be advanced to S109 when the amount of change in the posture is less than the threshold value.

In S105, the CPU 201 detects the user's line of sight during the period when the posture of the electronic binoculars 10 (head) is changing by using the line-of-sight detection unit 107, and the imaging direction of the camera 101 is controlled based on the line of sight. Switch the process as follows.

FIG. 4B shows a state in which the user's head unintentionally moves from the state of FIG. 4A. When the head moves unintentionally, the absolute line of sight is often kept fairly constant. Therefore, in FIG. 4B, the head moves to the left (positive direction) by an angle θ1, but the absolute line-of-sight direction is the original reference direction (reference direction before the head moves; FIG. 4 (A) reference direction) remains. That is, as the movement of the line of sight relative to the display, a movement of an angle θ1 occurs in the right direction (negative direction; the direction opposite to the movement of the head). When the head moves, the electronic binoculars 10 also move integrally with the head. Therefore, in FIG. 4B, the electronic binoculars 10 also move to the left by an angle θ1. In the state of FIG. 4B, the CPU 201 determines that the electronic binoculars 10 (head) have moved to the left by an angle θ1 using the gyro sensor 106. Further, the CPU 201 uses the line-of-sight detection unit 107 to determine that the line of sight (relative line of sight) has moved by an angle θ1 in the right direction (the direction opposite to the movement of the electronic binoculars 10 (head)).

In this way, if there is a change in the line of sight (relative line of sight) opposite to the direction of change in the posture of the electronic binoculars 10 (head), the user does not want to change the viewing direction (viewing direction) and the head. Is determined to have moved unintentionally, and processing proceeds to S106. Then, in S106, the CPU 201 determines (calculates) the amount of rotation A2 for rotating the camera 101 so as to maintain the angle of view immediately before detecting the movement of the head. In other words, the CPU 201 determines the amount of rotation A2 for rotating the camera 101 so as to cancel the movement of the head. In the state of FIG. 4B, the amount of rotation A2 for rotating the camera 101 to the right by an angle θ1 is determined. At this time, the rotation amount A2 may be determined from the detection result of the gyro sensor 106 (angle θ1 in the left direction) or may be determined from the detection result (angle θ1 in the right direction) of the line-of-sight detection unit 107. , May be determined from both of those detection results. The angle (magnitude) detected by the gyro sensor 106 may be different from the angle (magnitude) detected by the line-of-sight detection unit 107.

Although an example of determining the rotation amount A2 in which the change in the imaging direction of the camera 101 due to the change in the posture of the electronic binoculars 10 (head) is set to 0 (zero) in S106 has been described, the rotation amount A2 determined in S106 has been described. Is not limited to this. It suffices if the change in the imaging direction due to the change in posture becomes small,
The change in the imaging direction does not have to be completely 0 (zero).

FIG. 4C shows a state in which the user intentionally moves his / her head from the state of FIG. 4A in order to change the observation target. When the head is intentionally moved, the absolute line of sight often moves in the same direction as the head moves. Therefore, in FIG. 4C, the head and the absolute line of sight move integrally. Specifically, the head is moving to the left by an angle θ1, and the absolute line of sight is also moving to the left by an angle θ1. That is, there is no movement of the line of sight relative to the display. In the state of FIG. 4C, the CPU 201 determines that the electronic binoculars 10 (head) have moved to the left by an angle θ1 using the gyro sensor 106. Further, the CPU 201 uses the line-of-sight detection unit 107 to determine that the line of sight (relative line of sight) is not moving.

In this way, when only the change in the posture of the electronic binoculars 10 (head) is detected, it is determined that the user intentionally moves the head in order to change the viewing direction according to the movement of the head. Then, the process proceeds to S107. Then, in S107, the CPU 201 determines the rotation amount A2 = 0 that does not rotate the camera 101. In other words, the CPU 201 determines the amount of rotation A2 that maintains the imaging direction of the camera 101 in the line-of-sight direction (direction of the line-of-sight).

FIG. 4 (D) shows a state in which the user intentionally moves his / her head from the state of FIG. 4 (A) in order to follow (follow) the fast-moving observation object (moving object). When following a fast-moving observation object, the absolute line of sight moves in the same direction as the head moves, with a movement amount larger than the head movement amount (including the head movement amount). Often. Therefore, in FIG. 4D, the head is moving to the left by an angle θ1, and the absolute line of sight is moving to the left at an angle θ1 + θ2. That is, as the movement of the line of sight relative to the display, a movement of an angle θ2 occurs in the left direction. In the state of FIG. 4D, the CPU 201 determines that the electronic binoculars 10 (head) have moved to the left by an angle θ1 using the gyro sensor 106. Further, the CPU 201 uses the line-of-sight detection unit 107 to determine that the line of sight (relative line of sight) has moved to the left by an angle θ2.

In this way, if there is a change in the line of sight (relative line of sight) in the same direction as the direction of change in the posture of the electronic binoculars 10 (head), the user intends the head in order to significantly change the viewing direction. It is determined that the object has been moved, and the process proceeds to S108. Then, in S108, the CPU 201 determines the amount of rotation A2 that changes the imaging direction of the camera 101 in the line-of-sight direction.

The rotation amount A2 determined in S108 is not limited to the rotation amount that changes the imaging direction in the line-of-sight direction. It is sufficient that the change in the imaging direction due to the change in posture becomes large, and the imaging direction does not have to match the line-of-sight direction.

In S109, the CPU 201 determines the rotation amount A2 = 0. Generally, when the focal length is short (in the case of a wide angle; corresponding to NO in S103), the observation range (displayed on the display) due to an unintended posture change (unintentional movement of the head) of the electronic binoculars 10. Changes in the subject range (angle of view) of the image are inconspicuous. When the amount of change in the posture of the electronic binoculars 10 (head) is small (corresponding to NO in S104), the change in the observation range due to an unintended change in the posture of the electronic binoculars 10 is inconspicuous, and the user can see the entire observation range. You may move your gaze to look around. Therefore, in the first embodiment, the process proceeds to S109 in those cases. At this time, the line-of-sight detection is not performed, or even if the line-of-sight detection is performed, the detection result is not used. In the case where a long focal length (telephoto focal length; focal length in which the change in the observation range due to an unintended change in the posture of the electronic binoculars 10 is conspicuous) is assumed, the processing can proceed to S104 regardless of the focal length. In addition, the processing of S103 may be omitted.

In S110, the CPU 201 determines (calculates) the final rotation amount A = A1 + A2 of the camera 101 from the rotation amount A1 determined in S102 and the rotation amount A2 determined in any of S106 to S109. When it is decided in advance that the rotation amount A1 = 0, for example, when observing only a long-distance subject, the rotation amount A = A2 is determined regardless of the subject distance L. , S102 may be omitted.

In S111, the CPU 201 uses the camera rotating unit 202 to rotate the camera 101 with the rotation amount A determined in S110.

In S112, the CPU 201 monitors the information from the operating member 108 and determines whether or not a user operation for instructing the power off of the electronic binoculars 10 has been performed. The processes of S101 to S111 are repeated until the power is turned off, and when the power is turned off, the processing flow of FIG. 1 ends and the electronic binoculars 10 stop (the electronic binoculars 10 are turned off). ..

As described above, according to the first embodiment, when the amount of change in the detected posture (posture of the electronic binoculars 10 (head)) is smaller than a predetermined amount, the camera 101 is based on the detected line of sight. The imaging direction of is not controlled. Then, when the detected change in posture is larger than a predetermined amount, the imaging direction is controlled based on the detected line of sight. As a result, it is possible to suppress an unintended change in the observation range, and it is possible to give the user a feeling close to that of observation with the naked eye (a feeling of no discomfort or little feeling).

In addition to the control of the imaging direction in the processing flow of FIG. 1, the imaging direction may always be controlled based on the shaking of the electronic binoculars 10, such as camera shake correction. Specifically, the imaging direction may always be controlled so as to suppress the shaking of the imaging direction due to the shaking of the electronic binoculars 10. The shaking of the imaging direction (imaging position) due to the electronic binoculars 10 swinging back and forth, up, down, left and right without changing the posture of the electronic binoculars 10 may be suppressed, or the shaking of the imaging direction due to the shaking of the posture of the electronic binoculars 10 may be suppressed. It may be done, or both of those shakes may be suppressed. The shaking of the electronic binoculars 10 may be detected by the gyro sensor 106, or may be detected by a member (sensor) different from the gyro sensor 106. The method of detecting the shaking of the electronic binoculars 10 is not particularly limited. For example, shaking (micro-vibration) of a predetermined frequency may be detected, and shaking in the imaging direction due to the shaking may be suppressed. Based on the vibration signal (detection result) output from the vibration sensor, if the value of the vibration signal is less than the threshold value, it is judged as vibration (camera shake), and if the value of the vibration signal is more than the threshold value, it is judged as a posture change that is not vibration. May be done. Vibration and non-vibration posture change can be detected separately by using various techniques described in Japanese Patent Application Laid-Open No. 2015-075697.

Here, a configuration is conceivable in which a part of the captured image is displayed on the display as a display range. In that case, the position of the display range may be controlled instead of controlling the imaging direction based on the shaking of the electronic binoculars 10. Specifically, the display is performed so as to suppress the shaking of the display direction (imaging direction corresponding to the display range; for example, the direction from the camera 101 toward the subject position corresponding to the center position of the display range) due to the shaking of the electronic binoculars 10. The position of the range may be controlled.

Similarly, when displaying a part of the captured image as a display range on the display, the position of the display range is controlled instead of controlling the imaging direction based on the subject distance L (rotation with the rotation amount A1). May be done.

Further, the control method for changing the imaging direction in the pan direction may be different from the control method for changing the imaging direction in the tilt direction. For example, the imaging direction may not be changed to one of the pan direction and the tilt direction, and the imaging direction may be changed to the other of the pan direction and the tilt direction according to the processing flow of FIG.

<Example 2>
Hereinafter, Example 2 of the present invention will be described. In the first embodiment, when the amount of change in the posture of the electronic binoculars 10 (head) is larger than a predetermined amount, the comparison result between the change direction of the posture of the electronic binoculars 10 and the change direction of the line of sight and the presence or absence of the change in the line of sight are determined. An example of determining the rotation amount A2 by switching the processing accordingly has been described. In the second embodiment, when the change amount of the posture of the electronic binoculars 10 is larger than a predetermined amount, the rotation amount A2 is determined according to the detected line of sight without considering the posture of the electronic binoculars 10. .. In the following, points different from the first embodiment (configuration, processing, etc.) will be described in detail, and the same points as in the first embodiment will be omitted as appropriate.

FIG. 6 is a flowchart showing a processing flow (processing flow of the electronic binoculars 10) according to the second embodiment, and is a modification of the flowchart of FIG. In FIG. 6, the same processing as in FIG. 1 is designated by the same reference numerals as in FIG. When the posture changes with a change amount larger than the threshold value (YES in S104), the process proceeds to S201, and then the process proceeds to S201 and then to S110. In S201, the CPU 201 detects the user's line of sight using the line-of-sight detection unit 107, and determines the rotation amount A2 that aligns the imaging direction with the direction of the detected line of sight.

As described above, also in Example 2, when the amount of change in the detected posture (posture of the electronic binoculars 10 (head)) is smaller than the predetermined amount, the camera 101 is based on the detected line of sight. The imaging direction is not controlled. Then, when the detected change in posture is larger than a predetermined amount, the imaging direction is controlled based on the detected line of sight. As a result, the same effect as in Example 1 can be obtained.

It should be noted that Examples 1 and 2 (including modified examples) are merely examples, and configurations obtained by appropriately modifying or changing the configurations of Examples 1 and 2 within the scope of the gist of the present invention are also available. Included in the present invention. A configuration obtained by appropriately combining the configurations of Examples 1 and 2 is also included in the present invention.

<Other Examples>
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

10: Electronic binoculars 101: Cameras 102, 103: Display 104: Panning unit 105: Tilt unit 106: Gyro sensor 107: Line-of-sight detection unit 201: CPU 202: Camera rotation unit

Claims (12)

It ’s a display device,
Imaging means and
A display means for displaying an image captured by the image pickup means, and a display means.
A posture detecting means for detecting the posture of the display device and
A line-of-sight detecting means for detecting the line of sight of the user with respect to the display means,
When the amount of change in the posture detected by the posture detecting means is smaller than a predetermined amount, the imaging direction of the imaging means is not controlled based on the line of sight detected by the line-of-sight detecting means. A control means for controlling the imaging direction based on the line of sight when the amount of change in posture is larger than the predetermined amount.
A display device characterized by having. When the amount of change in the posture is larger than the predetermined amount, the control means changes the imaging direction due to the change in the posture if there is a change in the line of sight in a direction opposite to the change direction in the posture. The display device according to claim 1, wherein the imaging direction is controlled so as to be smaller. When the amount of change in the posture is larger than the predetermined amount, the control means changes the imaging direction due to the change in the posture if there is a change in the line of sight in the same direction as the change direction in the posture. The display device according to claim 1 or 2, wherein the imaging direction is controlled so as to increase the size. The display device according to any one of claims 1 to 3, wherein the control means aligns the imaging direction with the direction of the line of sight when the amount of change in the posture is larger than the predetermined amount. .. The focal length of the imaging means can be changed.
The control means does not control the imaging direction based on the detection result of the posture detecting means and the detection result of the line-of-sight detecting means when the focal length is shorter than a predetermined distance. The display device according to any one of claims 1 to 4. The imaging means further detects the subject distance and
The display device according to any one of claims 1 to 5, wherein the control means further controls the imaging direction based on the detected subject distance. The display device according to any one of claims 1 to 6, wherein the control means further controls the imaging direction based on the shaking of the display device. The display device according to any one of claims 1 to 7, wherein the imaging direction can be individually changed to a pan direction and a tilt direction. The display device according to claim 8, wherein the control method for changing the imaging direction in the pan direction is different from the control method for changing the imaging direction in the tilt direction. The display device according to any one of claims 1 to 9, wherein the display device is fixed to the head of the user. A method for controlling a display device having an image pickup means and a display means for displaying an image captured by the image pickup means.
A posture detection step for detecting the posture of the display device, and
A line-of-sight detection step for detecting the user's line of sight with respect to the display means,
When the amount of change in the posture detected in the posture detection step is smaller than a predetermined amount, the imaging direction of the imaging means is not controlled based on the line of sight detected in the line-of-sight detection step. A control step that controls the imaging direction based on the line of sight when the amount of change in posture is larger than the predetermined amount.
A control method characterized by having. A program for causing a computer to function as each means of the display device according to any one of claims 1 to 10.

Images