WO2015145541A1

WO2015145541A1 - Video display device

Info

Publication number: WO2015145541A1
Application number: PCT/JP2014/058074
Authority: WO
Inventors: 竜志鵜飼; 大内　敏; 川村　友人; 瀬尾　欣穂; 佑哉大木
Original assignee: 日立マクセル株式会社
Priority date: 2014-03-24
Filing date: 2014-03-24
Publication date: 2015-10-01

Abstract

This video display device (10) is provided with an information-amount modification unit (6) that modifies the amount of information per unit time in video provided by a video provision unit (1), a video display unit (8) that displays the resulting video, an image-capturing unit (3) that captures images of an object near a user, a movement detection unit (4) that detects the movement state of the user on the basis of the captured images, and an information-amount setting unit (5) that sets the modified amount of information per unit time for the video in accordance with the detected movement state of the user. If the user is moving, the information-amount setting unit (5) sets a lower modified amount of information per unit time for the video than if the user is not moving. This results in a video display device that displays video in such a manner so as not to cause discomfort in a moving user or reduce said user's attentiveness.

Description

Video display device

The present invention relates to a video display device, and more particularly to a video display device suitable for use by a user while moving.

As a video display device that is assumed to be used while the user moves, for example, there is a head-mounted display (hereinafter abbreviated as HMD) that the user wears on his / her head. The user can acquire various information while moving by visually recognizing the video displayed by the HMD.

Conventionally, video display devices such as HMDs are based on the premise that video is always displayed regardless of the user's environment and movement conditions. However, when a moving user is watching a video displayed by the video display device, a symptom (discomfort) similar to sickness may be felt. In addition, due to the presence of the video displayed by the video display device, the user may be obsessed with it and may be less alert to the surrounding environment.

With respect to this problem, for example, in Patent Document 1, there is provided an HMD that secures a field of view by providing movement detection means for detecting the movement of the user, and stopping the image output (display) when the movement of the user is detected. It is disclosed.

JP 2004-233903 A

In the method disclosed in Patent Document 1, since the image display of the HMD is stopped when the user is moving, the user cannot obtain any information from the HMD at all and the convenience is lowered. The magnitude of the above-mentioned unpleasantness and the reduction in attention given to a moving user when viewing a video depend on the type of video to be displayed, for example, the degree of motion of the video. In other words, the feeling of the user differs greatly between the moving image and the still image, and it is not always necessary to stop the display of the image.

In view of the above-described problems, an object of the present invention is to provide a video display device that displays video so as not to cause discomfort to a moving user and to reduce attention.

The present invention is an image display device that can be used while a user moves, and includes an information amount changing unit that changes an information amount per unit time for an image provided from the image providing unit, and an information amount changing unit. A video display unit that displays a video with a changed amount of information, an imaging unit that captures a subject around the user, a movement detection unit that detects a movement state of the user based on an image captured by the imaging unit, and a movement detection unit And an information amount determination unit that determines a change value of the information amount of the video per unit time with respect to the information amount change unit according to the movement state of the user detected in (1). Here, the information amount determination unit determines a change value of the information amount per unit time of the video smaller when the user is moving than when the user is not moving according to the detection result of the movement detection unit. To do.

According to the present invention, it is possible to provide an image display device that displays an image so as not to cause discomfort to a moving user and to prevent a reduction in attention.

1 is a block diagram of a video display device according to Embodiment 1. FIG. FIG. 3 is a diagram illustrating an example of an image captured by an imaging unit 3; The figure which shows an example of the relationship between a movement state and a frame rate. The figure explaining the specific example of frame rate conversion. The block diagram which shows the modification of a video display apparatus. FIG. 9 is a block diagram of a video display apparatus according to a third embodiment. The figure which shows the detection signal of the three-dimensional acceleration sensor with respect to a user's walking motion. The figure which shows the example of determination of a user's movement state. The block diagram which shows the modification of a video display apparatus. FIG. 10 is a block diagram of a video display apparatus according to a fourth embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following embodiments, an HMD (head mounted display) will be described as an example of a video display device. The user uses the HMD by wearing it on his / her head.

FIG. 1 is a block diagram of a video display apparatus according to the first embodiment. The video display device (HMD) 10 is changed by the video receiving unit 2 that receives the video provided from the video providing unit 1, the information amount changing unit 6 that changes the information amount of the received video, and the information amount changing unit 6. A video output unit 7 for outputting the video and a video display unit 8 such as a liquid crystal display for displaying the video output from the video output unit 7. Further, the video display device 10 is detected by the imaging unit 3 that images a surrounding subject, a movement detection unit 4 that detects the movement state of the user based on an image acquired by the imaging unit 3, and the movement detection unit 4. An information amount determination unit 5 that determines the amount of information per unit time of the video displayed on the video display unit 8 according to the moving state of the user. The information amount changing unit 6 changes the video received by the video receiving unit 2 to the information amount per unit time determined by the information amount determining unit 5. Although not shown in the figure, it has a mounting part for mounting the HMD 10 on the user's head.

The video providing unit 1 is a device that provides video to be displayed. For example, the video providing unit 1 provides video from a storage device disposed in an external data center. Alternatively, the video providing unit 1 may be built in the HMD 10 and may be a storage element such as a flash memory or an imaging element such as a camera.

The imaging unit 3 is attached to the HMD 10 and images a subject around the user. At that time, for example, it is attached to the front surface of the HMD 10 so that a subject in the forward direction that is the moving direction of the user can be imaged. The movement detection unit 4 detects the movement state of the user based on the image acquired by the imaging unit 3. That is, it is determined whether or not the user is moving, and when the user is moving, the speed is determined.

FIG. 2 is a diagram illustrating an example of an image captured by the imaging unit 3, and shows a captured image 21 at time T1 and a captured image 22 at time T2. The imaging unit 3 images a subject in the forward direction of the user at a predetermined time interval such as every second, and sends the captured image to the movement detection unit 4. The movement detection unit 4 recognizes an arbitrary subject existing around the user or a part thereof as a feature around the user from the captured images at each time. In FIG. 2, “tree” and “building” are recognized as

feature objects

21a, 21b, 22a, and 22b. At this time, it is not necessary to recognize the type and name of the feature (for example, a tree or a building), and the feature only needs to be composed of a specific graphic element (for example, a line or a polygon).

The movement detection unit 4 tracks the on-screen position of the feature in a plurality of continuous captured images. Then, when the characteristic object moves radially from the vicinity of the center in the screen as time passes, it is determined that the user is moving. In the example of FIG. 2, the feature 21a moves toward the lower left corner of the screen like 22a, and the feature 21b moves toward the lower right corner of the screen like 22b, so it is determined that the user is moving. it can. On the other hand, if the feature object has not moved in the four corner directions, it is determined that the user has not moved (still).

When attaching the imaging unit 3 to the HMD, in the above example, the subject in the front direction of the user is imaged, but the attachment position is not limited to this. You may attach the imaging part 3 so that it may image a user's right direction or left direction, or the both directions. However, in these cases, the determination of the movement state of the user is different. For example, when imaging the user's right direction, the feature moves from left to right in the screen, and when imaging the user's left direction, the feature moves from right to left in the screen. If it is, the user may determine that the user is moving. Further, the imaging unit 3 may be attached so as to image a subject in the user's foot direction. In this case, the movement detection unit 4 recognizes, for example, the user's limbs or road surface as a characteristic object, and determines whether or not the user is moving from those movements.

Next, when it is determined that the user is moving, the movement detection unit 4 estimates the moving speed. That is, even when moving, a case of “walking” with a low moving speed and a case of “running” with a high moving speed are distinguished. The moving speed of the user is estimated based on the moving speed of the feature in the captured image. At that time, the movement speed of the user can be estimated with higher accuracy by tracking the position of the feature in the screen at a plurality of times. For example, when the moving speed is less than 6 km / h, “walking” is determined, and when the moving speed is 6 km or more, “running” is determined.

The other moving state of the user is up and down stairs. To detect this, image recognition is used together. The movement detection unit 4 determines that the user is moving, and determines that the user is moving up and down the stairs when the captured image detects that the user is on the stairs.

Furthermore, as another movement state of the user, the user himself / herself may be driving and moving. Image recognition is also used when detecting this. The movement detection unit 4 determines that the user is driving the vehicle when detecting that the user is moving and detecting that the user is sitting in the driver's seat of the vehicle.

In the above example, the case of walking, running, up and down stairs, and driving is described as the user's moving state, but the moving state is not limited to these, and various moving states are detected by using image recognition together. It is possible.

Even when the user is in a moving state, for example, when the user is not driving and is in the passenger seat of the vehicle, or when the user is in the train, it should be distinguished from driving. . These distinctions can be determined by image recognition.

The information amount determination unit 5 receives the detection result of the movement state of the user detected by the movement detection unit 4, and determines the information amount per unit time of the video displayed on the user by the HMD 10 according to the movement state. The information amount changing unit 6 changes the video received from the video receiving unit 2 so as to have the information amount per unit time determined by the information amount determining unit 5.

Here, as a specific example of the amount of information per unit time, the video frame rate is taken up. The frame rate is the number of images per second (number of frames) (unit: fps), but here it is defined by the frequency at which the images constituting the video are updated to different images. When the movement detection unit 4 detects that the user is moving, the information amount determination unit 5 determines that the information amount per unit time, that is, the video frame rate is equal to or less than a predetermined frame rate. Then, the information amount changing unit 6 performs a conversion process of the video frame rate so as to be equal to or less than the frame rate determined by the information amount determining unit 5. The information amount determination unit 5 and the information amount change unit 6 determine the frame rate and change the frame rate as follows, for example.

FIG. 3 is a diagram illustrating an example of the relationship between the movement state and the frame rate. This relationship is stored in advance as a lookup table, and the information amount determination unit 5 determines the frame rate with reference to this table. In this table, the moving state of the user is classified into “at rest”, “walking”, “running”, “driving”, and the like, and the maximum frame rate of the output video in each state is defined. For example, the frame rate is not limited when stationary, but is limited to 10 fps or less during walking and 1 fps or less during traveling. In operation, it is set to 0 fps (still image). Thus, the frame rate is lowered when the user is moving compared to when the user is stationary. When the user is moving, the frame rate is lowered as the moving speed of the user increases. Especially when the user is driving, the still image is not updated.

The setting of the frame rate shown in FIG. 3 is merely an example, and can be appropriately set according to the moving state. For example, even in the same walking state, different frame rates may be set for walking at 3 km / h and walking at 4 km / h. Further, the frame rate may be continuously changed according to the speed.

Even when the moving speed is high, if you are in a vehicle driven by another person or if you are on a train, the frame rate limit is relaxed and the user feels uncomfortable (sickness ) Should not be given.

FIG. 4 is a diagram for explaining a specific example of frame rate conversion. (A) is an image before frame rate conversion, and (b) and (c) are images after frame rate conversion.

The information amount changing unit 6 extracts a frame corresponding to the converted frame rate from the video frame sequence (a) received by the video receiving unit 2, and generates a new video frame sequence (b) or (c). . The frames extracted from the received video are preferably at the same time, but when the times do not match, it is possible to convert the video to a smooth motion by selecting the frame that minimizes the time lag.

First, the case where the video 30 of (a) frame rate = 30 fps is converted into the video 31 of (b) frame rate = 10 fps will be described. The frame sequence before conversion is represented by numbers s = 0, 1,..., 6 and the frame sequence after conversion is represented by numbers t = 0, 1, 2, and so on. The frames of t = 0, 1, 2 of the converted video 31 are obtained by extracting the frames of s = 0, 3, 6 which are videos at the same time of the video 30 before conversion. As a result, the number of images (number of frames) per second of the video 31 to be output becomes the value (10 fps) determined by the information amount determination unit 5.

On the other hand, the conversion of (c) is also performed to convert the video rate 32 to 10 fps, but the number of images to be updated per second is changed without changing the number of images to be output per second. Is. Then, at the timing not to be updated, the previous image is repeatedly output. The frame sequence of the post-conversion video 32 is represented by numbers u = 0, 1,..., 6 and is equal to the number of images per second of the pre-conversion video 30. The frames of u = 0, 1, 2, 3, 4, 5, 6 of the converted image 32 are equal to the frames of s = 0, 0, 0, 3, 3, 3, 6 of the image 30 before conversion. . In this case, s = 0, 3, and 6 are extracted from the frame of the pre-conversion video 30, and these are repeated three times to reduce the number of image updates. Also in the conversion of (c), the substantial frame rate of the received video is changed to the frame rate determined by the information amount determination unit 5.

Also, conversion to frame rate = 0 fps means that a still image is output. In this case, the information amount changing unit 6 stores the image (frame) once output, and repeatedly outputs the same image at the next output timing. Thus, the frame rate = 0 fps means that a certain video frame is displayed every time, and does not stop the video display.

It should be noted that converting the frame rate may give a sense of discomfort to the user who sees the video at the time of switching. Therefore, it is desirable that the information amount changing unit 6 should gradually switch the frame rate of the video to be output instead of instantaneously with a certain time width.

The video output unit 7 receives the video whose information amount has been changed by the information amount changing unit 6 and generates a display signal for driving the video display unit 8. The video display unit 8 receives a display signal from the video output unit 7 and displays a video.

In this way, when the frame rate of the video displayed on the video display unit 8 decreases, the change in the video entering the user's field of view decreases. The moving user can reduce the degree of user's concentration (realism) with respect to the video by reducing the movement of the displayed video. As a result, discomfort similar to sickness is reduced, and attention to the surrounding environment is improved. In particular, when the user is driving, a still image is displayed so that the driving operation is not hindered. However, in any case, since the video display is continued for the user, the provision of information is not interrupted. Thereby, the video display apparatus which can provide information continuously can be implement | achieved, ensuring the cancellation and safety of the discomfort of the user who is moving.

In the first embodiment, as an example of reducing the amount of information per unit time, it has been described that the frame rate of the video is lowered. In the second embodiment, another method for reducing the amount of information per unit time will be described.

(1) A method of changing the display image to black and white or gray scale display.
In order to reduce the amount of information per unit time, the information amount determination unit 5 determines to display the video in black and white or gray scale, and the information amount change unit 6 converts the color video received from the video reception unit 2 into black and white. Alternatively, processing for changing to gray scale display is performed. First, the process of changing to grayscale is performed as follows. For the color video received from the video receiver 2, the RGB values constituting each pixel in each frame are (R, G, B). Then, A = (r × R + g × G + b × B) is calculated using coefficients r, g, and b satisfying r + g + b = 1, and the RGB value (A, A, A) of the pixel is calculated using the calculated value A. ). On the other hand, in the process of changing to black and white display, the threshold value Ath for the value A of each pixel is determined. If A ≧ Ath, the pixel is white, and if A <Ath, the pixel is black. By making the video to be displayed black and white or grayscale, the user's degree of concentration (realism) with respect to the video can be reduced.

(2) A method of matching the hue of the displayed image with the surrounding hue.
In order to reduce the amount of information per unit time, the information amount determination unit 5 determines to match the hue of the image with the surrounding hues, and the information amount change unit 6 applies to the image received from the image reception unit 2. Then, a process for matching the hue with the surrounding hue is performed. Specifically, based on the surrounding image captured by the image capturing unit 3, the movement detection unit 4 detects in what hue environment the user is moving, and the detection result is used as the information amount determination unit 5. Send to. The information amount determination unit 5 sends the presence / absence of the hue change and the surrounding hue acquired from the movement detection unit 4 to the information amount change unit 6. The information amount changing unit 6 changes the hue of each pixel of each frame of the video received from the video receiving unit 2 to the surrounding hue sent from the information amount determining unit 5. By matching the hue of the video to be displayed with the surrounding hue, the video can be assimilated to the surroundings, and the user's degree of concentration (realism) can be reduced.

(3) A method of reducing the angle of view of the image by reducing the image.
In order to reduce the amount of information per unit time, the information amount determination unit 5 determines to reduce and display the video, and the information amount change unit 6 performs processing to reduce and display the video received from the video reception unit 2 Apply. Specifically, the video received from the video receiver 2 is reduced in the vertical and / or horizontal directions, and the reduced video is arranged in a part of the video display area output by the information amount changing unit 6. . It is desirable to convert the display area where the reduced video is not arranged into a single color such as black. By reducing the video and reducing the angle of view of the video, it is possible to suppress a change in the user's vision with respect to the video and reduce the user's degree of concentration (realism) with respect to the video.

(4) A method of displaying a part of an image and reducing the angle of view of the image.
In order to reduce the amount of information per unit time, the information amount determination unit 5 determines to display a part of the video, and the information amount change unit 6 applies to the video received from the video reception unit 2 with respect to the video. A process of displaying a part is performed. Specifically, a part of the video received from the video receiver 2 is cut out and output. The remaining display area that is not cut out is preferably converted into a single color such as black. By displaying a part of the video and reducing the angle of view of the video, it is possible to suppress changes in the user's vision with respect to the video and to reduce the user's degree of concentration (realism) with respect to the video.

(5) A method of reducing the brightness of the displayed image.
In order to reduce the amount of information per unit time, the information amount determination unit 5 determines to reduce the luminance of the video, and the information amount change unit 6 applies to the video received from the video reception unit 2 with respect to the video. A process for reducing the luminance is performed. Specifically, the luminance, hue, and saturation of each pixel of each frame of the image received from the image receiving unit 2 are obtained, and the hue and saturation are processed while maintaining the hue and saturation. The changed luminance is used as the color data of the pixel. By reducing the luminance of the video, it is possible to suppress changes in the user's vision with respect to the video, and to reduce the user's degree of concentration (realism) with respect to the video.

(6) A method of reducing the brightness of the displayed image.
In the above (5), the information amount changing unit 6 performs processing for lowering the luminance of the video received from the video receiving unit 2. As a modification, the luminance can be lowered by the video output unit 7 instead of the information amount changing unit 6.

FIG. 5 is a block diagram showing a modification of the video display device. In the video display device 10a, the information amount changing unit 6 in FIG. 1 is deleted, the video output unit 7 receives the video from the video receiving unit 2, and changes the luminance of the video according to the luminance control signal determined by the information amount determining unit 5. It is configured to do. The video display unit 8 includes a light source (backlight or the like) (not shown) for displaying a video, and changes the luminance by changing the intensity of light emitted from the light source. When the information amount determination unit 5 determines to lower the luminance, the video output unit 7 sends a drive signal to the video display unit 8 so as to reduce the intensity of light emitted from the light source. Thereby, the brightness | luminance of the image | video displayed on the image | video display part 8 falls. According to this configuration, there is a secondary effect of reducing power consumption at the light source.

In the second embodiment, several methods for reducing the amount of information per unit time have been described, but these methods are not only used alone, but also a plurality of methods may be combined or used in combination with the method of the first embodiment. Can do. For example, in order to reduce the amount of information per unit time, a combination of reducing the frame rate of the video (Example 1) and displaying a part of the video (the method (4) above) may cause a sense of incongruity. There can be few images.

In the first and second embodiments, the image acquired by the imaging unit 3 is used to detect the movement of the user. However, in the third embodiment, the measurement is performed by another sensor in addition to the image acquired by the imaging unit 3. By using the result, a configuration in which the movement state of the user is detected with high accuracy is adopted. Here, the sensor is, for example, a three-dimensional acceleration sensor, and thereby detects the user's walk.

FIG. 6 is a block diagram of the video display apparatus according to the third embodiment. The video display device 10b newly adds a sensor 11. The same elements as those in the first embodiment (FIG. 1) are denoted by the same reference numerals, and redundant description is omitted.

The sensor 11 is, for example, a three-dimensional acceleration sensor, and the movement detection unit 4 detects the movement state of the user using both the image acquired by the imaging unit 3 and the result measured by the sensor 11. At that time, the movement detection unit 4 uses the two pieces of information (captured image and sensor measurement result) to obtain the probability values that the user is moving, respectively, and statistically determines the final movement state of the user from the two probability values. Make a decision. Alternatively, the movement determination criterion based on one information can be changed according to the other information. The operation of this embodiment will be described below.

The detection of the movement of the user using the image from the imaging unit 3 is basically the same as in the first embodiment. However, the probability that the user is moving is obtained instead of binaryly determining whether or not the user is moving by tracking the movement of the above-described feature in the image. For example, the moving speed of the feature can be obtained by comparing two or more captured images obtained at different times acquired from the imaging unit 3, and the probability that the user is moving can be obtained from the moving speed.

Next, detection of user movement using the sensor (three-dimensional acceleration sensor) 11 will be described. The body motion associated with the user walking (or running) is measured by an acceleration sensor. The three-dimensional acceleration sensor is installed on the HMD so that when the HMD is mounted on the head and the user stands up and faces the front, the acceleration in three axial directions can be measured. Hereinafter, the user's up and down direction (Sz) is the first axis, the user's left and right direction (Sy) is the second axis, and the user's front and rear direction (Sx) is the third axis.

FIG. 7 is a diagram illustrating a detection signal of the three-dimensional acceleration sensor with respect to the user's walking motion. The user's walking (or running) operation is repeated with the following four states (W1) to (W4) as one cycle.
(W1) A state in which the right foot is on the front side in the traveling direction from the trunk and the left foot is on the rear side in the traveling direction from the trunk.
(W2) A state in which both the left and right feet are directly under the torso and the right foot is in contact with the ground while the left foot is not in contact with the ground.
(W3) A state in which the left foot is on the front side in the traveling direction from the trunk and the right foot is on the rear side in the traveling direction from the trunk.
(W4) A state in which both the left and right feet are directly under the torso and the left foot is in contact with the ground, while the right foot is not in contact with the ground.

During one cycle (T) of the walking, the acceleration sensors (Sz, Sx) in the first axis direction and the third axis direction output signals of two cycles, and the acceleration sensor (Sy) in the second axis direction has one cycle. The signal is output. Accordingly, the frequency (fz, fx) of the output of the acceleration sensor in the first axis direction and the third axis direction is twice the frequency (fy) of the output of the acceleration sensor in the second axis direction, and further the frequency in the second axis direction. The frequency (fy) of the output of the acceleration sensor has a relationship that is the reciprocal (1 / T) of the time required for one cycle of walking.

Accordingly, in walking detection, the outputs of the acceleration sensors (Sz, Sx) in the first and third axes include a frequency component that is twice the reciprocal of the time required for one cycle of walking (that is, fz = fx = 2). / T). Similarly, the output of the acceleration sensor (Sy) in the second axis direction uses a frequency component that is the reciprocal of the time required for one cycle of walking (that is, fy = 1 / T). Therefore, the frequency ratio (fz / fy) of the output of the acceleration sensor in the first axis and second axis directions, or the frequency ratio (fx / fy) of the output of the acceleration sensor in the third axis and second axis directions are both A walking determination condition is a value of 2. However, the actual output of the acceleration sensor includes noise of other frequency components. Therefore, when determining whether the user is walking from the frequency ratio (fz / fy or fx / fy) of the output of the acceleration sensor, the determination condition is widened, and the frequency ratio is, for example, 1.7-2. It is better to judge that the possibility that the user is walking is in the range of .3.

Also, the time required for one cycle of walking (running) varies depending on the user, and changes depending on the walking (running) environment. Therefore, when determining the user's walking from the output frequency (fy) of the acceleration sensor (Sy) in the second axis direction, the determination condition is widened, and the frequency ranges from 0.8 Hz to 1.2 Hz, for example. It is determined that there is a high possibility that the user is walking. Further, when the frequency is, for example, 2 Hz or more, it is determined that the user is likely to travel.

In the above, in order to determine the user's walking, the frequency ratio (fz / fy, fx / fy) of the outputs between the acceleration sensors (Sz, Sy, Sx) in the triaxial direction and the acceleration sensor (Sy) in the second axial direction Both of the output frequencies (fy) are used, but only one of them may be used. In the above description, the output frequency (fy) of the acceleration sensor (Sy) in the second axis direction is used, but the first axis or third axis acceleration sensor (Sz, Sx), or the first to third axes. You may use the frequency of the output of several acceleration sensors among axes. When using acceleration in only one axial direction or using only acceleration in two axial directions, a one-dimensional acceleration sensor or a two-dimensional acceleration sensor may be used as the sensor 11.

The movement detection unit 4 determines the movement state of the user based on the probability of the user moving from the captured image of the imaging unit 3 and the probability of the user moving determined from the measurement result of the sensor 11. .

FIG. 8 is a diagram illustrating an example of determination of the movement state of the user. Here, the case of “walking” is shown as the movement state. The walking probability obtained from the captured image of the imaging unit 3 is P1, and the walking probability obtained from the measurement result of the sensor 11 is P2. The arithmetic average is obtained from the two probabilities P1 and P2. If the arithmetic average Pav is equal to or greater than the threshold value Pth, the user is walking, and if the arithmetic average Pav is less than the threshold value Pth, it is determined that the user is not walking. Here, the threshold value Pth is set to 60%.

For example, when the walking probability P1 obtained from the imaging unit 3 and the walking probability P2 obtained from the sensor 11 are both 80%, the arithmetic average Pav is also 80%, so that it is determined that the user is walking. On the other hand, when the walking probability P1 obtained from the imaging unit 3 is 80% and the walking probability P2 obtained from the sensor 11 is 20%, the arithmetic average Pav is 50%, so that it is determined that the user is not walking.

In addition, when calculating | requiring the average Pav of two probability P1, P2, you may employ | adopt a weighted average instead of an arithmetic average. The same determination can be made when the movement state of the user is “running”.

In the above example, the three-dimensional acceleration sensor is used as the sensor 11 for detecting the movement of the user, but it can be replaced with another sensor. The other applicable sensors are listed below.
(1) Vibration detection switch A vibration detection switch is installed in the HMD in order to measure the movement of the user in the first axis direction (vertical direction). By installing the vibration detection switch in the HMD as described above, the vibration detection switch detects two vibrations in one cycle of user walking. By detecting this vibration, the movement detector 4 determines whether or not the user is walking.

(2) Position measuring device A position measuring device using, for example, a GPS signal is installed in the HMD. The movement detection unit 4 obtains the moving speed of the user from the position information calculated by the position measurement device. If the value is 3 km to 6 km per hour, for example, it is determined that the user is likely to be walking.

(3) Imaging means for user eyes Apart from the imaging section 3 that acquires surrounding images, imaging means (camera) that detects the movement of the user's eyes is installed. When the user is walking, the movement of the eyes becomes active in order to grasp the surrounding environment. When the movement of the line of sight is detected and the user's line of sight is actively moving, it is determined that the user is likely to be walking.

(4) Sweating sensor A sweating sensor is attached to the user's body. Since the sweating action is active when the user is walking, it is determined that the possibility that the user is walking is higher as the sweating amount per unit time is larger.

(5) Pulse meter A pulse meter is attached to the user's body. Since the heart rate increases when the user is walking, it is determined that the higher the heart rate, the higher the possibility that the user is walking.

In the description of each sensor described above, it is assumed that the user's walking is detected, but it is needless to say that traveling can also be detected.
In the above description, the sensor 11 is assumed to be mounted on the HMD. However, the sensor measurement result may be input from an external device.

FIG. 9 is a block diagram showing a modification of the video display device. In the video display device (HMD) 10 c, a sensor result input unit 13 for inputting a measurement result of a sensor mounted on the external device 12 is provided instead of the sensor 11. The movement detection unit 4 determines the movement of the user using the sensor measurement result input to the sensor result input unit 13. As the external device 12, for example, a smartphone can be used. For example, Bluetooth (registered trademark) can be used as a communication unit between the external device 12 and the HMD 10c. In the case of using Bluetooth, the sensor result input unit 13 is a Bluetooth receiver.

According to the third embodiment, the movement detection unit 4 determines only the image acquired by the imaging unit 3 by determining the movement of the user using both the image acquired by the imaging unit 3 and the result measured by the sensor 11. Compared with the case where it uses, the detection accuracy of a movement state can be improved.

In the first to third embodiments, the video receiving unit 2 receives video from the video providing unit 1, and the information amount changing unit 6 changes the information amount per unit time of the received video according to the determination of the information amount determining unit 5. It was something to do. On the other hand, in the fourth embodiment, in accordance with the determination by the information amount determining unit 5, the video providing unit 2 is requested to provide a video having the determined information amount per unit time.

FIG. 10 is a block diagram of a video display apparatus according to the fourth embodiment. The video display device 10 d is provided with a video request unit 14 by deleting the information amount changing unit 6. The same elements as those in the first embodiment (FIG. 1) are denoted by the same reference numerals, and redundant description is omitted.

When the information amount determination unit 5 determines the information amount per unit time of the video to be displayed according to the movement state of the user, the information amount determination unit 5 sets a video specification (for example, a frame rate) for realizing this, and the video request unit 14 Send to. The video request unit 14 sends a request signal related to the video to be provided to the video providing unit 1 based on the video specification received from the information amount determination unit 5. The video providing unit 1 transmits a video corresponding to the request to the video receiving unit 2. As a result, the video output unit 7 and the video display unit 8 can display a video having an information amount per unit time determined by the information amount determination unit 5. That is, the request signal from the video request unit 14 to the video providing unit 1 reflects the movement state of the user detected by the movement detection unit 4.

Here, the video providing unit 1 is a storage device arranged in an external data center, for example, and is a device that can convert and provide a video in response to a request from the video requesting unit 14. The video providing unit 1 may be a configuration built in the video display device 10d, for example, an imaging device.

In the above example, the movement detection unit 4 detects the movement of the user based on the image acquired by the imaging unit 3. However, as in Example 3 (FIG. 6), the sensor 11 measured in addition to the imaging unit 3. A configuration in which the movement of the user is detected using the result may be used.

According to the fourth embodiment, there is an effect of minimizing the amount of video data transmitted from the video providing unit 1 to the video receiving unit 2. In particular, when the video providing unit 1 is a storage device arranged in an external data center, the amount of data transmitted over the network is reduced, and the load on the transmission path is reduced. Further, when the video providing unit 1 is an imaging device, the number of operations of the imaging element in the device is reduced, and there is a secondary effect of power saving and longer life of the device.

The present invention is not limited to the above-described embodiments, and includes various modifications. In the above description, the HMD has been described as an example of the video display device used while the user is moving. However, the present invention is not limited to this. For example, a head-up display (hereinafter abbreviated as HUD), which is a transmissive video display, can be used similarly. In the HUD mounted on the vehicle, the traveling state of the vehicle is detected as the movement state of the user, and the information amount per unit time of the video to be displayed is changed accordingly. Thereby, the gaze degree with respect to the user's image at the time of vehicle travel can be eased, and HUD which secured safety can be provided.

In the above-described embodiments, the configuration of each part is described in detail in order to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to one having all the configurations described. Further, a part of the configuration of a certain embodiment can be replaced with the configuration of another embodiment, or the configuration of another embodiment can be added to the configuration of a certain embodiment.

1: Video provider
2: Video receiver,
3: Imaging unit,
4: Movement detector
5: Information amount determination unit,
6: Information amount changing part,
7: Video output unit,
8: Video display section
10, 10a to 10d: video display device,
11: sensor,
12: External equipment
13: Sensor result input part,
14: Image request unit.

Claims

In a video display device that a user can use while moving,
An information amount changing unit for changing the amount of information per unit time for the video provided from the video providing unit;
A video display unit for displaying a video whose information amount has been changed by the information amount changing unit;
An imaging unit for imaging a subject around the user;
A movement detection unit that detects a movement state of the user based on an image captured by the imaging unit;
An information amount determining unit that determines a change value of the information amount of the image per unit time for the information amount changing unit according to the movement state of the user detected by the movement detecting unit; Display device.
In a video display device that a user can use while moving,
A video requesting unit for sending a request signal related to the video to be provided to the video providing unit;
A video display unit for displaying the video provided from the video providing unit according to the request signal;
An imaging unit for imaging a subject around the user;
A movement detection unit that detects a movement state of the user based on an image captured by the imaging unit;
Information for determining a change value of the information amount per unit time of the video provided by the video providing unit in response to a request signal sent from the video requesting unit according to the movement state of the user detected by the movement detecting unit An image display device comprising: an amount determination unit.
The video display device according to claim 1 or 2,
A sensor for measuring the state of the user;
The said movement detection part detects the movement state of the said user based on the measurement result by the said sensor with the image acquired from the said imaging part, The video display apparatus characterized by the above-mentioned.
The video display device according to claim 1 or 2,
A sensor result input unit for inputting a measurement result of the user state measured by a sensor of an external device;
The said movement detection part detects the movement state of the said user based on the sensor measurement result input in the said sensor result input part with the image acquired from the said imaging part, The video display apparatus characterized by the above-mentioned.
The video display device according to claim 1 or 2,
A video display device comprising a mounting portion for mounting the video display device on the user's head.
The video display device according to any one of claims 1 to 5,
According to the detection result of the movement detection unit, the information amount determination unit reduces a change value of the information amount per unit time of the video when the user is moving compared to when the user is not moving. An image display device characterized by determining.
The video display device according to claim 6,
When the movement detection unit detects that the user is moving, the information amount determination unit determines the amount of information per unit time of the video when the movement speed of the user is large, compared to when the movement speed is low. An image display device characterized in that the change value of the image is determined to be small.
The video display device according to claim 7,
When the movement detection unit detects that the user is moving and is driving the vehicle, the information amount determination unit displays a change value of the information amount per unit time of the video, and the user selects the vehicle. An image display device characterized in that it is determined to be smaller than that when not in operation.
The video display device according to any one of claims 1 to 5,
The amount of information per unit time of the image is a frame rate of the image displayed on the image display unit.
The video display device according to any one of claims 1 to 5,
Reducing the amount of information per unit time of the video is to change the video displayed on the video display unit from color display to black and white display or gray scale display.
The video display device according to any one of claims 1 to 5,
The amount of information per unit time of the image is the size of the angle of view of the image displayed on the image display unit.
The video display device according to any one of claims 1 to 5,
Reducing the amount of information per unit time of the video means reducing the luminance of the video displayed on the video display unit.
The video display device according to claim 3 or 4,
The image display device, wherein the sensor includes any one of a three-dimensional acceleration sensor, a vibration detection switch, and a position measurement device.