JP2024047670A - Display system and display method - Google Patents

Abstract

[Problem] To adjust a sudden change in brightness when a display device is removed. [Solution] A display system 10 includes a display unit 22 provided inside a housing 20, a time measurement unit 42 that measures the elapsed time from the start of display by the display unit 22, an end detection unit 44 that detects the end of display by the display unit 22, and an adaptation display determination unit 46 that determines an adaptation display pattern that reduces the difference between an internal illuminance value indicating the brightness inside the housing 20 and an external illuminance value indicating the brightness outside the housing 20 by changing the brightness of an image to be displayed on the display unit 22, in accordance with the elapsed time from the start to the end of display by the display unit 22. [Selected Figure] Figure 2

Description

The present invention relates to a display system and a display method.

A head-mounted display that is worn on the user's head is known as a display system. In order to reduce eye strain caused by continuous long-term use of a head-mounted display, technology has been proposed that reduces contrast and increases brightness as the image display time progresses (see, for example, Patent Document 1).

JP 2003-76353 A

In the case of a head-mounted display device that blocks external light, the brightness inside the device that the user perceives based on the brightness of the image displayed when using the device is unrelated to the brightness of the environment outside the device. As a result, when the user finishes using the device and removes it, there may be a large difference in brightness between the inside and outside of the device. In this case, it takes time for the user to get used to the brightness of the environment outside the device, which places a strain on the user's eyes.

The present invention was made in consideration of the above circumstances, and aims to adjust the sudden change in brightness when the display device is removed.

A display system according to one embodiment of the present invention includes a display unit that is provided inside a housing that is worn on the user's head and displays an image to the user, a time measurement unit that measures the elapsed time from the start of display by the display unit, an end detection unit that detects the end of display by the display unit, and an adaptive display determination unit that determines an adaptive display pattern that reduces the difference between an internal illuminance value indicating the brightness inside the housing and an external illuminance value indicating the brightness outside the housing by changing the brightness of the image displayed on the display unit, in accordance with the elapsed time from the start to the end of display by the display unit.

Another aspect of the present invention is a display method. This method includes the steps of displaying an image on a display unit provided inside a housing worn on a user's head, measuring the elapsed time from the start of display by the display unit, detecting the end of display by the display unit, and determining an adaptive display pattern that reduces the difference between an internal illuminance value indicating the brightness inside the housing and an external illuminance value indicating the brightness outside the housing by changing the brightness of the image displayed on the display unit, in accordance with the elapsed time from the start to the end of display by the display unit.

In addition, any combination of the above components or mutual substitution of the components or expressions of the present invention between methods, devices, systems, etc. are also valid aspects of the present invention.

The present invention makes it possible to adjust for sudden changes in brightness when the display device is removed.

FIG. 1 is a diagram illustrating a schematic configuration of a display system according to an embodiment. 1 is a block diagram illustrating a functional configuration of a display system according to an embodiment. 13 is a table showing an example of a display time according to age. 11 is a table showing an example of a display time according to a usage time. 11 is a table showing an example of a display time according to an illuminance difference between an internal illuminance value and an external illuminance value. 6A and 6B are diagrams showing schematic examples of adaptation display patterns. FIG. 13 is a diagram showing an example of an adaptation display image. 4 is a flowchart showing a display method according to the embodiment. FIG. 11 is a block diagram illustrating a functional configuration of a display system according to another embodiment. 13 is a flowchart showing a display method according to another embodiment.

The following describes an embodiment of the present invention with reference to the drawings. The specific numerical values and the like shown in the embodiment are merely examples to facilitate understanding of the invention, and do not limit the present invention unless otherwise specified. In the drawings, elements that are not directly related to the present invention are omitted.

FIG. 1 is a diagram showing a schematic configuration of a display system 10 according to an embodiment. The display system 10 includes a display device 12, a control device 14, and an input device 16.

The display device 12 is a so-called head-mounted display, and is worn on the head of the user 50. The display device 12 is fixed to the head of the user 50 by a fastener 18, such as a cable tie. The display device 12 includes a housing 20, a display unit 22, an external illuminance sensor 24, an internal illuminance sensor 26, and an internal camera 28.

The housing 20 is worn on the head of the user 50 so as to cover both eyes of the user 50. The housing 20 has, for example, a goggle shape that entirely covers both eyes of the user 50. The housing 20 is configured to block external light, and the inside of the housing 20 is in darkness when no image is displayed on the display unit 22.

The display unit 22 is provided inside the housing 20 and displays an image for the user 50. The display unit 22 is composed of an image display element arranged, for example, at a position facing both eyes of the user 50. As the image display element, a liquid crystal display (LCD) or an organic electroluminescence display (OELD) can be used. The display unit 22 may be a projection-type display device, and may be composed of a projector that projects image display light toward a display surface viewed by the user 50.

The external illuminance sensor 24 is provided outside the housing 20. The external illuminance sensor 24 measures the brightness outside the housing 20 and outputs an external illuminance value indicating the brightness outside the housing 20. The external illuminance sensor 24 is, for example, a sensor having a photodiode that detects the brightness of visible light, and is configured to output an illuminance value indicating the brightness of visible light incident on the photodiode.

The internal illuminance sensor 26 is provided inside the housing 20. The internal illuminance sensor 26 measures the brightness inside the housing 20 and outputs an internal illuminance value indicating the brightness inside the housing 20. The internal illuminance sensor 26 is, for example, a sensor having a photodiode that detects the brightness of visible light, and is configured to output an illuminance value indicating the brightness of visible light incident on the photodiode.

The internal camera 28 is provided inside the housing 20. The internal camera 28 is positioned so as to be able to capture an image of at least one of the eyes of the user 50, for example, at a position facing at least one of the eyes of the user 50. The internal camera 28 includes, for example, a light source that irradiates infrared illumination light toward the user 50, and an imaging element that captures an image of at least one of the eyes of the user 50 with infrared light. A semiconductor light-emitting element such as an LED (Light Emitting Diode) can be used as the light source. A two-dimensional image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) can be used as the imaging element.

The control device 14 controls the overall operation of the display device 12. In terms of hardware, the control device 14 can be realized by elements or mechanical devices such as a processor such as a computer's CPU (Central Processing Unit) and memories such as ROM (Read Only Memory) and RAM (Random Access Memory), and in terms of software, it is realized by a computer program, etc.

The control device 14 may be a dedicated terminal for controlling the operation of the display device 12, or may be a mobile terminal such as a smartphone owned by the user 50. The control device 14 may be configured to communicate with the display device 12 in a wired manner via a communication cable, or may be configured to communicate with the display device 12 wirelessly.

In the example of FIG. 1, the control device 14 is depicted as being separate from the display device 12, but the control device 14 may be configured as one unit with the display device 12. For example, if the control device 14 is a mobile terminal such as a smartphone, the display screen of the smartphone may function as the display unit 22. In addition, at least some of the functions of the control device 14 may be configured as one unit with the display device 12, and at least some of the functions of the control device 14 may be realized by a server device or the like that is installed away from the user 50.

The input device 16 is a device that accepts input operations from the user 50. The input device 16 is, for example, a joystick or a game pad that is configured to be held and used by the user 50 wearing the display device 12. The input device 16 may be a mouse or a keyboard. The input device 16 may be a microphone that accepts voice input from the user 50. When the input device 16 is a microphone, the microphone may be provided in the display device 12, or, for example, the microphone may be provided outside the housing 20.

FIG. 2 is a block diagram showing a schematic functional configuration of a display system 10 according to an embodiment. As described above, the display device 12 includes a display unit 22, an external illuminance sensor 24, an internal illuminance sensor 26, and an internal camera 28. The control device 14 includes a display control unit 32, an illuminance acquisition unit 34, an image recognition unit 36, an input acceptance unit 38, an age acquisition unit 40, a time measurement unit 42, an end detection unit 44, and an adaptation display determination unit 46.

The display control unit 32 generates an image to be displayed on the display unit 22 and controls the operation of the display unit 22 based on the generated image. The display control unit 32 generates an image for providing a virtual reality to the user 50, for example, and generates an image according to an input operation of the user 50. The display control unit 32 may generate an image based on data acquired from an arbitrary external device (not shown), and may generate an image based on video data for playing video content such as movies and animations, for example.

When the display unit 22 ends display, the display control unit 32 causes an image to be displayed according to the adaptive display pattern determined by the adaptive display determination unit 46. The adaptive display pattern determines a pattern in which the brightness of the image to be displayed on the display unit 22 changes over time, for example, a pattern in which the brightness of the image monotonically increases or decreases over time. By causing the display control unit 32 to display an image according to the adaptive display pattern on the display unit 22, a sudden change in brightness is prevented when the user 50 removes the display device 12. Details of the adaptive display pattern will be described separately later.

The illuminance acquisition unit 34 acquires an external illuminance value measured by the external illuminance sensor 24 and an internal illuminance value measured by the internal illuminance sensor 26. Instead of acquiring an internal illuminance value from the internal illuminance sensor 26, the illuminance acquisition unit 34 may acquire the internal illuminance value by calculating the internal illuminance value based on the brightness of an image displayed on the display unit 22. The illuminance acquisition unit 34 can calculate the internal illuminance value using, for example, a formula or table that associates the brightness of an image generated by the display control unit 32 with an internal illuminance value. The formula or table can be determined, for example, by measuring in advance the brightness inside the housing 20 when an image of a predetermined brightness is displayed on the display unit 22. In this case, the display device 12 does not need to include the internal illuminance sensor 26. The illuminance acquisition unit 34 may acquire the internal illuminance value based on the brightness of an image displayed on the display unit 22.

The image recognition unit 36 acquires an image captured by the internal camera 28 and analyzes the acquired image using image recognition technology. For example, the image recognition unit 36 estimates the gaze direction of the user 50 from an image that includes at least one of the user's 50 eyes. The image recognition unit 36 may estimate the age of the user 50 from an image that includes at least one of the user's 50 eyes. Note that if the gaze direction or age is not estimated by the image recognition unit 36, the display device 12 does not need to include the internal camera 28, and the control device 14 does not need to include the image recognition unit 36.

The input acceptance unit 38 acquires information indicating an input operation input to the input device 16. The input acceptance unit 38 accepts, for example, an input operation by the user 50 to input his/her own age. The display control unit 32 may generate an operation screen for the input operation by the user 50 and display it on the display unit 22. The input acceptance unit 38 may accept gaze input by the user 50 using the gaze direction of the user 50 estimated by the image recognition unit 36. The input acceptance unit 38 may accept voice input by the user 50 using a voice signal from a microphone.

The age acquisition unit 40 acquires a value related to the age of the user 50. The age acquisition unit 40 may acquire the age of the user 50 estimated by the image recognition unit 36, or may acquire the age of the user 50 accepted by the input acceptance unit 38. The age acquisition unit 40 may acquire the numerical value of the age of the user 50 itself, or may acquire a value indicating which range the age of the user 50 falls into. For example, if the first range is less than 30 years old, the second range is between 30 and 50 years old, and the third range is 50 years old or older, the age acquisition unit 40 may acquire any one of the values 1, 2, and 3 corresponding to the first range, the second range, and the third range.

The time measurement unit 42 measures the usage time of the display device 12 by the user 50. The time measurement unit 42 measures the elapsed time from the start of display by the display unit 22 as the usage time. Here, the display start timing of the display unit 22 may be the timing when the display device 12 is turned on, or the timing when some image or video content is displayed on the display unit 22 after the power is turned on. The elapsed time measured by the time measurement unit 42 may match the playback time of the video content.

The end detection unit 44 detects when the user 50 has finished using the display device 12. The end detection unit 44 detects when the display unit 22 has finished displaying. The end detection unit 44 can detect the end of the display based on an input operation by the user 50, and for example, detects the end of the display when an input operation is performed to turn off the power of the display device 12. The end detection unit 44 may also detect the end of the display based on the operation of the display control unit 32, and may detect the end, for example, when the user 50 has watched the video content to the end or when playback of the video content is interrupted.

The adaptive display determination unit 46 determines an adaptive display pattern that determines the change in brightness of the image to be displayed on the display unit 22 after the end is detected by the end detection unit 44. The adaptive display pattern determines, for example, an initial value p of the image brightness, an end value q of the image brightness, and a display time t required for the brightness to change from the initial value p to the end value q.

The initial value p of the adaptation display pattern is determined based on the internal illuminance value at the end timing detected by the end detection unit 44. Specifically, the initial value p is determined so that the internal illuminance value when an image with a brightness of the initial value p is displayed on the display unit 22 matches the internal illuminance value at the end timing. In other words, the initial value p matches the brightness of the image displayed on the display unit 22 at the end timing.

The end value q of the adaptation display pattern is determined based on the external illuminance value at the end timing detected by the end detection unit 44. Specifically, the end value q is determined so that the internal illuminance value when an image with a brightness of the end value q is displayed on the display unit 22 matches the external illuminance value at the end timing.

The display time t of the adaptation display pattern is determined according to at least one of a plurality of conditions. The plurality of conditions include, for example, the age a of the user 50, the elapsed time b of the display on the display unit 22, and the illuminance difference c between the internal illuminance value and the external illuminance value. When the age a is used, for example, the older the age of the user 50 acquired by the age acquisition unit 40, the longer the display time t of the adaptation display pattern is made. When the elapsed time b is used, for example, the longer the elapsed time measured by the time measurement unit 42, the longer the display time t of the adaptation display pattern is made. When the illuminance difference c is used, for example, the larger the illuminance difference between the internal illuminance value and the external illuminance value, the longer the display time t of the adaptation display pattern is made.

The adaptive display determination unit 46 can determine the display time t using a table or a formula that defines the relationship between at least one of the multiple conditions and the display time t. When a table is used, the corresponding relationship between at least one numerical range of the multiple conditions and the display time t is defined. When a formula is used, an addition value α or a multiplication value β of the display time that corresponds to the numerical value or numerical range of at least one of the multiple conditions is defined. Note that tables and formulas can be combined in any desired way.

FIG. 3 is a table showing an example of the display time t according to the age a. In the example of FIG. 3, the age a is divided into three ranges, and an additional value _αA of the display time t is associated with each range. Specifically, in the first range (under 30 years old), _αA = 0 seconds, in the second range (30 years old or older and under 50 years old), _αA = 5 seconds, and in the third range (50 years old or older), _αA = 10 seconds. Note that the number of ranges is not limited to three, and may be two, or may be four or more. Alternatively, a formula may be determined instead of the table. For example, the additional value _αA of the display time t may be determined as a function _fA (a) of the age a, and _αA = _fA (a).

4 is a table showing an example of a display time t according to an elapsed time b. In the example of FIG. 4, the elapsed time b is divided into three ranges, and an additional value α _B of the display time t is associated with each range. Specifically, in the first range (less than 1 hour), α _B = 0 seconds, in the second range (1 hour or more and less than 2 hours), α _B = 5 seconds, and in the third range (2 hours or more), α _B = 10 seconds. Note that the number of ranges is not limited to three, and may be two, or may be four or more. Alternatively, a formula may be determined instead of the table. For example, the additional value α _B of the display time t may be determined as a function f _B (b) of the elapsed time b, and α _B = f _B (b).

FIG. 5 is a table showing an example of the display time t according to the illuminance difference c between the internal illuminance value and the external illuminance value. In the example of FIG. 5, the illuminance difference c is divided into three ranges, and the multiplication value β _C of the display time t is associated with each range. Specifically, in the first range (less than the first reference value c1), β _C = 0.8, in the second range (not less than the first reference value c1 and not more than the second reference value c2), β _C = 1, and in the third range (not less than the second reference value c2), β _C = 1.2. Note that the number of ranges is not limited to three, and may be two, or may be four or more. Alternatively, a formula may be determined instead of the table. For example, the multiplication value β _C of the display time t may be determined by a function f _C (c) of the illuminance difference c, and may be β _C = f _C (c).

3 to 5 are used, the adaptation display determination unit 46 can determine the display time t by the calculation formula t = ( _αA + _αB ) · _βC . Note that the display time t can also be determined using only one or two of a plurality of conditions including age a, elapsed time b, and illuminance difference c. Furthermore, the display time t may be determined by arbitrarily combining a plurality of functions _fA (a), _fB (b), and _fC (c) defined for each of a plurality of conditions by addition, multiplication, or the like.

The adaptation display determination unit 46 may determine the time rate of change Δ of brightness from the initial value p to the final value q, and may determine the display time t using the determined time rate of change Δ. Here, the time rate of change Δ of brightness is the amount of change per unit time dτ of the brightness L of the image displayed on the display unit 22, and can be expressed as Δ=dL/dτ. The time rate of change Δ may be constant with respect to time τ, or may be a function Δ(τ) that changes with respect to time τ. When the time rate of change Δ is constant, the display time t can be calculated by the formula t=|q-p|/Δ. In the case of a function Δ(τ) with time τ as a variable, the display time t can be determined so that the amount of change in brightness ∫Δ(τ)dτ obtained by integrating the time rate of change Δ(τ) over the display time t is equal to the difference between the initial value p and the final value q (i.e., ∫Δ(τ)dτ=q-p). In the following, a case where the time rate of change Δ is constant will be described, but it is also applicable to a case where a function Δ(τ) with time τ as a variable is used.

The adaptation display determination unit 46 can determine the time change rate Δ according to at least one of the age a and the elapsed time b, and can determine the time change rate Δ such that the time change rate Δ decreases as the age a and the elapsed time b increase. The relationship between the age a or the elapsed time b and the time change rate Δ may be determined by a table such as that shown in FIG. 3 or FIG. 4, or may be determined by a formula using an arbitrary function.

The adaptation display determination unit 46 may determine the display time t according to the magnitude relationship between the internal illuminance value and the external illuminance value. In other words, the display time t may be determined according to the magnitude relationship between the initial value p and the final value q. FIGS. 6(a) and 6(b) are diagrams that diagrammatically show an example of an adaptation display pattern. FIG. 6(a) shows a case where the internal illuminance value is greater than the external illuminance value, and the initial value p _A is greater than the final value q _A (i.e., p _A >q _A ). In this case, the user 50 is required to adapt to the change from a bright state to a dark state (i.e., dark adaptation). FIG. 6(b) shows a case where the internal illuminance value is smaller than the external illuminance value, and the initial value p _B is smaller than the final value q _B (i.e., p _B <q _B ). In this case, the user 50 is required to adapt to the change from a dark state to a bright state (i.e., light adaptation). In general, dark adaptation is considered to take longer than light adaptation. Therefore, the time rate of change Δ _A = (p _A - q _A )/t _A in the dark adaptation pattern in Fig. 6(a) is set smaller than the time rate of change Δ _B = (q _B - p _B )/t _B in the light adaptation pattern in Fig. 6(b). In this way, by making the time rate of change of brightness in the dark adaptation pattern smaller than that in the light adaptation pattern, it is possible to determine an adaptation display pattern suitable for both dark adaptation and light adaptation.

When the end is detected by the end detection unit 44, the display control unit 32 causes the display unit 22 to display an image according to the adaptation display pattern determined by the adaptation display determination unit 46. The display control unit 32 first causes the display unit 22 to display an image whose brightness is an initial value p, causes the display unit 22 to display images whose brightness monotonically increases or decreases over time, and finally causes the display unit 22 to display an image whose brightness is an end value q.

The display control unit 32 may display an image to prompt the user 50 to wait for the display device 12 to be removed. When the display control unit 32 displays an image according to the adaptation display pattern, the display unit 22 may display a message indicating that the adaptation display is in progress and a number indicating the progress rate and remaining time until the adaptation display is completed. FIG. 7 is a diagram showing an example of an adaptation display image. In the example of FIG. 7, the message "Adjusting to the brightness of the external environment. Please wait a moment" is displayed as the message indicating that the adaptation display is in progress, and the number "40%" and a progress bar indicating the progress rate until the adaptation display is completed are displayed. The display control unit 32 causes the display unit 22 to display an image according to the adaptation display pattern by monotonically increasing or decreasing the overall brightness (or luminance) of the adaptation display image shown in FIG. 7.

FIG. 8 is a flowchart showing a display method according to an embodiment. The age acquisition unit 40 acquires the age of the user 50 (step S10). The display control unit 32 causes the display unit 22 to display an image (step S12), and the time measurement unit 42 measures the elapsed time from the start (step S14). If the end detection unit 44 does not detect the end (No in step S16), the processing of S12 to S14 is continued. If the end detection unit 44 detects the end (Yes in step S16), the illuminance acquisition unit 34 acquires an internal illuminance value and an external illuminance value (step S18), and the adaptation display determination unit 46 determines the adaptation display pattern based on the internal illuminance value and the external illuminance value (step S20). The adaptation display determination unit 46 may determine the adaptation display pattern according to the age of the user 50, or may determine the adaptation display pattern according to the elapsed time from the start to the end of the display measured by the time measurement unit 42. The display control unit 32 causes the display unit 22 to display an adaptation display image whose brightness changes according to the determined adaptation display pattern (step S22).

According to this embodiment, before the use of the display device 12 is finished and the display device 12 is removed, an adaptation display image that changes in brightness so that the illuminance inside and outside the housing 20 matches is displayed on the display unit 22, thereby suppressing a sudden change in brightness when the display device 12 is removed. Even if there is a large difference in brightness between the inside and outside of the display device 12 when use is finished, the eyes of the user 50 can be gradually accustomed to the brightness of the external environment before the display device 12 is removed, thereby reducing the strain on the eyes of the user 50.

According to this embodiment, by adjusting the display time of the adaptive display pattern according to the age of the user 50, an adaptive display more suitable for the user 50 can be realized. When the user 50 is young, the time it takes to adapt to changes in brightness is relatively short, so by shortening the display time of the adaptive display pattern, the waiting time until the display device 12 can be removed can be shortened and a decrease in convenience can be prevented. On the other hand, when the user 50 is old, the time it takes to adapt to changes in brightness is relatively long, so by lengthening the display time of the adaptive display pattern, the strain on the eyes of the user 50 viewing the adaptive display image can be reduced.

According to this embodiment, by adjusting the display time of the adaptation display pattern according to the usage time of the display device 12 by the user 50 (i.e., the elapsed time since the start of usage), it is possible to realize an adaptation display that is more suitable for the user 50. In a situation where the user 50 is considered to have a high degree of eye fatigue due to the long usage time of the display device 12, the burden on the user 50 viewing the adaptation display image can be reduced by lengthening the display time of the adaptation display pattern. On the other hand, in a situation where the user 50 is considered to have a low degree of eye fatigue due to the short usage time of the display device 12, the waiting time until the display device 12 can be removed can be shortened, and a decrease in convenience can be prevented.

According to this embodiment, by determining the time rate of change in brightness of the adaptive display image in the adaptive display pattern according to the age and usage time of the user 50, it is possible to realize an adaptive display that is more suitable for the user 50. When the difference in brightness between the inside and outside of the display device 12 at the end of use is small, the display time of the adaptive display pattern can be shortened by determining the display time based on the determined time rate of change. Furthermore, when the difference in brightness between the inside and outside of the display device 12 at the end of use is very large, using a fixed display time can prevent the time rate of change of the adaptive display pattern from becoming excessively large, which would cause a large strain on the eyes of the user 50.

According to this embodiment, when the internal illuminance value is greater than the external illuminance value, the time rate of change in brightness of the image in the adaptation display pattern is made smaller than when the internal illuminance value is smaller than the external illuminance value, thereby realizing an adaptation display that is more suitable for the user 50. In other words, by making the time rate of change in brightness smaller and the display time relatively longer in the case of dark adaptation compared to the case of light adaptation, it is possible to prevent the burden on the eyes of the user 50 from becoming too great in the case of dark adaptation, which tends to take a long time to adapt.

Figure 9 is a block diagram showing a schematic functional configuration of a display system 10A according to another embodiment. The display system 10A includes a display device 12, a control device 14A, and an input device 16. The display device 12 and the input device 16 are configured similarly to the above-described embodiment. The control device 14A differs from the above-described embodiment in that it further includes a display analysis unit 60A, a gaze analysis unit 62A, and a fatigue estimation unit 64A. Below, the present embodiment will be described focusing on the differences from the above-described embodiment, and the commonalities will be omitted as appropriate.

The control device 14A includes a display control unit 32, an illuminance acquisition unit 34, an image recognition unit 36, an input acceptance unit 38, an age acquisition unit 40, a time measurement unit 42, a display analysis unit 60A, a gaze analysis unit 62A, a fatigue estimation unit 64A, an end detection unit 44, and an adaptation display determination unit 46A. The display control unit 32, the illuminance acquisition unit 34, the image recognition unit 36, the input acceptance unit 38, the age acquisition unit 40, the time measurement unit 42, and the end detection unit 44 are configured in the same manner as in the above-described embodiment.

The display analysis unit 60A analyzes the brightness of the image displayed on the display unit 22 and calculates a brightness score indicating whether the brightness of the image is appropriate. For example, the display analysis unit 60A calculates the average value of the brightness of the image displayed on the display unit 22 for each predetermined time interval (for example, 10 seconds or 1 minute) and calculates the section score based on the average value. The display analysis unit 60A calculates the brightness score by averaging the calculated section scores over the playback time of the video content. For example, if the average value of the brightness in the time interval is within a predetermined range and the brightness of the image is appropriate, a low score (for example, 1) is set, if the average value is lower than the predetermined range and the image is too dark, a medium score (for example, 3), and if the average value is higher than the predetermined range and the image is too bright, a high score (for example, 5). For example, if the section scores in the five intervals are 1, 1, 3, 1, and 5, the brightness score S1 = (1 + 1 + 3 + 1 + 5) / 5 = 2.2. If the average value falls outside the specified range, the score may be changed according to the difference between the average value and the lower or upper limit of the specified range, and the score may be increased as the difference between the average value and the lower or upper limit is greater. In other words, the section score may be increased as the average brightness value of the image deviates from the appropriate range.

The gaze analysis unit 62A analyzes the change in the gaze direction of the user 50 and calculates a gaze change score indicating the amount of gaze movement. For example, the gaze analysis unit 62A acquires the gaze direction or gaze position of the user 50 estimated by the image recognition unit 36 every predetermined time period (for example, 0.1 seconds or 1 second), and calculates the amount of change or movement from the gaze direction or gaze position acquired immediately before. The gaze analysis unit 62A calculates the gaze change score by integrating the calculated amount of change or movement over the playback time of the video content. For example, if the amount of gaze movement is less than a first reference value, a low score (for example, 1) is set, if the amount of gaze movement is equal to or greater than the first reference value and less than the second reference value, a medium score (for example, 3), and if the amount of gaze movement is equal to or greater than the second reference value, a high score (for example, 5). Note that the score may be changed continuously according to the amount of gaze movement, or the score may be increased as the amount of gaze movement increases.

The fatigue estimation unit 64A estimates the fatigue level of the user 50 based on at least one of the luminance score S1 calculated by the display analysis unit 60A and the gaze change score S2 calculated by the gaze analysis unit 62A. The fatigue estimation unit 64A calculates the fatigue level so that the greater the luminance score S1, the greater the fatigue level, and the greater the gaze change score S2, the greater the fatigue level. The fatigue estimation unit 64A may calculate the fatigue level score S3 = S1 + S2 by simply adding the luminance score S1 and the gaze change score S2. The fatigue estimation unit 64A may calculate the fatigue level score S3 = k S1 + (1 - k) S2 by taking a weighted average of the luminance score S1 and the gaze change score S2 with a predetermined weighting coefficient k (0 < k < 1).

The fatigue estimation unit 64A may calculate the fatigue level for a recent, relatively short, specified time (e.g., 5 or 10 minutes) rather than calculating the fatigue level for the entire playback time of the video content (e.g., 1 or 2 hours). The fatigue estimation unit 64A may calculate the fatigue level score S3 using the luminance score S1 for the recent, specified time calculated by the display analysis unit 60A. Similarly, the fatigue estimation unit 64A may calculate the fatigue level score S3 using the gaze change score S2 for the recent, specified time calculated by the gaze analysis unit 62A.

When calculating the fatigue level for the most recent specified time, the fatigue estimation unit 64A may calculate and store a fatigue level score S3 for each specified time interval during playback of the video content. After the end detection unit 44 detects the end of use of the display device 12, the fatigue estimation unit 64A may calculate the average of the fatigue level scores S3 for each specified time interval for a relatively short specified time (e.g., 5 minutes or 10 minutes) before the end.

The fatigue estimation unit 64A may adjust the fatigue level score S3 according to the age a of the user 50 acquired by the age acquisition unit 40. The fatigue estimation unit 64A may, for example, add an additional value α _SA according to the age a to the fatigue level score S3. The additional value α _SA can be set to a larger value as the age increases, similar to the above-mentioned additional value α _A. The fatigue estimation unit 64A may multiply the fatigue level score S3 by a multiplier value β _SA according to the age a. The multiplier value β _SA according to the age a can be set to a larger value as the age a increases.

The fatigue estimation unit 64A may adjust the fatigue level score S3 according to the elapsed time b acquired by the time measurement unit 42. The fatigue estimation unit 64A may, for example, add an additional value α _SB according to the elapsed time b to the fatigue level score S3. The additional value α _SB can be set so that the value increases as the elapsed time b becomes longer, similar to the above-mentioned additional value α _B. The fatigue estimation unit 64A may multiply the fatigue level score S3 by a multiplier value β _SB according to the elapsed time b. The multiplier value β _SB according to the elapsed time b can be set so that the value increases as the elapsed time b becomes longer.

The adaptive display determination unit 46A determines an adaptive display pattern that determines the change in brightness of the image to be displayed on the display unit 22 after the end is detected by the end detection unit 44. The adaptive display pattern determines, for example, an initial value p of the image brightness, an end value q of the image brightness, and a display time t required for the brightness to change from the initial value p to the end value q. The adaptive display determination unit 46A can determine the initial value p and the end value q in the same way as the adaptive display determination unit 46 in the above-described embodiment.

The adaptation display determination unit 46A determines the adaptation display pattern according to the fatigue level estimated by the fatigue estimation unit 64A. The adaptation display determination unit 46A determines the display time t or the time change rate Δ in the adaptation display pattern according to the fatigue level score S3 calculated by the fatigue estimation unit 64A. For example, the adaptation display determination unit 46A increases the display time t or decreases the time change rate Δ as the fatigue level score S3 increases. The relationship between the fatigue level score S3 and the display time t or the time change rate Δ may be determined by a table or by a formula using an arbitrary function.

Figure 10 is a flowchart showing a display method according to another embodiment. The display control unit 32 causes the display unit 22 to display an image (step S30). The display analysis unit 60A analyzes the brightness of the image displayed on the display unit 22 based on the operation of the display control unit 32 (step S32). The gaze analysis unit 62A analyzes the gaze movement amount based on the gaze direction of the user 50 recognized by the image recognition unit 36 (step S34). The fatigue estimation unit 64A estimates the fatigue level of the user 50 based on the analysis results of at least one of the display analysis unit 60A and the gaze analysis unit 62A (step S36). If the end detection unit 44 does not detect the end (No in step S38), the processing of S30 to S36 is continued. If the end detection unit 44 detects the end (Yes in step S38), the illuminance acquisition unit 34 acquires an internal illuminance value and an external illuminance value (step S40). The adaptation display determination unit 46A determines an adaptation display pattern based on the acquired internal illuminance value and external illuminance value and the estimated fatigue level (step S42). The display control unit 32 causes the display unit 22 to display an adaptation display image whose brightness changes according to the determined adaptation display pattern (step S44).

According to this embodiment, the degree of fatigue of the user 50 caused by the use of the display device 12 by the user 50 is estimated, and the display time of the adaptation display pattern is adjusted according to the degree of fatigue, thereby realizing an adaptation display more suitable for the user 50. In a situation where the brightness score S1 is high because the image displayed on the display unit 22 is too dark or too bright, and the user 50's eye fatigue is considered to be high, the burden on the user 50 viewing the adaptation display image can be reduced by lengthening the display time of the adaptation display pattern. In a situation where the line of sight change score S2 is high because the movement of the video content is large, and the user 50's eye fatigue is considered to be high, the burden on the user 50 viewing the adaptation display image can be reduced by lengthening the display time of the adaptation display pattern. On the other hand, in a situation where the fatigue score S3 is low and the user 50's eye fatigue is considered not to be high, the waiting time until the display device 12 can be removed can be shortened, and a decrease in convenience can be prevented.

When an image is generated based on video data for playing video content, the gaze analysis unit 62A may calculate a gaze change score indicating the amount of gaze movement based on a motion vector included in the video data. That is, the gaze analysis unit 62A calculates a score such that the gaze change score increases as the magnitude of the motion vector increases. The gaze analysis unit 62A may calculate a score such that the gaze change score increases as the amount of data of the motion vector included in the video data increases. For example, when the movement of an object in the video is small in the video content or the change between video frames is small, the gaze analysis unit 62A estimates that the movement of the video content is small. When the gaze analysis unit 62A estimates that the movement of the video content is small, the gaze analysis unit 62A calculates the gaze change score S2 to be lower than a predetermined value. When the movement of an object in the video is large or the change between video frames is large, the gaze analysis unit 62A estimates that the movement of the video content is large. When the fatigue estimation unit 64A estimates that the movement of the video content is large, the gaze change score S2 is calculated to be higher than a predetermined value. In this case, the image recognition unit 36 does not need to estimate the gaze direction of the user 50. Additionally, the display device 12 does not need to include an internal camera 28, and the control device 14A does not need to include an image recognition unit 36.

The fatigue estimation unit 64A may estimate the fatigue level of the user 50 based on the biometric information of the user 50 acquired by a biometric sensor, instead of the fatigue level of the user 50's eyes. The fatigue estimation unit 64A may estimate the fatigue level of the user 50 based on motion information indicating the motions performed by the user 50 in the virtual space, and may estimate the fatigue level according to, for example, the distance traveled by the user 50 in the virtual space or the amount of exercise in which the user 50 moves his or her body (head, arms, torso, legs, etc.). The fatigue estimation unit 64A may identify the fatigued body part of the user 50 based on the biometric information or motion information of the user 50, and may change the adaptation display pattern or change the image displayed on the display unit 22 according to the fatigued body part of the user 50.

The present invention has been described above with reference to the above-mentioned embodiments, but the present invention is not limited to the above-mentioned embodiments, and appropriate combinations or substitutions of the configurations shown in each display example are also included in the present invention.

Several aspects of the present invention are described below.

A first aspect of the present invention is a display system that includes a display unit that is provided inside a housing that is worn on the user's head and that displays an image toward the user, a time measurement unit that measures the elapsed time from the start of display by the display unit, an end detection unit that detects the end of display by the display unit, and an adaptive display determination unit that determines an adaptive display pattern that reduces the difference between an internal illuminance value that indicates the brightness inside the housing and an external illuminance value that indicates the brightness outside the housing by changing the brightness of the image to be displayed on the display unit, in accordance with the elapsed time from the start to the end of display by the display unit.

In the first aspect, the adaptive display determination unit may determine the adaptive display pattern further depending on the age of the user.

In a first aspect, the adaptive display determination unit may determine a rate of change in brightness over time in the adaptive display pattern in accordance with at least one of the elapsed time and the age of the user.

In a first aspect, the adaptation display determination unit may reduce the time rate of change of brightness in the adaptation display pattern when the internal illuminance value is greater than the external illuminance value, compared to when the internal illuminance value is less than the external illuminance value.

The first aspect may be provided as a display method. This method may include the steps of: displaying an image on a display unit provided inside a housing worn on a user's head; measuring the elapsed time from the start of display by the display unit; detecting the end of display by the display unit; and determining an adaptation display pattern that reduces the difference between an internal illuminance value indicating the brightness inside the housing and an external illuminance value indicating the brightness outside the housing by changing the brightness of the image displayed on the display unit, according to the elapsed time from the start to the end of display by the display unit.

The first aspect may be provided as a program or a non-transitory recording medium storing a program. This program may be configured to cause a computer to realize a function of displaying an image on a display unit provided inside a housing worn on a user's head, a function of measuring the elapsed time from the start of display by the display unit, a function of detecting the end of display by the display unit, and a function of determining an adaptation display pattern that reduces the difference between an internal illuminance value indicating the brightness inside the housing and an external illuminance value indicating the brightness outside the housing by changing the brightness of the image displayed on the display unit, according to the elapsed time from the start to the end of display by the display unit.

A second aspect of the present invention is a display system including a display unit that is provided inside a housing worn on the user's head and displays an image to the user, a fatigue estimation unit that estimates the user's level of fatigue, an end detection unit that detects the end of display by the display unit, and an adaptive display determination unit that determines an adaptive display pattern that reduces the difference between an internal illuminance value indicating the brightness inside the housing and an external illuminance value indicating the brightness outside the housing by changing the brightness of the image displayed on the display unit over time according to the level of fatigue estimated by the fatigue estimation unit.

In a second aspect, the fatigue estimation unit may estimate the user's fatigue level based on the brightness of the image displayed on the display unit.

In a second aspect, the fatigue estimation unit may estimate the user's fatigue level based on the movement of content included in the image displayed on the display unit.

In a second aspect, the fatigue estimation unit may estimate the user's fatigue level further based on at least one of the user's age and the elapsed time from the start to the end of display by the display unit.

The second aspect may be provided as a display method. This method may include the steps of displaying an image on a display unit provided inside a housing worn on a user's head, estimating the user's fatigue level, detecting the end of display by the display unit, and determining an adaptation display pattern in accordance with the estimated fatigue level, which reduces the difference between an internal illuminance value indicating the brightness inside the housing and an external illuminance value indicating the brightness outside the housing by changing the brightness of the image displayed on the display unit.

The second aspect may be provided as a program or a non-transitory recording medium storing a program. This program may be configured to cause a computer to realize a function of displaying an image on a display unit provided inside a housing worn on a user's head, a function of estimating the user's fatigue level, a function of detecting the end of display by the display unit, a function of determining an adaptation display pattern according to the estimated fatigue level, which reduces the difference between an internal illuminance value indicating the brightness inside the housing and an external illuminance value indicating the brightness outside the housing by changing the brightness of the image displayed on the display unit, and a function of controlling the display of the display unit according to the determined adaptation display pattern when the end of display by the display unit is detected.

10, 10A...display system, 12...display device, 14, 14A...control device, 16...input device, 20...housing, 22...display unit, 24...external illuminance sensor, 26...internal illuminance sensor, 32...display control unit, 34...illuminance acquisition unit, 36...image recognition unit, 38...input acceptance unit, 40...age acquisition unit, 42...time measurement unit, 44...end detection unit, 46, 46A...adaptation display determination unit, 50...user, 60A...display analysis unit, 62A...gaze analysis unit, 64A...fatigue estimation unit.

Claims (5)

A display unit that is provided inside a housing that is worn on the user's head and displays an image toward the user;
a time measurement unit that measures an elapsed time from the start of display by the display unit;
an end detection unit that detects the end of display by the display unit;
and an adaptive display determination unit that determines an adaptive display pattern that reduces the difference between an internal illuminance value indicating the brightness inside the housing and an external illuminance value indicating the brightness outside the housing by changing the brightness of an image to be displayed on the display unit, in accordance with the elapsed time from the start to the end of display by the display unit. The display system according to claim 1, wherein the adaptive display determination unit determines the adaptive display pattern further depending on the age of the user. The display system according to claim 1, wherein the adaptive display determination unit determines a time rate of change in brightness in the adaptive display pattern according to at least one of the elapsed time and the age of the user. The display system according to any one of claims 1 to 3, wherein the adaptation display determination unit reduces the time rate of change of brightness in the adaptation display pattern when the internal illuminance value is greater than the external illuminance value, compared to when the internal illuminance value is less than the external illuminance value. A step of displaying an image on a display unit provided inside a housing that is worn on a user's head;
measuring an elapsed time from the start of display by the display unit;
detecting an end of display by the display unit;
and determining an adaptation display pattern that reduces a difference between an internal illuminance value indicating the brightness inside the housing and an external illuminance value indicating the brightness outside the housing by changing the brightness of an image to be displayed on the display unit, in accordance with the elapsed time from the start to the end of display by the display unit.

Images