JP6432159B2 - Information display device, information display method, and information display program - Google Patents

Description

  The present invention relates to an information display apparatus, an information display method, and an information display program for presenting information such as images and characters to a user, such as an electronic book, a tablet terminal, and a smartphone.

  In recent years, portable information display devices such as electronic books, tablets, and smartphones have become widespread, and the movement of replacing some or all of conventional paper media such as books and newspapers with these portable information display devices is rapidly progressing. Unlike a paper medium, a portable information display device can change the information presented by touch panel or keyboard operation at any time, and has multimedia properties that can display not only still images but also games and videos. It is a feature. In recent years, portable information display devices having a display medium with very high resolution and high brightness have been sold, and the improvement in image quality has been remarkable.

  One of the problems in browsing information through a portable information display device is reflection (reflection) of the surrounding environment on a so-called gloss type display screen. In the so-called gloss type display screen, the user's own face, surrounding objects, fluorescent lamps used as illumination, etc. are reflected on the display medium, reducing the visibility of the information presented. Another problem is that the apparent brightness of the display object to be seen decreases when the user's viewpoint is away from the vertical direction of the display medium due to the angle dependence of the display brightness of portable information information display devices such as liquid crystals. The point of doing.

  Regarding the decrease in visibility due to the reflection of the surrounding environment, for example, there is one that takes measures to prevent the reflection on the display medium by providing a light shielding hood (for example, Patent Document 1), but handle it by hand. Attaching the above-mentioned apparatus with a portable information display device that often has a problem causes a drop in convenience. In addition, portable information display devices often receive illumination light from directly above the display surface, and it is difficult to physically block the illumination light.

  Moreover, although there exists a method of suppressing reflection by sticking an antireflection film (for example, patent document 2) to the screen part of a portable information display device, the film of a low reflectivity which prevents all the reflections of illumination is implement | achieved. Things are difficult.

  In addition, there is a so-called non-glare type screen that irregularly reflects incident light by providing fine irregularities on the screen surface and suppresses reflections. Inferior to other display devices.

  There is a display device that takes measures to reduce the angular dependency of luminance with respect to a decrease in apparent brightness of a display image due to a change in viewpoint (for example, Patent Document 3). It is not possible to prevent the decline of the level.

JP 2012-161055 A Japanese Patent No. 3332482 JP 2008-90324 A

  As described above, the portable information display device has a problem that the visibility of the presented information is reduced due to the surrounding reflection and the apparent brightness is reduced depending on the surrounding environment and the viewpoint position.

  The present invention has been made in view of such a problem, and is accompanied by an apparent reflection of an image / character, such as an electronic book, a tablet / smartphone, or the like having a gloss type display screen, and a change in the viewpoint direction. Provided are an information display device, an information display method, and an information display program that compensate for a decrease in luminance. Hereinafter, all display information displayed by the information display device, such as still images, moving images, and characters, is referred to as “image”.

One aspect of the present invention is an information display device that displays image information, the display, a means for acquiring a luminance distribution of ambient light incident on the display surface as an image, and a user's viewpoint direction with respect to the display surface Means for calculating the luminance distribution of the reflection on the display in the user's viewpoint direction from the luminance distribution of the ambient light and the user's viewpoint direction, and the luminance of the reflection for the image to be presented A means for calculating a reflection compensation image by adding a bias to the result obtained by subtracting the distribution image, a means for displaying the reflection compensation image on the display, and a bias if the bias is larger than a predetermined value, a predetermined notification is given to the user. Means .

Alternatively, the information display device includes a display, a means for acquiring the luminance distribution of ambient light incident on the display surface as an image, a means for specifying the user's viewpoint direction with respect to the display surface, the luminance distribution of the ambient light, and the user. The luminance distribution of the reflection on the display device in the user's viewpoint direction as an image from the viewpoint direction of the user, and the relative luminance ratio with respect to the display luminance when observed from directly above the display in the user's viewpoint direction Means for calculating and means for calculating a luminance attenuation and reflection compensation image obtained by multiplying an image obtained by subtracting a reflection luminance distribution image from an image to be presented and adding a bias to the reciprocal of the relative luminance ratio. and the case means and the bias for displaying the compensated image narrowing luminance attenuation and reflection on the display is larger than the predetermined value, Tokoro the user And means for performing notification.

Of the possible combinations of the group of images desired to be presented and the reflected brightness distribution image, 0 is the largest value among the values obtained by subtracting the brightness of the image desired to be presented from the reflected brightness indicated by the reflected brightness distribution image. A means for calculating a larger value as a bias may be further provided.

  The means for acquiring the luminance distribution of ambient light as an image may include a fisheye lens and a camera.

  Another aspect of the present invention is an information display method executed by a processor of an information display device and an information display program executed by the processor of the information display device in order to realize the above-described means.

  According to the present invention, in a portable information display device such as an electronic book, a tablet, or a smartphone, a terminal user can compensate for a decrease in apparent brightness caused by a surrounding reflection or a viewpoint change, and a user can view an image as intended. The effect that it can be made to appreciate is obtained.

The block diagram which shows the structure of embodiment of this invention Configuration diagram of portable information display device as one embodiment of the present invention Example of brightness distribution image around the display area obtained with a camera with a fisheye lens Explanatory drawing which shows the reflection range in the luminance distribution image around a display part calculated from a terminal user viewpoint The flowchart which shows the process in embodiment of this invention Explanatory drawing which shows a terminal user's viewpoint position in the luminance distribution image around a display part

  Hereinafter, an information display device according to an embodiment of the present invention will be described with reference to the drawings.

(Device configuration)
FIG. 1 is a block diagram showing the configuration of this embodiment. In FIG. 1, reference numeral 10 denotes an information display device according to the present embodiment.
Reference numeral 11 denotes a computer.
Reference numeral 12 denotes an operation unit.
Reference numeral 13 denotes an image compensation unit.
Reference numeral 14 denotes a display unit.
Reference numeral 15 denotes an ambient luminance distribution acquisition unit.
Reference numeral 16 denotes a compensation calculation unit.
Reference numeral 17 denotes a storage unit.
Reference numeral 20 denotes a user who operates the information display device 10 via the operation unit 12 and views an image through the display unit 14.
The computer 11 has a storage device and a computing device, receives an operation from the user through the operation unit 12 storing the image, and sends the presented image data to the compensation computing unit according to the user's request.
The image compensation unit 13 compensates based on the ambient luminance distribution data from the ambient luminance distribution acquisition unit 15, the presentation image data from the computer 11, the display device refractive index data from the storage unit 17, and the relative luminance ratio data for each viewpoint position. It has a compensation calculation unit 16 that performs calculation, and sends compensated image data to the display unit 14.
The display unit 14 is a liquid crystal display or the like and displays post-compensation image data sent from the image compensation unit 13.

(Typical example)
FIG. 2 is a configuration diagram of a portable information display device as a typical example of the information display apparatus 10 of FIG. The portable information display device 30 includes a camera 31 with a fisheye lens and a display 32.
A camera 31 with a fisheye lens corresponds to the ambient luminance distribution acquisition unit 15.
The display 32 also serves as the display unit 14 and the operation unit 12 that receives an operation of the terminal user 51 by a touch panel operation.
The portable information display device 30 has a built-in computer and serves also as the computer 11 and the image compensation unit 13 in the embodiment. The terminal user 51 can perform operations such as selecting an image to be viewed using the touch panel function of the display 32.

(principle)
Next, the principle of presenting an image compensated for reflection to the user 20 using the information display device 10 will be described.

  When the user 20 appreciates the image information presented on the display unit 14 of the information display device 10, the actual appearance on the display device is expressed by the equation (Equation 1).

Here, (x, y) represents the x coordinate and the y coordinate on the display unit 14.
ω is the origin on the display unit 14, the angle formed with the normal direction of the display unit 14 is the zenith angle, the perpendicular formed with the display unit 14 and the angle formed with the reference line taken on the plane of the display unit 14 is azimuthed The viewpoint direction of the user 20 is shown.
P (x, y, ω) represents the luminance distribution on the display unit 14 that the user 20 actually appreciates.
I (x, y) represents the luminance distribution of the image information output from the information display device 10.
k (ω) represents the relative intensity (brightness) ratio of the luminance of the display device in the viewpoint direction of the user 20, and is 1.0 when viewing from the vertical direction of the display device.
R (x, y, ω) represents the luminance distribution of the reflection that the light incident on the display unit is regularly reflected and can be seen by the terminal user.

  In this way, the surrounding reflections are superimposed on the image information originally intended to be presented, and the user 20 appreciates the display brightness with a decrease in display luminance due to a change in viewpoint direction.

  As described above, with respect to the apparent image degradation, the information display apparatus 10 performs the compensated image I ′ (x, y) expressed by the equation (2) with respect to the image I (x, y) that is actually desired to be presented. , Ω).

  At this time, the apparent luminance distribution that the user 20 appreciates is expressed by the equation (Equation 3).

  Here, P ′ (x, y, ω) represents a luminance distribution on the display unit 14 that the user 20 actually appreciates when the compensated image is output. That is, by outputting an image that has been compensated in advance based on the formula (Equation 2), the user 20 cancels the reflection on the screen surface and the reduced luminance, and the luminance distribution of the image that is originally desired to be presented It becomes possible to appreciate.

(Prediction of reflection)
In order to correct the presented image based on the formula (Equation 2), the luminance distribution R (x, y, ω) of the reflection on the display unit 14 observed at the viewpoint position viewed by the user 20, In addition, it is necessary to predict the relative luminance ratio k (ω) of the luminance of the display device according to the change in viewing angle.

  In order to achieve the above, the luminance distribution of the light applied to the display unit 14 is obtained by measurement from the periphery on the display unit 14. This is possible by performing imaging using a camera equipped with a lens for condensing light on a hemisphere covering the display unit 14 such as a fisheye lens, as shown in FIG. The image acquired by the camera is appropriately coordinate-transformed, and as shown in FIG. 3, the luminance distribution around the terminal is plotted with the zenith angle on the horizontal axis and the azimuth angle on the vertical axis, with the camera as the origin. Such an image can be developed.

  Furthermore, the position of the viewpoint of the user 20 with respect to the display unit 14 is measured. This can be achieved, for example, by performing viewpoint detection using an image recognition technique or the like using a luminance distribution image around the terminal.

  As shown in FIG. 4, the range of the reflection that can be seen by the user 20 is determined from the viewpoint position by obtaining the luminance distribution of the light irradiating the display unit 14 and the viewpoint position information of the user 20. This can be done by specifying the range surrounded by the regular reflection direction at the point corresponding to the outermost periphery of the display unit 14.

  The reflection luminance distribution R (x, y, ω) is calculated from the surrounding luminance distribution on the reflection range and the reflectance on the surface of the display unit 14. For example, if the refractive index of the material on the surface of the display unit 14 is known, the reflectance of the surface of the display unit 14 viewed from the viewpoint position can be calculated using the Fresnel formula. The reflection luminance distribution R (x, y, ω) can be expressed by the equation (4).

Here, r (x, y, ω) represents the reflectance of the display unit 14 surface.
L (x, y, ω) represents the luminance distribution around the display unit 14 on the reflection range.

(Acquisition of relative luminance ratio)
The luminance relative ratio k (ω) of the display device is measured in advance from various directions using a radiance meter or the like, and the result is stored in the storage unit 17. By referring to the list of luminance relative ratios stored in the storage unit 17 from the detected viewpoint direction ω, the luminance relative ratio k (ω) in the viewpoint direction at the time of viewing can be acquired.

(Expansion of brightness range that can compensate for reflection by applying bias)
In order to cancel apparent image degradation by generating a compensated image based on the above principle, the luminance I (x, y) of the image originally desired to be presented must be larger than the luminance R (x, y, ω) of the reflection. It is clear from the equation (Equation 2). In order to perform the reflection compensation at all coordinates on the display unit 14, as shown in the equation (5), by applying a bias, the luminance range of the reflection that can be compensated is expanded.

Here, B is a bias applied uniformly to the entire presented image. The applied bias B may be set in advance by the user 20, for example, or may be calculated based on the equation (Equation 6) when it is desired to compensate all the images to be presented.

Here, i represents an image number when there are a plurality of images to be presented.
The function max (·) represents the maximum value that can be taken.
In other words, the luminance distribution R (x, y, ω) of all the reflected images that can occur in accordance with the movement range of the viewpoint of the user 20 and all the images I _i (x, y) to be presented are determined for each pixel. The difference may be calculated and the largest value may be used as the applied bias.

  If the applied bias B is too large, the contrast of the image P ′ (x, y, ω) observed by the user 20 is lowered, and the image quality required by the user 20 may not be satisfied. Therefore, for example, an upper limit value of the applied bias B is set in advance, and when the calculated applied bias B exceeds the upper limit value, the user 20 is warned that it is out of the reflection compensation range, and then the applied bias B Measures may be taken such as urging the user 20 to cease at the upper limit set, or to darken the surrounding lighting.

(procedure)
Next, with reference to the flowchart of FIG. 5, an example of the operation until the information display device 10 shown in FIG. 1 presents an image in which surrounding reflections are compensated based on the above-described principle will be described.

  First, a luminance distribution image around the terminal is acquired (step S2). As illustrated in FIG. 2, this is acquired by performing imaging using a camera equipped with a fisheye lens in the immediate vicinity of the display unit 14. When the dynamic range of the image sensor mounted on the camera is narrower than the luminance range of the surrounding luminance distribution, so-called HDR (High Dynamic Range) imaging is performed, in which the exposure amount is changed in multiple steps and then combined.

  Next, a bias to be applied to the compensated image is calculated (step S3). If it is desired to be able to compensate for all the images to be presented, the calculation is performed based on the formula (Equation 6) based on the luminance distribution image around the terminal obtained in step S2.

  Next, it is determined whether or not the reflection compensation should be performed (step S4). This is determined based on whether or not the applied bias value calculated in step S3 is smaller than a preset threshold value. When the applied bias is larger than the threshold value, the display unit 14 displays that it is out of the compensation range and darkens the surrounding illumination (step S5), and ends (step S11).

  If it is determined in step S4 that it is within the reflection compensation range, then the line-of-sight direction is detected (step S6). For example, a face area is identified from a luminance distribution image around the terminal acquired in step S2 using a known image recognition method such as a cascade classifier based on Haar-like features, and a pattern matching or the like is performed from the face image area. The line-of-sight direction is calculated from the coordinates of the eyes by specifying the position of the eye. Here, as shown in FIG. 6, it is assumed that the viewpoint direction is representative of the midpoint of the right eye and left eye gaze direction.

  Next, the reflection range is calculated (step S7). This is based on the viewpoint position and the size of the known display device as shown in FIG. 4, and the reflection range at the viewpoint position of the user 20 in the luminance distribution image around the terminal acquired in step S2. calculate.

  Next, a reflection image is generated (step S8). This is based on the refractive index of the display medium stored in the storage unit 17 and the line-of-sight direction acquired in step S6, and the surface reflectance is calculated according to the Fresnel formula. A reflected image is generated by multiplying the luminance distribution in the specified reflection range by the surface reflectance.

  Next, a compensation process for the presented image is performed (step S9). For this, the relative luminance ratio at the viewpoint position acquired in step S6 is acquired from the relative luminance ratio table recorded in the storage unit 17, and the relative luminance ratio, the applied bias acquired in step S3, and acquired in step S7. Based on (Equation 5), the compensation calculation unit 16 performs a compensation process for the presented image using the reflected image.

  Next, the post-compensation presentation image acquired in step S9 is displayed (step S10), and the process ends (step S11).

  As described above, by compensating using the viewpoint and the surrounding environment information, it is possible to remove apparent visibility degradation such as reflection on the display terminal and luminance reduction.

  The above-described embodiment shows an example of the present invention. The present invention is not limited to this, and appropriate changes or modifications may be made without departing from the gist of the present invention.

  For example, in the above-described embodiment, the applied bias is calculated based on the luminance distribution image around the terminal in the usage environment. However, the present invention is not limited to this, and for example, a predetermined applied bias value is held in the storage unit 17. In addition, it may be used with reference to this, or a configuration without applying an applied bias may be adopted.

  Further, it is determined whether or not compensation should be performed, and if it is determined that compensation should not be performed, the fact is presented to the user 20 and the image display is terminated. However, the present invention is not limited to this, and the applied bias is determined in advance. Compensation processing may be performed only for pixels that can be compensated within the upper limit value stored in the storage unit 17.

  In addition, when the reflected image is generated, the surface reflectance is calculated using the refractive index of the display medium. However, the present invention is not limited to this. For example, when the angle is changed using a goniorireflector or the like. The surface reflectance of the display medium may be measured and stored in the storage unit 17, and the surface reflectance may be calculated by referring to the stored value.

  Moreover, in the information display apparatus 10, although the compensation calculating part was separated from the display calculating part, it is not limited to this, You may process by the same calculating apparatus.

  Further, in the information display device 10, the display unit 14 is configured to be integrated with the computer 11 or the image compensation unit 13. However, the present invention is not limited to this, and the luminance distribution around the display unit 14 can be acquired. If there is an ambient luminance distribution acquisition unit 15 capable of detecting the viewpoint direction, an arithmetic processing device placed in another environment can perform the same compensation processing of the presented image.

  Further, although the viewpoint direction is treated as one representative point, the present invention is not limited to this. For example, if the display unit 14 can present images for the right eye and the left eye, such as a 3D display, It is possible to calculate the position of each of the right eye and the left eye and perform the compensation process separately for the right eye and for the left eye.

  In addition, the relative luminance ratio k (ω) is measured in advance and referenced to compensate for the luminance attenuation. However, the present invention is not limited to this, and luminance attenuation due to a change in viewing angle is ignored. It is not necessary to compensate for luminance attenuation assuming that the relative luminance ratio is always 1 as small as possible.

  The functions and processes of the information display device described above can be realized by causing one or more processors included in the information display device to execute a program.

  The present invention is useful for an information display device that presents information such as images and characters to an user, such as an electronic book, a tablet terminal, and a smartphone.

DESCRIPTION OF SYMBOLS 10 Information display apparatus 11 Computer 12 Operation part 13 Image compensation part 14 Display part 15 Ambient luminance distribution acquisition part 16 Compensation calculating part 17 Storage part 20 User 30 Portable information display device 31 Camera with fisheye lens 32 Display 51 Terminal user S1 Compensation image Display processing start S2 Ambient luminance distribution image acquisition S3 Applied bias calculation S4 Judgment within image compensation execution range S5 "Out of compensation" presentation S6 View direction detection S7 Reflection range calculation S8 Reflection image generation S9 Compensated image generation S10 Image display S11 Compensation image display processing completed

Claims (11)

An information display device for displaying image information,
Display,
Means for acquiring as an image a luminance distribution of ambient light incident on the display surface;
Means for identifying a user's viewpoint direction with respect to the display surface;
Means for calculating, as an image, a luminance distribution of reflection on the display in the viewpoint direction of the user from the luminance distribution of the ambient light and the viewpoint direction of the user;
Means for calculating a reflection compensation image by adding a bias to a result obtained by subtracting the luminance distribution image of the reflection from an image to be presented;
Means for displaying the reflection compensation image on the display;
An information display device comprising: a unit that performs a predetermined notification to a user when the bias is greater than a predetermined value . An information display device for displaying image information,
Display,
Means for acquiring as an image a luminance distribution of ambient light incident on the display surface;
Means for identifying a user's viewpoint direction with respect to the display surface;
Means for calculating, as an image, a luminance distribution of reflection on the display device in the user's viewpoint direction from the luminance distribution of the ambient light and the viewpoint direction of the user;
Means for calculating a relative luminance ratio with respect to display luminance when observed from directly above the display in the user's viewpoint direction;
Means for calculating a luminance attenuation and reflection compensation image obtained by multiplying an image obtained by subtracting the reflection luminance distribution image from the image desired to be presented and a bias added to the reciprocal of the relative luminance ratio;
Means for displaying the luminance attenuation and reflection compensation image on the display;
An information display device comprising: a unit that performs a predetermined notification to a user when the bias is greater than a predetermined value . Of the possible combinations of the group of images desired to be presented and the luminance distribution image of the reflection, the most of the values obtained by subtracting the luminance of the image to be presented from the luminance of the reflection indicated by the luminance distribution image of the reflection It means for calculating a large value greater than 0 as the bias, further comprising information display device according to claim 1 or 2. The information display device according to any one of claims 1 to 3 , wherein the means for acquiring the luminance distribution of the ambient light as an image includes a fisheye lens and a camera. An information display method executed by a processor of an information display device that displays image information,
Obtaining a luminance distribution of ambient light incident on the display surface as an image;
Identifying a user's viewpoint direction with respect to the display surface;
Calculating the luminance distribution of reflection on the display device in the user's viewpoint direction as an image from the luminance distribution of the ambient light and the viewpoint direction of the user;
Calculating a reflection compensation image by adding a bias to the result of subtracting the reflection luminance distribution image from the image to be presented; and
Displaying the reflection compensation image on the display;
And a step of notifying a user of a predetermined notification when the bias is larger than a predetermined value . An information display method executed by a processor of an information display device that displays image information,
Obtaining a luminance distribution of ambient light incident on the display surface as an image;
Identifying a user's viewpoint direction with respect to the display surface;
Calculating the luminance distribution of reflection on the display device in the user's viewpoint direction as an image from the luminance distribution of the ambient light and the user's viewpoint direction;
Calculating a relative luminance ratio with respect to display luminance when observed from directly above the display in the user's viewpoint direction;
Calculating a luminance attenuation and reflection compensation image obtained by multiplying an image obtained by subtracting the reflection luminance distribution image from the image desired to be presented and adding a bias to the reciprocal of the relative luminance ratio;
Displaying the luminance attenuation and reflection compensation image on the display;
And a step of notifying a user of a predetermined notification when the bias is larger than a predetermined value . The processor subtracts the luminance of the image to be presented from the luminance of the reflection indicated by the luminance distribution image of the reflection, out of the possible combinations of the group of images to be presented and the luminance distribution image of the reflection. largest 0 calculating a greater value as the bias, further including, information display method according to claim 5 or 6 of the values. The information display method according to claim 5 , wherein a fisheye lens and a camera are used in the step of acquiring the luminance distribution of the ambient light as an image. An information display program to be executed by a processor of an information display device that displays image information,
Processing to acquire the luminance distribution of ambient light incident on the display surface as an image;
Processing for identifying the user's viewpoint direction with respect to the display surface;
A process of calculating the luminance distribution of reflection on the display device in the user's viewpoint direction as an image from the luminance distribution of the ambient light and the viewpoint direction of the user;
A process of calculating a reflection compensation image by adding a bias to the result of subtracting the reflection luminance distribution image from the image to be presented;
Processing for displaying the reflection compensation image on the display;
An information display program for causing the processor to execute a predetermined notification process to a user when the bias is larger than a predetermined value . An information display program to be executed by a processor of an information display device that displays image information,
Processing to acquire the luminance distribution of ambient light incident on the display surface as an image;
Processing for identifying a user's viewpoint direction with respect to the display surface;
A process of calculating a luminance distribution of reflection on a display device in the user's viewpoint direction as an image from the luminance distribution of the ambient light and the user's viewpoint direction;
A process of calculating a relative luminance ratio with respect to display luminance when observed from directly above the display in the user's viewpoint direction;
A process of calculating a luminance attenuation and reflection compensation image obtained by multiplying the image obtained by subtracting the luminance distribution image of the reflection from the image to be presented and adding a bias to the reciprocal of the relative luminance ratio;
Processing to display the brightness attenuation and reflection compensation image on the display;
An information display program for causing the processor to execute a predetermined notification process to a user when the bias is larger than a predetermined value . Of the possible combinations of the group of images desired to be presented and the luminance distribution image of the reflection, the most of the values obtained by subtracting the luminance of the image to be presented from the luminance of the reflection indicated by the luminance distribution image of the reflection The information display program according to claim 9 or 10 , further causing the processor to execute a process of calculating a larger value than 0 as the bias.

Images