WO2023228391A1

WO2023228391A1 - Portable information terminal and display control method for portable information terminal

Info

Publication number: WO2023228391A1
Application number: PCT/JP2022/021661
Authority: WO
Inventors: 眞弓中出; 達也山本; 仁秋山; 和之滝澤; 康宣橋本
Original assignee: マクセル株式会社
Priority date: 2022-05-27
Filing date: 2022-05-27
Publication date: 2023-11-30

Abstract

This portable information terminal recognizes the face of a user or an eye of the user on the basis of an image captured by a camera mounted thereto, acquires the distance from a distance sensor to a face of the user or to the eye of the user, and performs visibility reduction control for reducing the visibility of a display if a first elapsed time, from when the distance had become equal to or less than a predetermined eyesight reduction criterion distance, has become equal to or more than a start criterion time. In addition, if a second elapsed time, from when the distance to the face of the user or to the eye of the user had become more than the eyesight reduction criterion distance, has become equal to or more than an end criterion time, the visibility reduction control is ended and a normal display is restored.

Description

Mobile information terminal and display control method for mobile information terminal

The present invention relates to an information display device and a display control method, and more particularly to a mobile information terminal and a display control method for a mobile information terminal that calls attention to the effects of screen viewing on visual acuity.

There is a concern that staring at the screen of a mobile information device such as a smartphone for a long time may cause vision loss. In particular, young people such as children can easily focus their eyes even at close distances, so they become absorbed in games, videos, etc., putting strain on their eyes and causing visual acuity to deteriorate without them noticing. In particular, the closer the distance, the greater the influence.

Conventionally, there has been a problem with children watching TV nearby when watching TV, and as an example of a countermeasure, Patent Document 1 describes ``a distance measuring means for detecting the distance to a human body existing in a predetermined range in front of a display device.'' and an alarm control means for outputting an alarm when the measured distance detected by the distance measuring means becomes less than a preset limit distance (Summary excerpt)" is disclosed. There is.

Japanese Patent Application Publication No. 2010-44607

Since the television is large and heavy, the position of the television does not move while watching. Also, people rarely move around while watching TV. Therefore, as in Patent Document 1, since the distance between the person's position and the TV screen does not change very often, it is possible to determine whether or not to output an alarm using the distance between the person's position and the TV screen. It is.

On the other hand, with mobile information terminals such as smartphones, which are held in the hand and operated while viewing the screen, the distance between the screen and the eyes can be easily adjusted by moving the hand holding the smartphone, even if the person's position does not change. Changes to Therefore, the screen proximity warning device of Patent Document 1, which performs control based on the distance between a person's position and the screen, cannot be applied to mobile information terminals, and therefore, a technology for preventing visual acuity deterioration suitable for mobile information terminals is desired. .

The present invention has been made in view of the above-mentioned problems, and aims to provide a mobile information terminal and a display control method for a mobile information terminal that maintains an appropriate distance between the eyes and the screen.

In order to achieve the above object, the present invention includes the configurations described in the claims. To give one example, the present invention is a mobile information terminal, which includes a display, a camera, a distance sensor, a timer, and a processor, and the processor is configured to provide user information based on the image taken by the camera. recognizes the face of the user or the eyes of the user, obtains the distance from the distance sensor to the face of the user or the eyes of the user, and determines a decrease in visual acuity based on the predetermined distance to the face of the user or the eyes of the user. A first elapsed time from when the distance has fallen below is acquired from the timer, and when the first elapsed time becomes equal to or greater than a predetermined start determination time, visibility reduction control is performed to reduce the visibility of the display, and the A second elapsed time from when the distance to the user's face or the user's eyes exceeds the visual acuity deterioration determination distance is acquired from the timer, and when the second elapsed time exceeds a predetermined end determination time, the visual recognition is performed. The display is returned to normal display by completing the performance reduction control.

According to the present invention, it is possible to provide a mobile information terminal and a display control method for the mobile information terminal that maintain an appropriate distance between the eyes and the screen. Note that objects, configurations, and effects other than those described above will be made clear by the description of the embodiments below.

It is a front view of a smartphone. FIG. 3 is a rear view of the smartphone. It is a hardware configuration diagram of a smartphone. FIG. 2 is a functional block diagram according to the present embodiment. FIG. 3 is an explanatory diagram of distances measured in this embodiment. FIG. 2 is a diagram illustrating an overview of a visual acuity reduction prevention mode according to the present embodiment. It is a flowchart which shows the flow of processing of a 1st embodiment. It is a flow chart which shows the flow of visual acuity decline prevention processing in a 2nd embodiment. It is a figure which shows the example of a screen (size change) in which visibility reduction strength was changed according to the distance to an eye. It is a figure which shows the example of a screen (blur degree change) in which visibility reduction strength was changed according to the distance to an eye. It is a figure which shows the distance start determination time table. 7 is a flowchart showing the flow of processing according to a third embodiment. FIG. 7 is a diagram showing the relationship between the execution time of visibility reduction processing and visibility reduction intensity. It is a flow chart which shows the flow of processing concerning a 4th embodiment. It is a flowchart which shows the flow of processing concerning a 5th embodiment. It is a figure which shows another example of the screen during execution of a visual acuity decline prevention process. It is a figure which shows another example of hardware. It is an external view of a head mounted display. FIG. 2 is a hardware configuration diagram of a head-mounted display. FIG. 3 is a diagram showing an example of visual acuity reduction prevention processing using a head-mounted display. FIG. 2 is a diagram showing a system for transmitting a log of a child's viewing history to a parent.

Hereinafter, embodiments of a portable information terminal and a display control method thereof according to the present invention will be described with reference to the drawings. The same inventions are denoted by the same reference numerals throughout all the figures, and redundant explanation will be omitted.

The present invention is expected to contribute to the prevention of visual acuity deterioration caused by viewing on mobile information terminals. It can be expected to contribute to welfare.

In this embodiment, a smartphone will be described as a specific example of a mobile information terminal, but the mobile information terminal is not limited to a smartphone as long as it is a mobile information terminal that can be held in the hand and operated while viewing the screen. For example, a tablet is also an example of a portable information terminal to which the present invention can be applied. The present invention can also be applied to wearable terminals that do not need to be held in the hand, such as head-mounted displays and smart glasses.

FIG. 1A is a front view of the smartphone. FIG. 1B is a rear view of the smartphone.

As shown in FIG. 1A, the front of the smartphone 100 is equipped with an in-camera 131, a display 133, a speaker 141, and a distance sensor 153.

Further, as shown in FIG. 1B, an out camera 132 is provided on the back of the smartphone 100.

FIG. 2 is a diagram of the hardware configuration of the smartphone.

As shown in FIG. 2, the smartphone 100 includes a main processor 101, a storage device 110, an input interface (I/F) 120, an image processing device 130, an audio processing device 140, a sensor group 150, a communication I/F 160, and an expansion I/F 172. , and a timer 173, which are connected to each other via the bus 102.

The storage device 110 includes a RAM 111, a ROM 112, and a flash memory 113.

The input I/F 120 includes a button switch 121 and a touch panel 122.

The image processing device 130 includes an in-camera 131, an out-camera 132, and a display 133.

The audio processing device 140 includes a speaker 141 and a microphone 142.

The sensor group 150 includes a GPS receiver 151, a geomagnetic sensor 152, a distance sensor 153, an acceleration sensor 154, a gyro sensor 155, and a biological information acquisition sensor 156.

The communication I/F 160 includes a wireless communication I/F 161 and a telephone network communication I/F 162.

FIG. 3 is a functional block diagram according to this embodiment.

The smartphone 100 includes a face recognition section 101a, a visibility reduction processing section 101b, a timekeeping section 101c, a user authentication section 101d, and a mode setting section 101e. These are realized by the main processor 101 loading a vision reduction prevention program into the RAM 111 and executing it. Details of the functions of each part will be described later.

Further, the flash memory 113 is formed with a various table storage section 113a and a user registration information storage section 113b. The various table storage unit 113a records tables necessary for executing the visual acuity decline prevention program. The user registration information stored in the user registration information storage section 113b is used to execute a visual acuity reduction prevention program related to the shared terminal, which will be described later.

FIG. 4A is an explanatory diagram of distances measured in this embodiment. FIG. 4B is a diagram showing an overview of the visual acuity reduction prevention mode according to the present embodiment.

As shown in FIG. 4A, in this embodiment, the distance D from the display 133 of the smartphone 100 to the user's face or the user's eyes is measured, and the operation in the visual acuity reduction prevention mode is performed based on this distance D. A mode may be adopted in which the subject is recognized as the user (a person) without performing face recognition or eye recognition of the user.

Then, as shown in FIG. 4B, the main processor 101 determines that when the state (near state) in which the distance D is less than or equal to a predetermined distance (Dth: visual acuity deterioration determination distance) continues for a certain period of time (TS: start determination time), Visibility reduction processing is executed on the screen of the display 133 (also referred to as visibility reduction control). Visibility reduction processing involves blurring the screen or reducing the display size.

If the distance D continues to be farther than the visual acuity deterioration determination distance Dth for a certain period of time (TE: end determination time), the main processor 101 ends the visibility reduction process on the screen of the display 133 and returns the screen to the normal screen. Thereby, the display 133 is displayed in the display state before the visibility reduction process is performed.

<First embodiment>
The first embodiment is an embodiment in which visual acuity reduction prevention processing is performed on a smartphone operated by one user. FIG. 5 is a flowchart showing the process flow of the first embodiment.

When the visibility reduction prevention mode is turned on in the smartphone 100, the face recognition unit 101a executes face recognition processing on the image of the in-camera 131, recognizes the user's face or eye position, and uses the distance sensor 153 to detect the face or Measure the distance D to the eye (S01, see FIG. 4).

The visibility reduction processing unit 101b compares the distance D and the visual acuity deterioration determination distance Dth, and determines that the distance D is larger than the visual acuity deterioration determination distance Dth, that is, the eyesight of the user A is lower than the visual acuity deterioration determination distance Dth. If it is determined that it is farther than the distance Dth (S02: NO), the process returns to step S01 and continues measuring the distance D.

On the other hand, when the visibility reduction processing unit 101b determines that the distance D is less than or equal to the visual acuity deterioration determination distance Dth, that is, the user A's eyes are the same as or closer than the visual acuity deterioration determination distance Dth to the display 133 of the smartphone 100 (S02: YES), measurement of the first elapsed time T1 after the distance D becomes equal to or less than the visual acuity deterioration determination distance Dth is started (S03).

A timer 171 as hardware may be provided, and the timer 171 may be activated to measure the elapsed time and the execution time described below, or the timer 171 may be configured as software. In this case, the timer 101c may acquire the time at the start of measurement and the time at the end of measurement from the RTC, and measure the elapsed time and execution time from the difference between them.

If the first elapsed time T1 is shorter than the start determination time TS (S04: NO), the visibility reduction processing unit 101b returns to step S01 and continues measuring the distance and counting the first elapsed time T1.

When the first elapsed time T1 is equal to or longer than the start determination time TS (S04: YES), the main processor 101 initializes the first elapsed time T1 (S05) and The sex reduction process is started (S06).

The smartphone 100 continues measuring the distance D (S07), and the visibility reduction processing unit 101b compares the distance D with the visual acuity reduction determination distance Dth (S08).

When the visibility reduction processing unit 101b determines that the distance D is less than or equal to the visual acuity reduction determination distance Dth (S08: YES), the process returns to step S06 and continues the visibility reduction processing.

When the visibility reduction processing unit 101b determines that the distance D is farther than the visual acuity reduction determination distance Dth (S08: NO), the visibility reduction processing unit 101b determines the elapsed time (second elapsed time T2) after the distance D becomes equal to or less than the visual acuity reduction determination distance Dth. ) measurement is started (S09).

When the visibility reduction processing unit 101b determines that the second elapsed time T2 is shorter than the end determination time TE (S10: NO), the process returns to step S06 and continues the visibility reduction processing.

When the second elapsed time T2 becomes equal to or greater than the end determination time TE (S10: YES), the visibility reduction processing unit 101b stops the visibility reduction processing, switches to normal display, and initializes the second elapsed time T2 ( S11).

According to the present embodiment, when the start determination time TS has elapsed after the distance between the eyes and the mobile information terminal becomes less than or equal to the visual acuity deterioration determination distance, which is a distance at which deterioration of visual acuity is a concern, visibility reduction processing is performed; The user is alerted to the closeness of the display. In this case, since the visibility of the display is reduced, it is necessary to move the display away from the eyes in order to improve the display quality. As a result, simply notifying that the screen is close to the eyes has little hope of improving the distance between the eyes and the display, but according to this embodiment, the distance between the eyes and the display can be improved to restore the visibility of the screen. movement is required to maintain an appropriate distance between the user's eyes and the display.

Additionally, since mobile information terminals are operated in the hand, the distance D from the user's eyes is more likely to change than with a fixed display. According to the present embodiment, the visibility reduction process is not performed unless the distance D approaches the visual acuity deterioration determination distance Dth for a time equal to or longer than the start determination time TS, and the visibility reduction process does not occur unless the distance D approaches the visual acuity deterioration determination distance Dth. By not returning to the normal screen unless the end judgment time TE is exceeded, the distance between the eyes and the mobile information terminal is optimized while preventing visibility reduction processing from being performed frequently due to camera shake or slight changes in posture. can be achieved.

<Second embodiment>
The second embodiment is an embodiment in which the visibility reduction strength is varied according to the distance between the eyes and the mobile information terminal in the visual acuity reduction prevention process. FIG. 6 is a flowchart showing the flow of visual acuity reduction prevention processing in the second embodiment. Steps that are the same as those in the process flow of the first embodiment shown in FIG. 5 are given the same reference numerals, and redundant explanation will be omitted.

The visibility reduction processing unit 101b stops measuring the first elapsed time T1 when the first elapsed time T1 after the distance D becomes less than or equal to the visual acuity reduction determination distance Dth becomes equal to or greater than the start determination time TS (S05), The visibility reduction strength is set according to the distance D (S11). The lower the visibility reduction strength is, the smaller the distance D is below the visual acuity reduction determination distance Dth, the lower the visibility of the display 133 is, that is, the stronger the visibility reduction strength is.

The visibility reduction processing unit 101b starts visibility reduction processing according to the determined visibility reduction intensity (S06).

The distance D is measured during the visibility reduction process (S07), the visibility reduction processing unit 101b resets the visibility reduction intensity according to the distance D (S12), and the visibility reduction processing unit 101b resets the visibility reduction intensity according to the reset visibility reduction intensity. The performance reduction process is re-executed (S13). Depending on the performance and processing capacity of the control unit, the time required from measuring distance to setting the visibility reduction strength may exceed one frame, or the visibility reduction strength may be set at predetermined frame intervals to reduce processing load and power consumption. Processing may be performed. There is no particular limitation on the interval of the visibility reducing strength treatment.

If the visibility reduction strength is not updated, the visibility reduction strength of the previous frame is inherited.

The distance measurement interval may be changed during the visual acuity deterioration prevention process and other than the visual acuity deterioration prevention process. For example, the interval may be every 10 frames when the visual acuity reduction prevention process is not performed, and the interval may be 1 frame during the visibility reduction process. Before starting the visibility reduction process, followability is not required so much, so priority may be given to the effect of reducing the processing load.

When the visibility reduction processing unit 101b determines that the distance D is less than or equal to the visual acuity reduction determination distance Dth (S08: YES), the process returns to step S07 to measure the distance D, reset the visibility reduction intensity (S12), and perform the visibility reduction process. The re-execution of the process (S13) is repeated. This repeated process continues until the visibility reduction processing unit 101b determines that the distance D is greater than the visual acuity reduction determination distance Dth (S08: NO). Thereafter, measurement of the second elapsed time T2 is started (S09), and the second elapsed time T2 is compared with the end determination time TE (S10). If the second elapsed time T2 is less than the end determination time TE (S10: NO), the process returns to step S07. If the second elapsed time T2 is equal to or longer than the end determination time TE (S10: YES), the process advances to step S11, switches to normal display, and stops measuring the second elapsed time T2 (S11). After that, the process ends.

FIG. 7 is a diagram showing an example of a screen (change in size) in which the visibility reduction strength is changed depending on the distance to the eye.

In FIG. 7, the size of the display screen is changed depending on the degree of visibility reduction. That is, as shown in the distance screen size table 1333, when the distance D to the eyes is less than or equal to the visual acuity deterioration determination distance Dth, as the distance D becomes smaller, the screen size is changed to a small screen 1332 that is smaller than the standard size normal screen 1331. . When the distance D is equal to or greater than the visual acuity deterioration determination distance Dth, the normal screen 1331 is displayed regardless of the distance D.

In addition to changing the screen size, the visibility reduction process may include lowering the brightness of the display 133 as the distance D becomes smaller, or increasing the amplitude (intensity) of the vibrator and shaking the smartphone 100 to lower the visibility. . In addition, image processing is performed on the display 133 using a central projection method that uses the center of the screen as a reference point, without changing the screen size, and processing that reduces the overall visibility by lowering the visibility of the center of the screen. You may go. At this time, the center of the central projection method may be set farther away as the distance D becomes smaller. Further, a tactile device using a piezoelectric element or the like may be placed in a place where the smartphone 100 is held, and the smartphone 100 may be guided in the direction opposite to the eye by giving a pulling sensation due to pressure to the finger or hand.

FIG. 8 is a diagram showing an example of a screen (change in degree of blur) in which the intensity of visibility reduction is changed depending on the distance to the eye.

In FIG. 8, when the visibility reduction strength is 0 (minimum value), a normal screen 1331 that is not blurred at all is displayed. If the visibility reduction strength is greater than 0 (minimum value) and less than the maximum value, for example, medium, a slightly blurred moderately blurred screen 1334 is displayed. When the degree of visibility reduction increases, an intensely blurred screen 1335 with a high degree of blurring is displayed.

According to the present embodiment, by reducing visibility as the distance D becomes shorter during visibility reduction processing, it is possible to urge the user to ensure a distance D that is equal to or greater than the visual acuity reduction determination distance Dth.

<Third embodiment>
The third embodiment is an embodiment in which the start determination time TS until the visibility reduction process is started is changed according to the distance D. FIG. 9 is a diagram showing a distance start determination time table.

As shown in the distance start determination time table 1336 shown in FIG. 9, if the distance D is within the close range, the start determination time TS may be set to 0 seconds and the visibility reduction process may be started immediately. From beyond the close distance to the visual acuity deterioration determination distance Dth, the start determination time TS may be configured to become longer as the distance D becomes longer.

FIG. 10 is a flowchart showing the flow of processing according to the third embodiment.

As shown in FIG. 10, in the third embodiment, the visibility reduction processing unit 101b measures the distance D (S01), compares it with the visual acuity reduction determination distance Dth (S02), and starts measuring the first elapsed time T1 ( S03).

The visibility reduction processing unit 101b refers to the distance start determination time table 1336 and determines the TS corresponding to the distance D measured in step S01 (S21).

Then, the visibility reduction processing unit 101b compares the start determination time TS with the first elapsed time T1 (S04), and determines whether or not it is necessary to execute the visibility reduction processing. Thereafter, the process flow of the first embodiment continues to S05, and the process flow of the second embodiment continues to S11.

According to this embodiment, if the distance D is a close distance, the visibility reduction process can be started immediately. Furthermore, even if the distance D is smaller than the visual acuity deterioration determination distance Dth, the closer the visual acuity deterioration determination distance Dth is, the less strain on the eyes will be placed on the eyes, so the start determination time TS can be made longer to delay the start of the visibility deterioration process. Even if the user's eyes accidentally come close to the display, it is possible to prevent visibility reduction processing from being executed frequently and improve operability.

<Fourth embodiment>
The fourth embodiment is an embodiment in which the visibility reduction strength is increased as the execution time of the visibility reduction process runs out. FIG. 11 is a diagram showing the relationship between the execution time of the visibility reduction process and the visibility reduction intensity.

As shown in FIG. 11, the longer the execution time T3 after starting the visibility reduction process, the stronger the visibility reduction intensity. However, once the visibility reduction intensity reaches the maximum value, the visibility of the screen does not change thereafter even if the execution time T3 of the visibility reduction process becomes longer.

FIG. 12 is a flowchart showing the flow of processing according to the fourth embodiment. Steps that are the same as those in the process flow of the second embodiment are given the same reference numerals and redundant explanations will be omitted.

The visibility reduction processing unit 101b stops measuring the first elapsed time T1 when the first elapsed time T1 after the distance D becomes less than or equal to the visual acuity reduction determination distance Dth becomes equal to or greater than the start determination time TS (S05), Visibility reduction processing is started (S06). At this time, the visibility reduction intensity may be a visibility reduction intensity depending on the distance D as in S11, or a predetermined visibility reduction intensity.

When the visibility reduction process is started, the measurement of the execution time T3 (S31) and the measurement of the distance D (S07) are started.

The visibility reduction processing unit 101b resets the visibility reduction intensity according to the execution time T3 (S32), and re-executes the visibility reduction process according to the reset visibility reduction intensity (S33).

When the visibility reduction processing unit 101b determines that the distance D is less than or equal to the visual acuity reduction determination distance Dth (S08: YES), the process returns to step S31 and continues measuring the execution time T3, and repeats the subsequent steps. This repeated process continues until the visibility reduction processing unit 101b determines that the distance D is greater than the visual acuity reduction determination distance Dth (S08: NO). Thereafter, the measurement of the second elapsed time T2 is started (S09), and the visibility reduction processing unit 101b compares the second elapsed time T2 and the end determination time TE (S10). If the second elapsed time T2 is less than the end determination time TE (S10: NO), the process returns to step S31. If the second elapsed time T2 is equal to or longer than the end determination time TE (S10: YES), the display is switched to normal display and the measurement of the second elapsed time T2 and execution time T3 is stopped (S34). After that, the process ends.

According to the present embodiment, the visibility reduction intensity is increased according to the execution time T3 of the visibility reduction process, so even if the distance D is smaller than the visual acuity reduction determination distance Dth, even if the distance D is smaller than the visual acuity reduction determination distance Dth. In the case where the recovery is not made beyond the deterioration determination distance Dth, the user can be made aware of the recovery posture of the distance D by increasing the visibility.

<Fifth embodiment>
The fifth embodiment is an embodiment in which, in a shared terminal, on/off of the visual acuity reduction prevention mode is controlled according to the age of the user.

FIG. 13 is a flowchart showing the flow of processing according to the fifth embodiment.

The user's face is photographed with the in-camera of the smartphone 100 (S41), and the face recognition unit 101a performs face recognition processing (S42). The user authentication unit 101d compares the recognized user with user registration information that associates whether the visual acuity reduction mode is turned on or off for the user (S43).

If the user indicated by the face image is registered in the user registration information (S43: YES), the registered mode is set (S44) and the process ends. The registered mode here means registered content such that user A (for example, a child) turns on the visual acuity loss prevention mode, and user B (for example, a parent) turns off the visual acuity loss prevention mode.

If there is no user registration (S43: NO), the mode setting unit 101e performs a process of estimating the user's age based on the recognized face image (S44). When the mode setting unit 101e determines that the estimated age is less than or equal to the designated age (S45: YES), the mode setting unit 101e sets the visual acuity reduction prevention mode to ON (S46).

If the vision reduction prevention mode is automatically turned on, a button or the like may be displayed to make it easier to cancel the mode. For example, a message may be displayed that says, "Visual deterioration prevention mode has been set. To release it, please enter a passcode from the release button." A passcode is set in advance by a parent, etc. in order to cancel the mode. If it is not set, you can simply cancel it.

On the other hand, if the mode setting unit 101e determines that the estimated age is higher than the designated age (S45: NO), it sets the visual acuity decline prevention mode to OFF (S47). The designated age here is an age set in consideration of the age of infants and school children who are susceptible to visual acuity and esotropia due to screen viewing.

According to this embodiment, since the vision deterioration prevention mode can be turned on and off according to registered contents set according to the user and, for unregistered users, according to the estimated age, even on a shared terminal, the vision deterioration prevention mode can be set according to the user. Prevention processing can be performed.

<Other change examples>
FIG. 14 is a diagram showing another example of the screen during execution of the visual acuity reduction prevention process.

As a visual acuity reduction prevention process, a warning message 1337a may be displayed as shown in a screen 1337.

Additionally, as a visual acuity reduction prevention process, a gauge 1338a indicating the current distance D relative to the visual acuity decrease determination distance Dth may be displayed as shown in a screen 1338.

Furthermore, as a visual acuity reduction prevention process, a screen 1339 may be used in which the optimal position of the smartphone 100 is shown as an AR display 1339b on a background 1339a taken by the out-camera 132. The configuration may be configured such that when the user moves the smartphone 100 away from the eye and the actual smartphone 100 overlaps the AR display 1339b, the visual acuity deterioration determination distance Dth can be secured.

FIG. 15 is a diagram showing another example of hardware.

The smartphone 100 shown in FIG. 15 includes an embedded in-camera 131 and a distance sensor 153 on the back side of the display 133. As a result, the in-camera 131 and the distance sensor 153 are not obstructed by hands or the like while the display 133 is being viewed, so that the visibility reduction prevention process can be executed stably.

Although the smartphone 100 has been described above as an example of a portable information terminal, the present invention can also be applied to a head-mounted display. FIG. 16 is an external view of the head mounted display, and FIG. 17 is a hardware configuration diagram of the head mounted display.

As shown in FIG. 16, an outside camera 232 and a distance sensor 253 are provided on the outside front of the HMD 200.

FIG. 17 is a hardware configuration diagram of the head mounted display.

As shown in FIG. 17, the HMD 200 includes a main processor 201, a storage device 210, an input interface (I/F) 220, an image processing device 230, an audio processing device 240, a sensor group 250, a communication I/F 260, an expansion I/F 272, A timer 273 is provided, and these are connected to each other via the bus 202.

The storage device 210 includes a RAM 211, a ROM 212, and a flash memory 213.

The input I/F 220 includes a button switch 221. Furthermore, when using gesture input and voice input, the out camera 232 and the microphone 242 also function as the input I/F 220.

The image processing device 230 includes an out camera 232 and a transmissive display 233.

The audio processing device 240 includes a speaker 241 and a microphone 242.

The sensor group 250 includes a GPS receiver 251, a geomagnetic sensor 252, a distance sensor 253, an acceleration sensor 254, a gyro sensor 255, and a biological information acquisition sensor 256.

The communication I/F 260 includes a wireless communication I/F 261.

FIG. 18 is a diagram illustrating an example of visual acuity reduction prevention processing using a head-mounted display.

The main processor 201 of the HMD 200 performs image recognition processing on the image from the out-camera 232, and recognizes whether the smartphone 100 is viewing a book.

Next, the distance sensor 253 measures the distance D to the smartphone 100 or the book. The same processing as in each of the above embodiments may be performed using the distance D. In that case, the change of screen display size is executed by the smartphone 100 in cooperation with the smartphone 100 and the HMD 200. Furthermore, in the case of performing a process of blurring the display screen as the visibility reduction process in the embodiment, a process of reducing the brightness or transparency of the transmissive display 233 may be used instead. That is, the process of lowering the visibility of the book or the smartphone 100 may be performed by the HMD 200 or a smartphone linked to the HMD 200.

As another example, when the HMD 200 recognizes an action of bringing the screen of the smartphone 100 closer, it is predicted that the user will want to enlarge the screen. Therefore. An enlarged screen of the smartphone 100 may be displayed on the transparent display 233 of the HMD 200. This makes it possible to view the screen of the smartphone 100 while keeping the smartphone 100 away. Furthermore, if the HMD 200 and the smartphone 100 are working together, the screen may be enlarged on the smartphone 100 side.

FIG. 19 is a diagram showing a system for transmitting a log of a child's viewing history to a parent.

Via the communication network 300, a smartphone 100A viewed by user A (child) is communicatively connected to an HMD 200 worn by user B (parent), or a smartphone 100B operated by user B. A communication network 300 is connected to a server 310 and

wireless routers

321 and 322. The smartphone 100A connects to the communication network 300 via a wireless router 321, and the HMD 200 and smartphone 100B connect to the communication network 300 via a wireless router 322.

The viewing log of the smartphone 100A is recorded on the server 310. This viewing log includes a viewing start time, a viewing end time, and a visibility reduction process execution log during that time.

When the HMD 200 or the smartphone 100B sends a viewing log transmission request to the server 310, the viewing log is sent from the server 310 to the HMD 200 or the smartphone 100B.

This allows User B (parent) to know not only the viewing time of User A (child), but also the history of the visibility reduction processing executed at that time, so that user B (parent) can learn about the viewing posture and eyesight of the child, which is not reflected in viewing time. The impact can be estimated.

Note that the present invention is not limited to the embodiments described above, and includes various modifications. For example, the above-described embodiments have been described in detail to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to having all the configurations described. Furthermore, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace some of the configurations of each embodiment with other configurations.

Further, each of the above-mentioned configurations, functions, processing units, processing means, etc. may be partially or entirely realized by hardware, for example, by designing an integrated circuit. Furthermore, each of the above configurations, functions, etc. may be realized by software by a processor interpreting and executing a program for realizing each function. Information such as programs, tables, files, etc. that realize each function may be stored in a memory, a recording device such as a hard disk, an SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD. However, it may also be stored in a device on a communication network.

In addition, control lines and information lines are shown that are considered necessary for explanation, and not all control lines and information lines are necessarily shown in the product. In reality, almost all components may be considered to be interconnected.

The embodiment includes the following embodiments.

(Additional note 1)
A mobile information terminal,
display and
camera and
distance sensor;
timer and
comprising a processor;
The processor includes:
Recognizing the user's face or the user's eyes based on the image captured by the camera,
obtaining the distance from the distance sensor to the user's face or the user's eyes;
A first elapsed time from when the distance to the user's face or the user's eyes becomes less than or equal to a predetermined visual acuity deterioration determination distance is obtained from the timer, and the first elapsed time is greater than or equal to a predetermined start determination time. , perform visibility reduction control to reduce the visibility of the display,
A second elapsed time from when the distance to the user's face or the user's eyes exceeds the visual acuity deterioration determination distance is acquired from the timer, and when the second elapsed time exceeds a predetermined end determination time, the terminating the visibility reduction control and returning the display to normal display;
Mobile information terminal.

(Additional note 2)
A mobile information terminal,
display and
camera and
distance sensor;
timer and
comprising a processor;
The processor includes:
Recognizing the user based on the image captured by the camera,
obtaining the distance from the distance sensor to the user;
A first elapsed time from when the distance to the user becomes less than or equal to a predetermined visual acuity deterioration determination distance is obtained from the timer, and when the first elapsed time becomes equal to or greater than a predetermined start determination time, visual recognition of the display is performed. Visibility reduction control that reduces visibility
A second elapsed time from when the distance to the user exceeds the visual acuity reduction determination distance is acquired from the timer, and when the second elapsed time exceeds a predetermined end determination time, the visibility reduction control is terminated. to return the display to normal display;
Mobile information terminal.

(Additional note 3)
A display control method in a mobile information terminal, the method comprising:
The mobile information terminal includes a display, a camera, a distance sensor, a timer and a processor,
The processor includes:
Recognizing the user's face or the user's eyes based on the image captured by the camera, and obtaining the distance from the distance sensor to the user's face or the user's eyes;
A first elapsed time from when the distance to the user's face or the user's eyes becomes less than or equal to a predetermined visual acuity deterioration determination distance is obtained from the timer, and the first elapsed time is greater than or equal to a predetermined start determination time. the step of starting visibility reduction control to reduce the visibility of the display;
A second elapsed time from when the distance to the user's face or the user's eyes exceeds the visual acuity deterioration determination distance is acquired from the timer, and when the second elapsed time exceeds a predetermined end determination time, the terminating the visibility reduction control and returning the display to normal display;
A display control method for a mobile information terminal.

100: Smartphone, 100A: Smartphone, 100B: Smartphone, 101: Main processor, 101a: Face recognition section, 101b: Visibility reduction processing section, 101c: Time measurement section, 101d: User authentication section, 101e: Mode setting section, 102: bus, 110: storage device, 111: RAM, 112: ROM, 113: flash memory, 113a: table storage section, 113b: user registration information storage section, 120: input I/F, 121: button switch, 122: touch panel, 130: Image processing device, 131: In-camera, 132: Out-camera, 133: Display, 140: Audio processing device, 141: Speaker, 142: Microphone, 150: Sensor group, 151: GPS receiver, 152: Geomagnetic sensor, 153: distance sensor, 154: acceleration sensor, 155: gyro sensor, 156: biological information acquisition sensor, 160: communication I/F, 161: wireless communication I/F, 162: telephone network communication I/F, 171: timer, 172: Expansion I/F, 173: Timer, 200: HMD, 201: Main processor, 202: Bus, 210: Storage device, 211: RAM, 212: ROM, 213: Flash memory, 220: Input I/F, 221 : Button switch, 230: Image processing device, 232: Out camera, 233: Transparent display, 240: Audio processing device, 241: Speaker, 242: Microphone, 250: Sensor group, 251: GPS receiver, 252: Geomagnetic sensor , 253: distance sensor, 254: acceleration sensor, 255: gyro sensor, 256: biological information acquisition sensor, 260: communication I/F, 261: wireless communication I/F, 272: expansion I/F, 273: timer, 300 : Communication network, 310: Server, 321: Wireless router, 322: Wireless router, 1331: Normal screen, 1332: Small screen, 1333: Distance screen size table, 1334: Screen, 1335: Screen, 1336: Distance start judgment time table , 1337: Screen, 1337a: Warning text, 1338: Screen, 1338a: Gauge, 1339: Screen, 1339a: Background, 1339b: AR display, A: User, B: User, D: Distance, Dth: Visual acuity deterioration judgment distance, T1: First elapsed time, T2: Second elapsed time, T3: Execution time, TE: End determination time, TS: Start determination time

Claims

A mobile information terminal,
display and
camera and
distance sensor and
timer and
comprising a processor;
The processor includes:
Recognizing the user's face or the user's eyes based on the image captured by the camera,
obtaining the distance from the distance sensor to the user's face or the user's eyes;
A first elapsed time from when the distance to the user's face or the user's eyes becomes less than or equal to a predetermined visual acuity deterioration determination distance is obtained from the timer, and the first elapsed time is greater than or equal to a predetermined start determination time. , perform visibility reduction control to reduce the visibility of the display,
A second elapsed time from when the distance to the user's face or the user's eyes exceeds the visual acuity deterioration determination distance is acquired from the timer, and when the second elapsed time exceeds a predetermined end determination time, the terminating the visibility reduction control and returning the display to normal display;
Mobile information terminal.
The mobile information terminal according to claim 1,
The processor changes the visibility reduction strength of the display according to the distance to the user's face or the user's eyes during the visibility reduction control.
Mobile information terminal.
The mobile information terminal according to claim 2,
The visibility reduction intensity is changed by changing at least one of the screen size of the display, the degree of blur of the screen of the display, and the intensity of the vibrator.
Mobile information terminal.
The mobile information terminal according to claim 1,
The processor changes the start determination time according to the distance to the user's face or the user's eyes.
Mobile information terminal.
The mobile information terminal according to claim 1,
The processor obtains the execution time of the visibility reduction control from the timer, and changes the visibility reduction strength of the display according to the execution time.
Mobile information terminal.
The mobile information terminal according to claim 1,
The processor includes:
collating registered information that associates on or off of the visibility reduction control for each user of the mobile information terminal with the user recognized from the image;
When the recognized user and the user registered in the registration information match, controlling on or off of the visibility reduction control according to the registration information,
If the recognized user and the user registered in the registration information do not match, the age of the user is estimated, and if the estimated age is below a pre-specified age, the visibility reduction control is turned on. set,
Mobile information terminal.
A mobile information terminal,
display and
camera and
distance sensor;
timer and
comprising a processor;
The processor includes:
Recognizing the user based on the image captured by the camera,
obtaining the distance from the distance sensor to the user;
A first elapsed time from when the distance to the user becomes less than or equal to a predetermined visual acuity deterioration determination distance is obtained from the timer, and when the first elapsed time becomes equal to or greater than a predetermined start determination time, visual recognition of the display is performed. Visibility reduction control that reduces visibility
A second elapsed time from when the distance to the user exceeds the visual acuity reduction determination distance is acquired from the timer, and when the second elapsed time exceeds a predetermined end determination time, the visibility reduction control is terminated. to return the display to normal display;
Mobile information terminal.
The mobile information terminal according to claim 7,
The recognition is the user's face or eyes,
The distance to the user is the distance from the distance sensor to the user's face or eyes.
Mobile information terminal.
The portable information terminal according to claim 8,
Changing the visibility reduction strength of the display is performed by changing at least one of the screen size of the display, the degree of blur of the screen of the display, and the intensity of the vibrator.
Mobile information terminal.
The mobile information terminal according to claim 7,
The processor changes the start determination time according to the distance to the user's face or the user's eyes.
Mobile information terminal.
The mobile information terminal according to claim 7,
The processor obtains an execution time of visibility reduction control for reducing the visibility of the display from the timer, and changes the visibility reduction intensity of the display according to the execution time.
Mobile information terminal.
A display control method in a mobile information terminal, the method comprising:
The mobile information terminal includes a display, a camera, a distance sensor, a timer and a processor,
The processor includes:
Recognizing the user's face or the user's eyes based on the image captured by the camera, and obtaining the distance from the distance sensor to the user's face or the user's eyes;
A first elapsed time from when the distance to the user's face or the user's eyes becomes less than or equal to a predetermined visual acuity deterioration determination distance is obtained from the timer, and the first elapsed time is equal to or greater than a predetermined start determination time. the step of starting visibility reduction control to reduce the visibility of the display;
A second elapsed time from when the distance to the user's face or the user's eyes exceeds the visual acuity deterioration determination distance is acquired from the timer, and when the second elapsed time exceeds a predetermined end determination time, the terminating the visibility reduction control and returning the display to normal display;
A display control method for a mobile information terminal.