JP5279737B2 - Display control device, line-of-sight guidance method, and program - Google Patents

Description

  The present invention relates to a display control device, a gaze guidance method, and a program that can appropriately determine a gaze state on a screen and guide a user (player or the like) gaze outside the screen with a simple configuration.

Conventionally, a game system called a network game (online game), which can be enjoyed by connecting each game device via a network and playing against other players or fighting enemy characters in cooperation with each other Is widely spread.
In such a game system, in general, conversation (message transmission / reception) with other players can be enjoyed during the game by character chat. Recently, a game system is also known in which each player wears a headset and can perform voice voice chat.

  In such a game system capable of voice chatting, a technology is also disclosed that can synthesize a voice uttered in a remote place in a virtual space so as to have a sense of reality (see, for example, Patent Document 1).

JP 2006-343447 A (page 6-11, FIG. 2)

In the game system as described above, the player may be addicted to the game and continue to play without a break. Meanwhile, the player continues to look at the screen of the game device. That is, the state of gaze on the screen continues for a long time, and there is concern about the burden on the eyes of the player.
Therefore, an attempt has been made to encourage the player to take a break by displaying a caution message when the play time exceeds a certain time on the game device.

However, since such a warning message is generally displayed according to the play time, it is often annoying for a player who is taking a proper break. For example, such a warning message is not appropriate for a player who is resting moderately while taking his / her own glance off the screen between play, and the player's concentration may be caused by unnecessary display. It leads to being disturbed.
Further, such a caution message is considered to be effective to some extent for a player who has not taken a break. Nevertheless, in response to the caution message, it was impossible to determine whether or not the player actually took a break, and even if the player took a break, whether or not the break was appropriate. For example, even if the player who saw the caution message intends to take a break by interrupting play and removing his physical strength, if the player keeps gazing at the screen during that time, he cannot rest his eyes and It's not a good break.

In order to determine the presence or absence of such a break and the suitability of the break, it is conceivable to detect the line of sight of the player. For example, the line-of-sight tracking camera or the like is used to detect the line of sight of the player and to sequentially determine the state of gaze on the screen. Even so, it is necessary to add a new configuration only for tracking the line of sight, and the configuration becomes large, so it is considered that it is not so popular.
Therefore, there has been a demand for a technique that can determine the state of gaze on the screen using a necessary configuration on the player side without prompting a large-scale configuration and prompt the player to take an appropriate break.

  The present invention has been made in view of the above circumstances, and with a simple configuration, a display control device and a gaze guidance method that can appropriately determine a gaze state on a screen and guide a user's gaze to the outside of the screen And to provide a program.

  A display control device according to a first aspect of the present invention is a display control device capable of acquiring sensor information from a headset worn on a user's head, and includes a calculation unit, a detection unit, and a display control unit. It is configured with.

First, the calculation unit sequentially calculates the user's line-of-sight information based on sensor information obtained from the headset (for example, acceleration information in three axial directions). For example, when the operation according to the display image is performed, the calculation unit sets the origin of the line of sight directed toward the screen, and sequentially obtains the amount of change in the line of sight with respect to the origin from the sensor information obtained thereafter. The gaze information including the current gaze direction is calculated.
The detection unit detects a screen gaze state in which the user's line of sight is directed into the display screen based on the calculated line-of-sight information. For example, when the user's current line of sight is within a specified range determined with the origin as a reference, the screen is detected as a screen gaze state.
When the detected screen gaze state continues beyond the reference time, the display control unit changes the display form of the image displayed on the display screen in order to guide the user's line of sight outside the display screen. For example, the display control unit guides the user's line of sight outside the display screen by stopping the display of an image or displaying a guidance image.

As described above, when the screen gaze state is detected by the sensor information obtained from the headset and the screen gaze state is continued beyond the reference time, the display screen is displayed in order to guide the user's line of sight outside the display screen. Change the display form of the image to be displayed.
As a result, it is possible to appropriately determine the state of gaze on the screen and to guide the user's line of sight outside the screen with a simple configuration.

  A display control device according to a second aspect of the present invention is a display control device capable of acquiring acceleration information in a plurality of directions (for example, three axis directions) from a headset worn on a user's head, And a calculation unit, a detection unit, a timing unit, and a display control unit.

First, a display part displays the game image etc. about the game which a user plays, for example. The calculating unit sequentially calculates the line-of-sight information of the user based on the acceleration information in a plurality of directions obtained from the headset. For example, the calculation unit sets the origin of the line of sight directed to the screen at the time when an operation according to the displayed game image is performed, and the amount of change in the line of sight with respect to the origin from the acceleration information obtained thereafter Are obtained sequentially, and gaze information including the current gaze direction is calculated.
Further, the detection unit detects a screen gaze state in which the user's line of sight is directed into the screen of the display unit based on the calculated line-of-sight information. For example, the screen gaze state is detected when the user's line of sight is directed within a predetermined rectangular range determined with the origin as a reference.
Further, the timer unit measures the duration time of the detected screen gaze state. That is, the time during which the user keeps watching the screen is counted.
And a display control part will display the guidance image for guide | inducing a user's eyes | visual_axis outside the screen of a display part, if the measured duration exceeds reference time. For example, the display control unit displays the game image dimly, and displays an arrow symbol or the like for guiding the user's line of sight on the game image.

As described above, when the screen gaze state is detected based on the acceleration information in a plurality of directions obtained from the headset, the measurement of the duration of the screen gaze state is started. When the measured duration exceeds the reference time, a guidance image for guiding the user's line of sight outside the screen of the display unit is displayed.
As a result, it is possible to appropriately determine the state of gaze on the screen and to guide the user's line of sight outside the screen with a simple configuration.

The detection unit may further detect a state in which the user's line of sight is directed outside the screen of the display unit after the display control unit displays the guidance image.
In this case, according to the display of the guidance image, it can be confirmed that the actual user's line of sight is directed outside the screen.

The above display control device
An operation reception unit for receiving operations from users;
An operation discriminating unit that discriminates whether or not the accepted operation is an operation corresponding to the image displayed on the display unit;
When it is determined that the operation corresponds to the image, the calculation unit sets the origin of the line of sight directed to the display unit, and sequentially calculates the current line of sight information based on the acceleration information obtained after the setting. May be.
In this case, the user's line-of-sight information can be accurately calculated.

  The line-of-sight guidance method according to the third aspect of the present invention is capable of acquiring sensor information from a headset worn on a user's head, and includes a calculation unit, a detection unit, and a display control unit. A line-of-sight guidance method in the apparatus, comprising a calculation step, a detection step, and a display control step.

First, in the calculation step, the user's line-of-sight information is sequentially calculated based on sensor information obtained from the headset (for example, acceleration information in three axial directions). For example, in the calculation step, when the operation corresponding to the display image is performed, the origin of the line of sight directed to the screen is set, and the amount of change in the line of sight with respect to the origin is sequentially obtained from the sensor information obtained thereafter. The gaze information including the current gaze direction is calculated.
In the detection step, a screen gaze state in which the user's line of sight is directed into the display screen is detected based on the calculated line-of-sight information. For example, when the user's current line of sight is within a specified range determined with the origin as a reference, the screen is detected as a screen gaze state.
In the display control step, when the detected screen gaze state is continued beyond the reference time, the display form of the image displayed on the display screen is changed in order to guide the user's line of sight outside the display screen. For example, in the display control step, the user's line of sight is guided out of the display screen by stopping the display of the image or displaying a guidance image.

As described above, when the screen gaze state is detected by the sensor information obtained from the headset and the screen gaze state is continued beyond the reference time, the display screen is displayed in order to guide the user's line of sight outside the display screen. Change the display form of the image to be displayed.
As a result, it is possible to appropriately determine the state of gaze on the screen and to guide the user's line of sight outside the screen with a simple configuration.

  A program according to a fourth aspect of the present invention causes a computer (including an electronic device) that can acquire sensor information from a headset attached to a user's head to function as the display control device. Configure.

  This program can be recorded on a computer-readable information recording medium such as a compact disk, flexible disk, hard disk, magneto-optical disk, digital video disk, magnetic tape, and semiconductor memory.

  The above program can be distributed and sold via a computer communication network independently of the computer on which the program is executed. The information recording medium can be distributed and sold independently of the computer.

  According to the present invention, it is possible to appropriately determine the state of gaze on the screen and to guide the user's line of sight outside the screen with a simple configuration.

It is a block diagram which shows the schematic structure of the typical game device with which the display control apparatus which concerns on this embodiment is implement | achieved. It is a schematic diagram for demonstrating an example of the external appearance of a headset. It is a block diagram for demonstrating the schematic structure of the display control apparatus which concerns on this embodiment. (A)-(d) is a schematic diagram for demonstrating a mode that all eyes | visual_axis are calculated. It is a schematic diagram which shows an example of the game image displayed on a monitor. (A)-(c) is a schematic diagram for demonstrating operation according to an image all. (A)-(c) is a schematic diagram for demonstrating the relationship between a virtual screen and all eyes | visual_axis. (A) is an example of an arrow symbol, (b) is a schematic diagram which shows an example of an attention message. (A)-(c) is all the schematic diagrams for demonstrating the display of a guidance image. It is a flow chart for explaining line-of-sight guidance processing concerning this embodiment. It is a flowchart for demonstrating the detail of a guidance image display process. It is a block diagram for demonstrating the schematic structure of the display control apparatus which concerns on other embodiment. It is a schematic diagram which shows an example of the navigation image displayed on a monitor.

  Embodiments of the present invention will be described below. In the following, for ease of understanding, an embodiment in which the present invention is applied to a game device will be described. However, the present invention can be similarly applied to information processing devices such as various computers, PDAs, and mobile phones. it can. That is, the embodiment described below is for explanation, and does not limit the scope of the present invention. Therefore, those skilled in the art can employ embodiments in which each or all of these elements are replaced with equivalent ones, and these embodiments are also included in the scope of the present invention.

  FIG. 1 is a block diagram showing a schematic configuration of a typical game device in which the display control device according to the embodiment of the present invention is realized. Hereinafter, a description will be given with reference to FIG.

  The game apparatus 100 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an interface 104, a controller 105, a headset 106, an external memory 107, A DVD (Digital Versatile Disc) -ROM drive 108, an image processing unit 109, a monitor 110, an audio processing unit 111, and a NIC (Network Interface Card) 112 are provided.

  Note that a DVD-ROM storing a game program and data is loaded into the DVD-ROM drive 108 and the game apparatus 100 is turned on to execute the program, and the display control apparatus of the present embodiment is Realized.

The CPU 101 controls the overall operation of the game apparatus 100 and is connected to each component to exchange control signals and data.
Note that the CPU 101 includes a timer module as an example, and can measure elapsed time and the like.

  The ROM 102 records an IPL (Initial Program Loader) that is executed immediately after the power is turned on, and when this is executed, the program recorded on the DVD-ROM is read out to the RAM 103 and execution by the CPU 101 is started. The The ROM 102 stores an operating system program and various data necessary for operation control of the entire game apparatus 100.

  The RAM 103 is for temporarily storing data and programs, and holds programs and data read from the DVD-ROM and other data necessary for game progress and chat communication.

  The controller 105 connected via the interface 104 receives an operation input performed when the player executes the game. For example, the controller 105 accepts input of a character string (message) or the like according to the operation input.

The headset 106 connected via the interface 104 is mounted on the player's head and receives the voice (voice chat voice, etc.) emitted by the player, and measures the movement of the player's head.
As an example, as shown in FIG. 2, the headset 106 includes a microphone 201 for collecting the player's voice and an arithmetic unit 202 that performs various arithmetic processes. Note that the arithmetic unit 202 is stored in a housing in the figure.

Specifically, the arithmetic unit 202 converts analog sound collected from the microphone 201 into digital sound information. The arithmetic unit 202 has, for example, a triaxial acceleration sensor, and generates sensor information (acceleration information in three axial directions) indicating the movement of the player's head when the headset 106 is worn. . Specifically, the head movement (movement in the three-axis direction) of the player wearing the headset 106 is sequentially measured by a three-axis acceleration sensor, and acceleration information indicating the values is sequentially generated.
The arithmetic unit 202 sequentially performs such conversion of voice information and generation of sensor information (measurement of acceleration). The headset 106 supplies these audio information and sensor information to the game apparatus 100 in real time via the interface 104.
The headset 106 (calculation unit 202) may measure the movement of the player's head using a gyro sensor or the like instead of the acceleration sensor. Further, the headset 106 may supply the audio information and sensor information to the game apparatus 100 wirelessly by wireless communication or the like.

  Returning to FIG. 1, the external memory 107 detachably connected via the interface 104 stores data indicating game progress, chat communication log (record) data, and the like in a rewritable manner. The player can record these data in the external memory 107 as appropriate by inputting an instruction via the controller 105.

  A DVD-ROM mounted on the DVD-ROM drive 108 stores a program for realizing the game and image data and audio data associated with the game. Under the control of the CPU 101, the DVD-ROM drive 108 performs a reading process on the DVD-ROM loaded therein, reads out necessary programs and data, and these are temporarily stored in the RAM 103 or the like.

  The image processing unit 109 processes the data read from the DVD-ROM by an image arithmetic processor (not shown) included in the CPU 101 or the image processing unit 109, and then processes the processed data in a frame memory ( (Not shown). The image information recorded in the frame memory is converted into a video signal (video signal) at a predetermined synchronization timing and supplied to the monitor 110. As a result, various images can be displayed on the monitor 110.

The image calculation processor can execute a two-dimensional image overlay calculation, a transmission calculation such as α blending, and various saturation calculations at high speed.
In addition, the polygon information arranged in the virtual three-dimensional space and added with various kinds of texture information is rendered by the Z buffer method, and a rendered image is obtained by overlooking the polygon arranged in the virtual three-dimensional space from a predetermined viewpoint position. High speed execution of the obtained operation is also possible.

  Further, the CPU 101 and the image arithmetic processor operate in a coordinated manner, so that a character string can be drawn as a two-dimensional image in a frame memory or drawn on the surface of each polygon according to font information that defines the character shape. is there. The font information is recorded in the ROM 102, but it is also possible to use dedicated font information recorded in the DVD-ROM.

  The monitor 110 includes, for example, a television monitor (television receiver) or the like, and displays an image generated by the image processing unit 109. The monitor 110 has a built-in speaker, and can output the sound supplied from the sound processing unit 111.

  The audio processing unit 111 converts audio data read from the DVD-ROM into an analog audio signal and supplies it to the monitor 110. For example, the sound processing unit 111 generates sound effects and music data to be generated during the progress of the game under the control of the CPU 101, and supplies the corresponding sound to the speaker of the monitor 110. As a result, various sounds can be output from the speaker of the monitor 110.

  The NIC 112 is for connecting the game apparatus 100 to a computer communication network (not shown) such as the Internet, and conforms to the 10BASE-T / 100BASE-T standard used when configuring a LAN (Local Area Network). Analog modem for connecting to the Internet using a telephone line, ISDN (Integrated Services Digital Network) modem, ADSL (Asymmetric Digital Subscriber Line) modem, cable modem for connecting to the Internet using a cable television line And an interface (not shown) that mediates between these and the CPU 101.

In addition, the game apparatus 100 uses a large-capacity external storage device such as a hard disk so that it performs the same function as the ROM 102, RAM 103, external memory 107, DVD-ROM attached to the DVD-ROM drive 108, and the like. It may be configured.
In addition, it is possible to adopt a form in which a keyboard for receiving a character string editing input from a player, a mouse for receiving various position designations and selection inputs, and the like are connected.

  Further, instead of the game apparatus 100 of the present embodiment, a general computer (general-purpose personal computer or the like) can be used as a display control apparatus. For example, a general computer includes a CPU, a RAM, a ROM, a DVD-ROM drive, and a NIC as well as the game apparatus 100, an image processing unit that has a simpler function than the game apparatus 100, and an external device. In addition to having a hard disk as a storage device, flexible disks, magneto-optical disks, magnetic tapes, and the like can be used. In addition, a keyboard, a mouse or the like is used as an input device instead of a controller. Further, a headset similar to the headset 106 described above is used. Then, after the game program is installed, when the program is executed, it functions as a display control device.

(Outline of display control device)
FIG. 3 is a block diagram showing a schematic configuration of the display control apparatus 300 according to the present embodiment. As an example, the display control device 300 is a device used by a player wearing the above-described headset 106 to play an online game. The display control device 300 displays a game image on the above-described monitor 110 while displaying the game image from the headset 106. It is a device that can acquire supplied sensor information. Hereinafter, the display control apparatus 300 will be described with reference to the drawing, taking a case where an online RPG is specifically played as an example.

  As shown in FIG. 3, the display control apparatus 300 includes a sensor information acquisition unit 301, a calculation unit 302, an operation reception unit 303, a game information storage unit 304, an image generation unit 305, an operation determination unit 306, The timer 307, the detection part 308, the guidance image storage part 309, and the display control part 310 are provided.

First, the sensor information acquisition unit 301 acquires sensor information supplied from the headset 106. That is, the acceleration information in the triaxial direction measured by the arithmetic unit 202 (triaxial acceleration sensor) in FIG. 2 described above is sequentially acquired.
Then, the sensor information acquisition unit 301 sequentially supplies the acquired sensor information (acceleration information) to the calculation unit 302.
Note that the above-described interface 104 or the like can function as such a sensor information acquisition unit 301.

The calculation unit 302 sequentially calculates the line-of-sight information of the player based on the sensor information obtained from the headset 106 via the sensor information acquisition unit 301.
Specifically, as shown in FIG. 4A, when the player performs an operation corresponding to the game screen, the line of sight S0 is set as the origin of the line of sight directed to the screen (monitor 110), and thereafter Based on the obtained sensor information (acceleration information in three axis directions), the amount of change in the line of sight from the origin is sequentially obtained, and the line of sight information including the current line of sight direction is calculated. Note that whether or not an operation according to the game screen has been performed by the player is determined by the operation determination unit 306 as described later.

In other words, the calculation unit 302 displays the line of sight S0 in three axis directions (X axis, Y axis, and Z axis) as shown in FIG. 4B when an operation corresponding to the game screen is performed by the player. Set as origin.
For example, when the player rotates his head a little to the right (when the player turns his head slightly to the right, etc.), the calculation unit 302 obtains the amount of change in the line of sight from the acceleration information in the three-axis directions, As shown in FIG. 4C, the line-of-sight information of the line of sight S1 obtained by rotating the line of sight S0 around the Y axis is calculated.
Further, when the player moves his head a little to the left (when the neck is left as it is and the upper body is translated a little to the left, etc.), the calculation unit 302 similarly uses the acceleration information in the three-axis directions. The line-of-sight change amount is obtained, and as shown in FIG. 4D, the line-of-sight information of the line of sight S2 obtained by translating the line of sight S1 along the X axis is calculated.
Then, the calculation unit 302 sequentially supplies the line-of-sight information calculated in this way to the detection unit 308.
When the player performs an operation corresponding to the game screen again, the calculation unit 302 resets the line of sight S0 as the origin in the three-axis directions, as described above. Then, in the same manner as described above, the line-of-sight information is calculated sequentially from the sensor information obtained thereafter, and the line-of-sight information is calculated.
Note that the CPU 101 and the like described above can function as such a calculation unit 302.

Returning to FIG. 3, the operation accepting unit 303 accepts various operations performed by the player during the game.
For example, the operation accepting unit 303 sequentially accepts an operation for instructing a moving direction of the player character, an operation for selecting an enemy character to fight, an acquisition operation for picking up a found item, and the like.
When the operation accepting unit 303 accepts such an operation, the operation accepting unit 303 supplies operation information indicating the content (which button has been pressed, etc.) to the operation determining unit 306 or the like.
The controller 105 described above can function as such an operation receiving unit 303.

The game information storage unit 304 stores various information necessary for the game.
For example, the game information storage unit 304 stores image information such as game fields, characters (player characters, enemy characters, etc.), and various items.
The game information storage unit 304 also stores management information such as the position information of each character (current position in the game field, etc.) and the current state of the player character (physical strength, experience value, possession money, owned items, etc.).
Note that the above-described DVD-ROM mounted on the DVD-ROM drive 108, the RAM 103, and the like can function as such a game information storage unit 304.

The image generation unit 305 generates a game image based on the image information stored in the game information storage unit 304.
For example, as illustrated in FIG. 5, the image generation unit 305 generates a game image in which the player character Pc, the enemy character Tc, and the like are appropriately arranged in the game field according to the managed position information.
The image processing unit 109 described above can function as such an image generation unit 305.

Returning to FIG. 3, the operation determination unit 306 determines whether an operation corresponding to the game screen has been performed.
For example, the operation determination unit 306 is an operation performed while looking at the game screen based on the relationship between the operation content of the player received by the operation reception unit 303 and the information stored in the game information storage unit 304. It is determined whether or not.

Explaining with a specific example, for example, as shown in FIG. 6A, in the situation where a plurality of enemy characters Tc exist in the vicinity of the player character Pc, the selection cursor Se is moved to move any enemy. When the character Tc is selected, the operation determination unit 306 determines that an operation corresponding to the game screen has been performed. In other words, the operation of selecting the enemy character Tc by moving the selection cursor Se can be performed only by looking at the game screen. Therefore, the operation determination unit 306 is an operation corresponding to the game screen. Is determined.
Also, for example, as shown in FIG. 6B, when any item is selected by moving the selection cursor Se in a situation where the menu Mn for the selected enemy character Tc is displayed, The operation determination unit 306 determines that an operation corresponding to the game screen has been performed. In other words, since an operation for selecting any item by moving the selection cursor Se can be performed only while viewing the game screen, the operation determination unit 306 determines that the operation is in accordance with the game screen. To do.
In addition to this, for example, as shown in FIG. 6C, in the situation where the player character Pc touches the item I and the menu Mn for the item I is displayed, the selection cursor Se is moved to either When the item is selected, the operation determination unit 306 similarly determines that an operation corresponding to the game screen has been performed.
Such a situation shown in FIGS. 6A to 6C is only an example, and can be appropriately changed according to the game content and the game genre. That is, the operation determination unit 306 determines in advance that the operation is in accordance with the game screen when such an operation is performed by the player by preliminarily defining the operation that can only be performed while viewing the game screen. .
When the operation determination unit 306 determines that the operation is in accordance with the game screen in this way, the operation determination unit 306 notifies the calculation unit 302 to that effect.
Note that the CPU 101 and the like described above can function as such an operation determination unit 306.

Returning to FIG. 3, the time measuring unit 307 measures the time during which the player keeps gazing at the screen.
For example, the timing unit 307 starts timing when the detection unit 308 first detects a state in which the line of sight is directed into the screen (that is, the in-screen gaze state). If the detection unit 308 detects a state in which the line of sight is directed outside the screen (that is, an off-screen gaze state) during the time measurement, the time measurement is temporarily stopped. Then, when the in-screen gaze state is detected again by the detection unit 308, time measurement is resumed. That is, the time measuring unit 307 measures the accumulation of time during which the player keeps gazing at the screen.
And if the time measuring part 307 measures time until it exceeds the reference time (for example, 60 minutes etc.), it will notify the detection part 308 that the reference time has been exceeded.

In addition, after notifying the detection unit 308 that the reference time has been exceeded, the time measuring unit 307 measures the time during which the player is turning his / her line of sight outside the screen.
For example, the time counting unit 307 starts time counting when the detection unit 308 is notified of the off-screen gaze state for the first time. If the gaze state in a screen is detected by the detection part 308 during the time measurement, time measurement will be stopped once. Then, when the detection unit 308 detects the off-screen gaze state again, the time measurement is resumed. That is, the time measuring unit 307 measures the accumulation of time during which the player is out of sight.
And if the time measuring part 307 measures time until it exceeds a rest time (for example, 5 minutes etc.), it will notify the detection part 308 that the rest time is exceeded. Thereafter, the time counting unit 307 again counts the time during which the player keeps gazing at the screen from zero.
Note that the above-described timer module of the CPU 101 or the like can function as such a timer unit 307.

Based on the line-of-sight information calculated by the calculation unit 302, the detection unit 308 detects whether the player's line of sight is directed within the screen of the monitor 110 or the screen.
Specifically, when the line of sight S0 is set as the origin by the calculation unit 302, the detection unit 308 is centered on a point P obtained by extending the line of sight S0 by a predetermined distance L as shown in FIG. A rectangular range having a predetermined size is set as the virtual screen Sc. As the size of the virtual screen Sc, a value experimentally determined in consideration of the sensitivity of the acceleration sensor of the headset 106 described above is used.

Then, the detection unit 308 detects the gaze state in the screen when the current gaze direction is within the virtual screen Sc based on the gaze information sequentially calculated by the calculation unit 302, and conversely, the current gaze direction Is out of the virtual screen Sc, the off-screen gaze state is detected.
That is, when the line of sight Sm as illustrated in FIG. 7B is calculated by the calculation unit 302, the detection unit 308 detects the in-screen gaze state because the line of sight Sm is within the virtual screen Sc.
On the other hand, when the line-of-sight Sn as shown in FIG. 7C is calculated by the calculation unit 302, the detection unit 308 detects the off-screen gaze state because the line of sight Sn is deviated from the virtual screen Sc.
In addition, when the detecting unit 308 first detects the in-screen gaze state, the detecting unit 308 causes the time measuring unit 307 to start measuring time as described above. Then, at the time when the reference time is notified from the time measuring unit 307, the detection unit 308 detects the in-screen gaze state for a long time. That is, based on the line-of-sight information sequentially calculated by the calculation unit 302, it is detected that the player has continued to watch the screen for a long time.
When detecting the in-screen gaze state for a long time, the detection unit 308 notifies the display control unit 310 to that effect.

Further, after detecting the in-screen gaze state for a long time to the display control unit 310, the detection unit 308 next detects the off-screen gaze state, and causes the time counting unit 307 to start measuring time as described above. When the time measuring unit 307 is notified that the break time has been exceeded, the detection unit 308 detects the off-screen gaze state for the break time. That is, based on the line-of-sight information sequentially calculated by the calculation unit 302, it is detected that the player has left the line of sight outside the screen for the rest time.
When detecting the off-screen gaze state for the break time, the detection unit 308 notifies the display control unit 310 to that effect. Thereafter, the detection unit 308 again tries to detect the in-screen gaze state for a long time.
Note that the CPU 101 and the like described above can function as such a detection unit 308.

Returning to FIG. 3, the guide image storage unit 309 stores a guide image for guiding the player's line of sight to the outside of the screen.
For example, the guidance image storage unit 309 stores an arrow symbol Ar as shown in FIG. 8A and an image of a caution message Ms as shown in FIG.
The arrow symbol Ar and the caution message Ms are examples, and can be changed as appropriate.
The DVD-ROM mounted on the above-described DVD-ROM drive 108, the RAM 103, and the like can function as such a guide image storage unit 309.

Returning to FIG. 3, the display control unit 310 displays the game image generated by the image generation unit 305 as it is on the monitor 110 during normal play. On the other hand, when the detection unit 308 detects the on-screen gaze state for a long time described above, a guidance image for guiding the player's line of sight to the outside of the screen of the monitor 110 is displayed. As an example, the display control unit 310 displays the guidance image as follows.
First, as shown in FIG. 9A, the display control unit 310 displays a dimmed display by reducing the brightness and saturation of the game image, and then displays a caution message for a normal image without decreasing the brightness or the like on the game image. Ms is synthesized and displayed. Unlike the game image, the caution message Ms is displayed with normal brightness or the like, so the player naturally turns his attention to the caution message Ms.
Then, the display control unit 310 displays the caution message Ms and, for example, after a few seconds have elapsed, as shown in FIG. Lower and display dimly. At the same time, as shown in FIG. 9B, the arrow symbol Ar of the normal image whose brightness is not lowered is synthesized and displayed on the attention message Ms. Since this arrow symbol Ar is displayed with normal brightness or the like instead of the caution message Ms, the player now turns his gaze on the arrow symbol Ar.

Further, the display control unit 310 sequentially moves the arrow symbol Ar toward the outer edge of the screen in the direction indicated by the arrow (in this example, the right direction) as shown in FIG. 9C. . At this time, the display control unit 310 sequentially obtains the player's line-of-sight direction (line-of-sight information) via the detection unit 308, and moves the player's line of sight slightly ahead (in this case, slightly to the right) with an arrow. The symbols Ar may be moved sequentially. That is, when the player tries to pay attention to the displayed arrow symbol Ar, the arrow symbol Ar is displayed so as to move slightly ahead of the line of sight. By sequentially moving and displaying the arrow symbol Ar slightly ahead of such line of sight, the player's line of sight is guided out of the screen of the monitor 110. When the player's line of sight deviates from the screen, the display control unit 310 ends the display of the arrow symbol Ar.
If the player turns his gaze on the screen before the detection unit 308 notifies that the break time is exceeded (the off-screen gaze state for the break time), the display control unit 310 Again, the arrow symbol Ar is displayed, and the arrow symbol Ar is sequentially moved slightly ahead of the player's line of sight, as described above. That is, when the detecting unit 308 notifies the in-screen gaze state before the break time elapses, the display control unit 310 moves and displays the arrow symbol Ar as described above, and displays the player's line of sight on the screen. Guide outside.
Then, when the off-screen gaze state for the rest time is notified from the detection unit 308, the display control unit 310 returns to the display during normal play. In other words, the display with the brightness reduced is stopped, and the normal game image generated by the image generation unit 305 is displayed on the monitor 110 as it is.

Note that the shape of the arrow icon Ar (direction of the arrow, etc.) and the method of moving the arrow icon Ar in such guidance display are examples, and can be changed as appropriate. That is, an arrow icon or the like having a shape different from that shown in FIG. 8A may be used, and the arrow icon or the like may be appropriately moved so that the player follows the eye.
The image processing unit 109 described above can function as such a display control unit 310.

(Outline of operation of display control device)
Hereinafter, the operation of the display control apparatus 300 having such a configuration will be described with reference to the drawings. As an example, an operation from detecting a long-time in-screen gaze state based on sensor information obtained from the headset 106 to guiding the player's line of sight to the outside of the screen will be described with reference to FIGS. 10 and 11. FIG. 10 is a flowchart showing the flow of the line-of-sight guidance process executed by the display control apparatus 300. FIG. 11 is a flowchart (subroutine) showing details of the guide image display process in FIG.

As shown in FIG. 10, first, the display control apparatus 300 displays a game image (step S401).
That is, the image generation unit 305 generates a game image being played based on the image information stored in the game information storage unit 304. That is, a game image as shown in FIG. 5 is generated.

The display control device 300 calculates line-of-sight information (step S402).
That is, the calculation unit 302 calculates the line-of-sight information of the player based on sensor information obtained from the headset 106 via the sensor information acquisition unit 301. That is, as shown in FIG. 4B described above, the calculation unit 302 sets the line of sight S0 as the origin in the three-axis directions when an operation corresponding to the game screen is performed by the player. Then, the amount of change in the line of sight is obtained from the acceleration information in the three-axis directions, and the line-of-sight information such as the lines of sight S1, S2 that change sequentially is calculated as shown in FIGS. 4 (c) and 4 (d).

The display control device 300 determines whether or not the screen is in the on-screen gaze state (step S403).
That is, based on the line-of-sight information calculated by the calculation unit 302, the detection unit 308 detects whether the player's line of sight is directed within the screen of the monitor 110 or the screen. That is, as shown in FIG. 7B described above, the detection unit 308 detects the in-screen gaze state when the current line of sight Sm is within the virtual screen Sc. On the other hand, as illustrated in FIG. 7C described above, when the current line of sight Sn is out of the virtual screen Sc, the detection unit 308 detects an off-screen gaze state.

If the display control device 300 determines that it is in the in-screen gaze state (step S403; Yes), it keeps counting time (step S404).
That is, the time measuring unit 307 that measures the time during which the player keeps gazing at the screen (in-screen gazing time) continues to time while the in-screen gazing state is detected by the detection unit 308. It is assumed that the timing unit 307 has started counting from zero when the in-screen gaze state is detected for the first time.

On the other hand, when it is determined that the screen is not in the on-screen gaze state (out-of-screen gaze state) (step S403; No), the display control device 300 stops timing (step S405).
That is, the time measuring unit 307 that measures the time that the player keeps gazing at the screen once the off-screen gaze state is detected by the detection unit 308, stops the time measurement.

The display control device 300 determines whether or not the reference time has been exceeded (step S406).
That is, the detection unit 308 determines whether or not the time measuring unit 307 has timed the in-screen gaze state until the reference time is exceeded.
If the display control apparatus 300 determines that the reference time has not been exceeded (step S406; No), the display control apparatus 300 returns the process to step S401, and repeatedly executes the processes of steps S401 to S406 described above.

On the other hand, when it is determined that the reference time has passed (step S406; Yes), the display control device 300 performs a guidance image display process (step S407).
The guidance image display process will be described in detail with reference to FIG. In the guidance image display process, the time measuring unit 307 measures the time from which the line of sight is directed to the outside of the screen (out-of-screen gaze time) from zero.

  In the guidance image display process of FIG. 11, first, the display control device 300 displays the game image dimly (step S501). In other words, the display control unit 310 displays the game image with a low brightness and saturation.

The display control apparatus 300 displays a caution message (step S502).
For example, as shown in FIG. 9A described above, the display control unit 310 synthesizes and displays a caution message Ms of a normal image that has not been reduced in brightness or the like on the game image that is displayed dimly. Unlike the game image, the caution message Ms is displayed with normal brightness or the like, so that the player naturally turns his gaze toward the caution message Ms.

The display control apparatus 300 displays the caution message in a dim state and also displays an arrow symbol (step S503).
For example, as shown in FIG. 9B described above, the display control unit 310 also displays the caution message Ms with low brightness and saturation as in the game image, and displays the dimness on the caution message Ms. The arrow symbol Ar of the normal image that is not lowered is combined and displayed. Since this arrow symbol Ar is displayed with normal brightness or the like instead of the caution message Ms, the player now turns his gaze on the arrow symbol Ar.

  The display control apparatus 300 acquires line-of-sight information (step S504). That is, the display control unit 310 acquires line-of-sight information via the detection unit 308.

The display control apparatus 300 determines whether or not the screen is in the on-screen gaze state (step S505).
That is, based on the line-of-sight information calculated by the calculation unit 302, the detection unit 308 determines whether the player's line of sight is directed within the screen of the monitor 110 (in-screen gaze state) or is directed outside the screen. Is detected (out-of-screen gaze state).

If the display control device 300 determines that it is in the on-screen gaze state (step S505; Yes), it moves the arrow symbol slightly ahead of the line of sight (step S506).
For example, as shown in FIG. 9C described above, the display control unit 310 sequentially moves the arrow symbols Ar slightly ahead (in this example, slightly right) where it is desired to move the line of sight of the player. That is, when the player tries to pay attention to the displayed arrow symbol Ar, the arrow symbol Ar is displayed so as to move slightly ahead of the line of sight. By sequentially moving and displaying the arrow symbol Ar slightly ahead of such line of sight, the player's line of sight is guided out of the screen of the monitor 110.

The display control device 300 stops timing (step S507).
In other words, the timekeeping unit 307, which measures the time during which the player turns the line of sight outside the screen (out-of-screen gaze time), indicates that the in-screen gaze state is detected by the detection unit 308, including during the gaze guidance. Temporarily stop timing.

On the other hand, when it is determined that the screen is not in the on-screen gaze state (out-of-screen gaze state) (step S505; No), the display control device 300 deletes the arrow symbol (step S508).
That is, since the display control unit 310 guides the player's line of sight to the outside of the screen, the display control unit 310 deletes the arrow symbol Ar as shown in FIG.

The display control device 300 keeps timing (step S509).
That is, the time measuring unit 307 that measures the off-screen gaze time keeps counting while the detection unit 308 detects the off-screen gaze state. It is assumed that the time counting unit 307 starts counting from zero when the off-screen gaze state is detected for the first time.

The display control device 300 determines whether or not the break time has been exceeded (step S510).
That is, the detection unit 308 determines whether or not the time measuring unit 307 has timed until a predetermined break time is exceeded.
If it is determined that the break time has not been exceeded (step S510; No), the display control device 300 returns the process to step S504, and repeatedly executes the processes of steps S504 to S510 described above.

On the other hand, when it is determined that the break time has been exceeded (step S510; Yes), the display control device 300 returns the game image to the normal display (step S511).
That is, the display control unit 310 stops displaying the brightness and the like, and displays the normal game image generated by the image generation unit 305 on the monitor 110 as it is.
Then, the display control apparatus 300 ends the guidance image display process of FIG.

When the gaze state in the screen is detected based on the sensor information (acceleration information in the three-axis directions) obtained from the headset 106 by such gaze guidance processing (guidance image display processing), the duration of the gaze state in the screen is determined. Start timing. When the measured duration exceeds the reference time, a guidance image is displayed to guide the player's line of sight outside the screen of the monitor 110.
As a result, it is possible to appropriately determine the state of gaze on the screen and to guide the user's line of sight outside the screen with a simple configuration.

(Other embodiments)
In the above embodiment, the line-of-sight guidance processing and the like in the case of playing an online game have been described, but the target game is an example and can be changed as appropriate. Further, the present invention is not limited to the case of playing such a game. For example, the present invention can be appropriately applied to a scene where a telephone operator or the like works while looking at a screen of business software.
Furthermore, the present invention can be applied as appropriate to an apparatus that is desired to detect a gaze state on the screen from the viewpoint of safety or the like. For example, in a vehicle-mounted navigation system, the present invention can also be applied to a display control device that is used to enable voice instructions and mobile phone calls using the headset 106 described above.
Hereinafter, a display control device used in a navigation system will be described with reference to the drawings.

  FIG. 12 is a block diagram showing a schematic configuration of a display control apparatus 600 according to another embodiment of the present invention. As an example, the display control device 600 is a device that allows a user wearing the above-described headset 106 to give an instruction to search for a guidance route by voice. In the same manner as described above, sensor information supplied from the headset 106 can be acquired. Hereinafter, the display control apparatus 600 will be described with reference to FIG.

  As shown in FIG. 12, the display control device 600 includes a sensor information acquisition unit 301, a calculation unit 302, a touch panel 603, a navigation information storage unit 604, an image generation unit 305, an operation determination unit 306, and a time measuring unit. 607, a detection unit 308, a vehicle speed information acquisition unit 609, and a display control unit 610.

The sensor information acquisition unit 301 acquires sensor information supplied from the headset 106 in the same manner as the display control device 300 in FIG. 3 described above.
The calculation unit 302 sequentially calculates the line-of-sight information of the player based on the sensor information obtained from the headset 106 via the sensor information acquisition unit 301 in the same manner as the display control device 300 of FIG.

The touch panel 603 is arranged on the entire surface of the monitor 110 described above, and detects the pressed position in response to a touch operation (pressed by a finger or the like) by the user.
Then, the touch panel 603 supplies information on the detected pressed position to the operation determination unit 306 and the like.

The navigation information storage unit 604 stores various types of information necessary for route guidance for the guidance route.
For example, the navigation information storage unit 604 stores image information such as map images and icons of various facilities.
The navigation information storage unit 604 also stores road information (node and link information) corresponding to the map, guidance route information searched from the road information, and the like.

The image generation unit 305 generates a navigation image based on image information, guidance route information, and the like stored in the navigation information storage unit 304.
For example, the image generation unit 305 generates a navigation image including a map image Nv showing the vehicle position and the guidance route, and various button images Bt as shown in FIG.

Returning to FIG. 12, the operation determination unit 306 determines whether an operation corresponding to the navigation image has been performed by the user.
For example, when the navigation image as shown in FIG. 13 is displayed on the monitor 110 and any one of the button images Bt is pressed via the touch panel 603, the operation determination unit 306 displays the navigation image. It is determined that the corresponding operation has been performed by the user. That is, since the operation of the button image Bt can be performed only while viewing the screen, the operation determination unit 306 determines that the operation is in accordance with the navigation screen.

The time measuring unit 607 measures the time during which the user keeps gazing at the screen continuously.
For example, the timer unit 607 starts measuring time while the in-screen gaze state is detected by the detection unit 308. If the detection unit 308 detects an off-screen gaze state during the time measurement, the time measurement ends. That is, the time measuring unit 607 measures the continuous time that the player is gazing at the screen.
And if the time measuring part 607 measures time until it exceeds the allowable reference time for safety (for example, 5 seconds etc.), it notifies the detection part 308 that the allowable reference time has been exceeded. The permissible reference time may be dynamically changed according to the vehicle speed acquired by the vehicle speed information acquisition unit 609. For example, the higher the speed of the vehicle, the shorter the allowable reference time may be taken into consideration in terms of safety.

Similarly to the display control device 300 in FIG. 3 described above, the detection unit 308 determines whether the player's line of sight is directed within the screen of the monitor 110 based on the line-of-sight information calculated by the calculation unit 302 (in-screen gaze). State) or whether it is directed outside the screen (out-of-screen gaze state).
Then, when detecting the in-screen gaze state, the detection unit 308 causes the time measurement unit 607 to perform time measurement as described above. Then, at the time when the time limit unit 607 notifies that the allowable reference time has been exceeded, the detection unit 308 detects an in-screen gaze state that exceeds the allowable reference time for safety. That is, based on the line-of-sight information sequentially calculated by the calculation unit 302, it is detected that the user has continued to watch the screen beyond the allowable reference time.
When the in-screen gaze state exceeding the allowable reference time is detected, the detection unit 308 notifies the display control unit 610 to that effect.

The vehicle speed information acquisition unit 609 acquires vehicle speed information indicating the speed of the vehicle on which the display control device 600 is mounted.
For example, the vehicle speed information acquisition unit 609 sequentially acquires vehicle speed information obtained from a vehicle speed sensor or a vehicle controller. Then, the acquired vehicle speed information is supplied to the display control unit 610 and the detection unit 308.

The display control unit 610 displays the navigation image generated by the image generation unit 305 as it is on the monitor 110 during normal display.
On the other hand, when the detection unit 308 detects an in-screen gaze state that exceeds the allowable reference time described above and the vehicle is traveling based on the vehicle speed information, the display control unit 610 displays the user's line of sight on the monitor 110. In order to guide to the outside of the screen, the display of the navigation image is stopped. For example, the display control unit 610 stops supplying the video signal to the monitor 110 and turns off the display of the navigation image.
Then, after the display of the navigation image is stopped, for example, when the off-screen gaze state is maintained for a certain time or longer, the display control unit 610 returns to the normal display and displays the navigation image generated by the image generation unit 305.

Even with the display control device 600 having such a configuration, when the in-screen gaze state is detected based on the sensor information (acceleration information in the three-axis directions) obtained from the headset 106, the time duration of the in-screen gaze state is measured. Start. When the measured duration exceeds the allowable reference time, display of the navigation image is stopped in order to guide the user's line of sight outside the screen of the monitor 110.
As a result, it is possible to appropriately determine the state of gaze on the screen and to guide the user's line of sight outside the screen with a simple configuration.

  As described above, according to the present invention, with a simple configuration, it is possible to appropriately determine the state of gaze on the screen and guide the user's line of sight outside the screen, the line of sight guidance method, and Can provide a program.

100 game machine 101 CPU
102 ROM
103 RAM
104 Interface 105 Controller 106 Headset 107 External Memory 108 DVD-ROM Drive 109 Image Processing Unit 110 Monitor 111 Audio Processing Unit 112 NIC
DESCRIPTION OF SYMBOLS 201 Microphone 202 Arithmetic unit 300,600 Display control apparatus 301 Sensor information acquisition part 302 Calculation part 303 Operation reception part 603 Touch panel 304 Game information storage part 604 Navigation information storage part 305 Image generation part 306 Operation discrimination | determination part 307,607 Timing part 308 Detection 309 Guide image storage unit 609 Vehicle speed information acquisition unit 310, 610 Display control unit

Claims (10)

A display control device capable of acquiring sensor information from a headset worn on a user's head,
Based on sensor information obtained from the headset, a calculation unit that sequentially calculates the user's line-of-sight information,
Based on the calculated line-of-sight information, a detection unit that detects a screen gaze state in which the user's line of sight is directed within the display screen;
When the detected screen gaze state is continued beyond the reference time, the user's line of sight is guided to the outside of the display screen , so that a predetermined symbol is sequentially moved toward the outer edge and displayed on the display screen. A control unit,
A display control device characterized by that. The display control device according to claim 1,
The display control unit, after reducing the brightness or saturation of the display screen, the symbol is displayed as a normal image without reducing the brightness and saturation,
A display control device characterized by that. A display control device capable of acquiring sensor information from a headset worn on a user's head,
Based on sensor information obtained from the headset, a calculation unit that sequentially calculates the user's line-of-sight information,
Based on the calculated line-of-sight information, a detection unit that detects a screen gaze state in which the user's line of sight is directed within the display screen;
A display control unit for stopping display of the display screen in order to guide the user's line of sight outside the display screen when the detected screen gaze state is continued beyond a reference time;
A display control device characterized by that. A display control device capable of acquiring acceleration information in a plurality of directions from a headset worn on a user's head,
A display unit for displaying a predetermined image;
Based on acceleration information in a plurality of directions obtained from the headset, a calculation unit that sequentially calculates the user's line-of-sight information,
Based on the calculated line-of-sight information, a detection unit that detects a screen gaze state in which the user's line of sight is directed into the screen of the display unit;
A time measuring unit for measuring the duration of the detected screen gaze state,
If timed the duration exceeds the reference time, for guiding the line of sight of the user to the screen outside of the display unit, a display control unit which displays by sequentially moved toward the outer edge of the predetermined symbol on the screen Comprising
A display control device characterized by that. The display control device according to claim 4 ,
The display control unit, after reducing the brightness or saturation of the screen , displays the symbol in a normal image that does not reduce the brightness and saturation,
A display control device characterized by that. A display control device capable of acquiring acceleration information in a plurality of directions from a headset worn on a user's head,
A display unit for displaying a predetermined image;
Based on acceleration information in a plurality of directions obtained from the headset, a calculation unit that sequentially calculates the user's line-of-sight information,
Based on the calculated line-of-sight information, a detection unit that detects a screen gaze state in which the user's line of sight is directed into the screen of the display unit;
A time measuring unit for measuring the duration of the detected screen gaze state,
A display control unit for stopping display of the screen in order to guide the user's line of sight outside the screen of the display unit when the measured duration exceeds a reference time;
A display control device characterized by that. The display control apparatus according to any one of claims 4 to 6 ,
The detection unit, after the display control unit Gosei display of the display unit further detects a state in which the line of sight of the user is directed outside the screen of the display unit,
A display control device characterized by that. The display control device according to any one of claims 4 to 7 ,
An operation reception unit for receiving operations from users;
An operation discriminating unit that discriminates whether or not the accepted operation is an operation corresponding to the image displayed on the display unit;
When the calculation unit determines that the operation corresponds to the image, the calculation unit sets the origin of the line of sight directed to the display unit, and sequentially calculates the current line of sight information based on the acceleration information obtained after the setting. To calculate,
A display control device characterized by that. Sensor information can be obtained from a headset worn on the user's head, and is a line-of-sight guidance method in a display control device having a calculation unit, a detection unit, and a display control unit,
The calculation unit sequentially calculates user's line-of-sight information based on sensor information obtained from the headset;
Based on the calculated line-of-sight information, the detection unit detects a screen gaze state in which the user's line of sight is directed into the display screen; and
The display control section, when detected the screen gaze state continues exceeds the reference time, for guiding the display screen outside the line of sight of the user, sequentially toward the outer edge of the predetermined symbol on the display screen A display control step for moving and displaying ,
A gaze guidance method characterized by the above. A computer that can acquire sensor information from a headset worn on the user's head,
Based on sensor information obtained from the headset, a calculation unit that sequentially calculates the user's line-of-sight information,
Based on the calculated line-of-sight information, a detection unit that detects a screen gaze state in which the user's line of sight is directed into the display screen,
When the detected screen gaze state is continued beyond the reference time, the user's line of sight is guided to the outside of the display screen , so that a predetermined symbol is sequentially moved toward the outer edge and displayed on the display screen. Control unit,
A program characterized by functioning as

Images