JP2018077417A - Display system, program, and display control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To guide the line of sight of a first user to a partial image indicated by a second user in a display image.SOLUTION: A display system comprises: a determination part that determines a gazed position at which a first user gazes in an image displayed in a display area; an information acquisition part that acquires information on a partial image designated by a second user in the image; and a determination part that determines whether the partial image is gazed by the first user on the basis of the gazed position determined in the display area while the image is displayed and the information of an area of the partial image. When determining that the partial image is not gazed, the display system displays a guidance image for guiding the line of sight to the partial image in the display area.SELECTED DRAWING: Figure 6

Description

  The present disclosure relates to a display system, a program, and a display control method, and more particularly, to a display system, a program, and a display control method for prompting a user to pay attention to a display image.

  When presenters make presentations, analog information materials such as whiteboards and posters or digital materials such as dedicated applications and conference solutions are often used. Participants listen to the presentation while viewing the presentation material presented by the presenter.

  As a technique for outputting information and encouraging the user's attention to the output information, for example, Patent Document 1 (Japanese Patent Laid-Open No. 2000-184345) discloses a user's voice in information shared by a user with a participant. And a multimodal communication system for extracting user attention information from video and transmitting the extracted attention information to other participants.

  Patent Document 2 (Japanese Patent Laid-Open No. 2015-191554) discloses an electronic device that determines a target that a user pays attention to from a captured image and guides the user's line of sight to the determined target.

  Japanese Patent Application Laid-Open No. 2000-339309 discloses a character string analysis device that converts input information on a screen that a user is paying attention to into a character string by a line-of-sight tracking means.

JP 2000-184345 A JP-A-2015-191554 JP 2000-339309 A

  There is a demand for a presenter who makes a presentation to guide the participant's line of sight to the information presented by the presenter in order to obtain the effect of the presentation. Each of the above patent documents does not provide a configuration for guiding such a line of sight.

  This disclosure is to provide a display system, a program, and a display control method capable of guiding the line of sight of a first user to a partial image indicated by a second user in a display image.

  A display system according to an aspect of the present disclosure includes a determination unit that determines a gaze position at which a first user gazes in an image displayed in a display area, and information acquisition that acquires information on a partial image specified by a second user in the image And a determination unit that determines whether or not the partial image is attracting attention by the first user based on the gaze position determined during display of the image in the display region and the information on the partial image region, When it is determined that the partial image is not focused, a guide image for guiding the line of sight to the partial image is displayed in the display area.

  Preferably, the display area includes a first display area for displaying an image for the first user and a second display area for displaying an image for the second user, and the determination unit includes the first display area. The gaze position at which the first user gazes is determined, the information acquisition unit acquires information on the partial image specified by the second user in the image displayed in the second display area, and the determination unit includes the first display area. The display system determines whether or not the partial image is attracting attention by the first user based on the gaze position determined during the display of the image and the information on the partial image area, and the display system does not attract the partial image. If it is determined, the guide image is displayed in the first display area.

  Preferably, the display area includes an area on a screen for displaying an image, or an area on a projection surface for projecting an image.

  Preferably, the information acquisition unit includes a matching unit that detects partial information that matches a voice uttered by the second user, and a line-of-sight acquisition unit that acquires the line-of-sight information of the second user. The partial image designated by the second user is a partial image corresponding to the gaze position in the display area based on the acquired line-of-sight information, a partial image on which a pointer image is superimposed or displayed, or a portion matching audio Includes partial images that display information.

  Preferably, a visual line detection unit that detects visual line information indicating a visual line directed to the display area of the first user, and the determination unit determines the gaze position from the detected visual line information.

  Preferably, the determination unit determines whether or not attention is made based on at least one of a distance between the gaze position and the partial image area or a time when the gaze position exists in the area.

  Preferably, the image processing apparatus further includes a route acquisition unit that acquires a history of the movement path of the gaze position in the image, and when the movement path indicates a history of passing the region of the partial image of the image, the guide image is not displayed. To.

  Preferably, when the movement route history does not indicate that the gaze position has passed the area of the partial image, the guide image is displayed.

  Preferably, the type of the guide image is an image of an arrow directed to the partial image, an image of a cursor that moves in the direction of the partial image, an image that emphasizes the partial image, or an image that pops up information represented by the partial image including.

Preferably, the type of the guide image to be displayed is changed based on the size of the display area.
In another aspect of the present disclosure, a program for causing a computer to execute a display control method is provided. The display control method includes a step of acquiring a gaze position at which the first user gazes in an image displayed in the display area, a step of acquiring information on a partial image designated by the second user in the image, and an image in the display area. A step of determining whether or not the partial image has received attention by the first user based on the gaze position acquired during display and the information on the area of the partial image, and a case in which it is determined that the partial image has not received attention And displaying an image for guiding the line of sight to the partial image in the display area.

  In the display control method according to another aspect of the present disclosure, a step of acquiring a gaze position at which the first user gazes in an image displayed in the display area, and information on a partial image specified by the second user in the image are acquired. Determining whether the partial image is attracting attention by the first user based on the step, the gaze position acquired while displaying the image in the display area, and the information on the partial image area; And a step of displaying an image for guiding the line of sight to the partial image in the display area when it is determined that attention has not been paid.

  According to this disclosure, it is possible to guide the line of sight of the first user to the partial image indicated by the second user in the display image.

1 is a diagram schematically showing a configuration of a presentation system according to Embodiment 1. FIG. 2 is a diagram schematically illustrating an example of a usage mode of the presentation system according to Embodiment 1. FIG. 6 is a diagram illustrating a display example of a guide image according to Embodiment 1. FIG. 3 is a diagram illustrating a specific example of a hardware configuration of a participating terminal 300 according to Embodiment 1. FIG. 3 is a diagram illustrating a specific example of a hardware configuration of a presentation-side terminal 200 according to Embodiment 1. FIG. FIG. 3 is a diagram schematically illustrating a configuration of functions of a presentation-side terminal 200 according to Embodiment 1. It is a figure which shows typically the structure of the function which the participating terminal 300 which concerns on Embodiment 1 has. FIG. 5 is a diagram illustrating a line of sight and a gaze position according to Embodiment 1. 6 is a process flowchart of the presentation side terminal 200 according to Embodiment 1. 4 is a process flowchart of a participating terminal 300 according to the first embodiment. 10 is a diagram for explaining a movement path of a gaze position according to Embodiment 2. FIG. 10 is a diagram for explaining a movement path of a gaze position according to Embodiment 2. FIG. 10 is a diagram for explaining a movement path of a gaze position according to Embodiment 2. FIG. 10 is a diagram for explaining a movement path of a gaze position according to Embodiment 3. FIG. It is a figure which shows the other example of the participation side terminal.

  Hereinafter, embodiments will be described with reference to the drawings. The same parts and corresponding parts are denoted by the same reference numerals, and redundant description may not be repeated.

  In each embodiment, the presenter and the participant can share the same image. The presenter is a person instructing a partial image to be noticed by the participant's line of sight in the image. The participant is an example of “first user”, and the presenter is an example of “second user”.

  In each embodiment, when an image is displayed, the gaze position is determined from the gaze information of the participant, and when it is determined that the partial image that the presenter wants to pay attention based on the gaze position is not focused A guide image for guiding the second user's line of sight to the partial image is displayed.

[Embodiment 1]
FIG. 1 is a diagram schematically showing a configuration of a presentation system according to the first embodiment. The presentation system is an example of a “display system”. Referring to FIG. 1, the presentation system mainly includes a presenting terminal 200 used by a presenter and one or a plurality of participating terminals 300 operated by a participant. The presenting terminal 200 and each participating terminal 300 communicate with each other via the network NT. The network NT may include various wired or wireless communication lines. In the first embodiment, the image displayed by the presentation side terminal 200 and the image displayed by the participation side terminal 300 show the same content.

(Usage mode)
FIG. 2 is a diagram schematically illustrating an example of a usage mode of the presentation system according to the first embodiment. With reference to FIG. 2, the presenter displays an image in the display area S by the presentation side terminal 200. The presentation side terminal 200 can transmit image data displayed in the display area S to each of the participation side terminals 300 via the network NT or an external network. The participating terminal 300 displays an image on the display 36 according to the data received from the presentation terminal 200. Therefore, the presenter and the participant can view the same image at the same time.

(Presenter's gaze position and partial image)
The presenter instructs the partial image C in the image displayed in the display area S. The partial image C indicates information that the presenter wants the participant to pay attention to, for example. In the first embodiment, the partial image C is defined as an image within a circle having a radius R centered on the gaze position GP of the presenter, but the defining method is not limited to this. In the first embodiment, the gaze position GP includes the position based on the line of sight directed toward the display area S of the presenter, the position indicated by the indicator (finger, pointer, etc.), and information indicated by the voice of the utterance as will be described later. The displayed position can be included, but is not limited thereto.

  When the gaze position GP is detected by gaze with the gaze of the presenter or by using an indicator, the display area is based on the image captured by the camera 201 (described later) connectable to the presentation side terminal 200 A gaze position GP in S is detected. The camera 201 is arranged so as to be able to photograph the presenter (mainly the face portion) directed to the display area.

  Further, when the gaze position GP is detected by the voice uttered by the presenter, the matching process using the voice is performed. In the matching process, the presentation-side terminal 200 collects speech from the speaker's utterance using the microphone 24 (described later). And the presentation side terminal 200 converts the voice into text data, and performs a matching process using the converted text data (character string or the like). For example, the presentation-side terminal 200 searches for information such as a character string represented by an image being displayed in the display area S based on the converted text data, and extracts and extracts a matching character string based on the search result. The display position of the character string thus identified is identified (detected) as the gaze position GP.

  The presentation side terminal 200 transmits information on the partial image C to the participation side terminal 300. Based on the information received from the presentation-side terminal 200, the participation-side terminal 300 displays a guide image R1 (see FIG. 3) for guiding the line of sight to the partial image in the image being displayed on the display 36.

(Participant gaze position and guidance image)
FIG. 3 is a diagram illustrating a display example of the guide image R1 according to the first embodiment. Referring to FIG. 3, guide image R1 guides a participant's line of sight (more specifically, a participant's gaze position C1) viewing the image on display 36 to partial image C2 (corresponding to partial image C). It is an image that is displayed to do (make attention). As the type of the guide image R1, for example, an image of an arrow directed from the gaze position C1 in the direction of the partial image C2, an image of a cursor moving in the direction of the partial image C2, or an image that emphasizes the partial image C2, or The information (character string or the like) represented in the partial image C2 includes an image displayed in a pop-up near the partial image C2. The guide image R1 may be an image that can guide the participant's line of sight when displayed, and the type is not limited to these.

  In the first embodiment, the display area of the image viewed by the presenter (corresponding to the display area S in FIG. 2) is the display screen of the presenting terminal 200 or the projector 202 (connected to the presenting terminal 200). A projection plane 203 (described later) on which an image is projected can be included. The display area of the image viewed by the participant is a screen of the display 36 in FIG. 2 or a projection plane 38C (described later) on which an image is projected by a projector 38B (described later) to which the participant terminal 300 can be connected. Can be included.

  In the first embodiment, the above-described type of the guide image R1 may be changed based on the size of the display area of the image. For example, the guide image R1 may be a pop-up image on a relatively small screen such as the display 36 such as a personal computer instead of a large screen such as the projector 38B.

(Configuration of participating terminal 300)
FIG. 4 is a diagram illustrating a specific example of the hardware configuration of the participating terminal 300 according to the first embodiment. Referring to FIG. 4, a participating terminal 300 includes a CPU (Central Processing Unit) 30 corresponding to a control unit that controls the whole, and a ROM (Read Only Memory) 31 for storing programs and data executed by the CPU 30. And a RAM (Random Access Memory) 32, a projector I / F (interface omitted) 33, an operation unit 34 for accepting a user (participant) instruction to the participation-side terminal 300, and a communication controller for controlling communication via the network NT 35. A display control unit 38 for controlling the display 36 is included. Further, the participating terminal 300 includes a camera 37 and a memory interface 39.

  The projector I / F 33 connects a projector 33A that projects an image on the projection surface 33B. The communication controller 35 includes a modem circuit and an amplifier circuit.

  The memory interface 39 is detachably mounted with a memory card 40 that is an example of a storage medium. The memory interface 39 includes a circuit for writing data to the memory card 40 or a circuit for reading data from the memory card 40 under the control of the CPU 30.

(Configuration of the presentation terminal 200)
FIG. 5 is a diagram illustrating a specific example of the hardware configuration of the presenting terminal 200 according to the first embodiment. Referring to FIG. 5, presentation terminal 200 includes a CPU (Central Processing Unit) 20 corresponding to a control unit that controls the whole, and a ROM (Read Only Memory) 21 for storing programs and data executed by CPU 20. And a random access memory (RAM) 22, a display 23, an operation panel 25 that a user operates to input information to the presentation terminal 200, a display control unit 26 that controls the display 23, and communication via the network NT. A communication controller 27 and a memory interface 28. The presentation-side terminal 200 includes a projector I / F (abbreviation of interface) 26 </ b> A that connects a microphone 24, a camera 201, and a projector 202 that projects an image onto a projection surface 203.

  The display 23 and the operation panel 25 may be provided as an integrated touch panel. The communication controller 27 includes a modem circuit, an amplifier circuit, and the like for communication. The display control unit 26 includes a drive circuit that drives the display 23 based on the display data.

  A memory card 29 as an example of a storage medium is detachably attached to the memory interface 28. Memory interface 28 includes a circuit that is controlled by CPU 20 to write data to memory card 29 or a circuit that reads data from memory card 29.

(Functional configuration of the presentation terminal 200)
FIG. 6 is a diagram schematically illustrating a functional configuration of the presenting terminal 200 according to the first embodiment. Each unit shown in FIG. 6 is realized by a program executed by the CPU 20 or a combination of a program and a circuit.

  Referring to FIG. 6, the CPU 20 includes a communication processing unit 41 that performs communication processing with the participating terminal 300 via the communication controller 27 and an image display processing unit 50.

  The communication processing unit 41 receives information on the gaze position of the participant from the participation side terminal 300. Specifically, the participating terminal 300 determines the gaze position of the participant while displaying an image on the display area (the display 36 or the projection surface 33B), and provides information indicating the determined gaze position on the presentation terminal 200. Send to.

  The display processing unit 50 includes a line-of-sight detection unit 51, a position determination unit 52, a determination unit 53, an attention determination unit 54, a generation unit 55, and a route acquisition unit 56.

  The line-of-sight detection unit 51 detects the line of sight of the presenter. The position determination unit 52 determines the gaze position GP where the presenter gazes. The position determination unit 52 includes a function of determining the gaze position GP from the line-of-sight information and a matching unit 52A that determines the gaze position GP by the above-described matching processing.

  The determination unit 53 determines a partial image C in the display image based on the determined gaze position GP and outputs information indicating the partial image C when displaying an image on the display area (the display 23 or the projection surface 203). To do.

  The attention determination unit 54 determines whether the participant is paying attention to the partial image C determined by the determination unit 53 based on the information on the gaze position of the participant received by the communication processing unit 41.

  When the attention determining unit 54 determines that the participant does not pay attention to the partial image C, the generation unit 55 guides the guide image R1 for guiding the line of sight to the partial image C in the display area of the display 36 or the projection surface 33B. Image data for displaying (see FIG. 3) is generated. The generated image data is transmitted to the participating terminal 300 via the communication processing unit 41.

  The participation side terminal 300 outputs the image data received from the presentation side terminal 200 to the display control unit 38 or the projector I / F 33. The display control unit 38 drives the display 36 according to the image data. As a result, the guide image R1 for guiding the line of sight to the partial image C2 (corresponding to the partial image C) in the image being displayed on the display 36 is displayed. Similarly, the projector I / F 33 controls the projector 33A according to the image data. Thereby, in the image being displayed on the projection surface 33B, the guide image R1 for guiding the line of sight to the partial image C2 is displayed.

  The route acquisition unit 56 detects a moving route including a history of position information arranged in time series by sampling the information on the gaze position of the participant received from the participating terminal 300 at a predetermined cycle.

(Functional configuration of participating terminal 300)
FIG. 7 is a diagram schematically illustrating a functional configuration of the participating terminal 300 according to the first embodiment. Each unit illustrated in FIG. 7 is realized by a program executed by the CPU 30 or a combination of a program and a circuit.

  Referring to FIG. 7, CPU 30 includes an image display processing unit 60 and a communication processing unit 70. The display processing unit 60 includes an information acquisition unit 63. The information acquisition unit 63 includes a line-of-sight detection unit 61 and a position determination unit 62. The display processing unit 60 generates image data for display and outputs it to the display control unit 38 or the projector I / F 33. The communication processing unit 70 performs communication processing with the presentation side terminal 200 via the communication controller 35.

  The information acquisition unit 63, when an image is displayed in the display area of the display 36 or the projection surface 33B (display area different from the display area of the presentation terminal 200), information on the gaze position of the participant in the displayed image To get. Specifically, the gaze detection unit 61 detects the gaze information of the participant, and the position determination unit 62 determines the gaze position in the displayed image based on the gaze information.

  The communication processing unit 70 generates transmission data for transmitting information on the gaze position of the participant to the presentation side terminal 200, and outputs the generated data to the communication controller 27. Thereby, the participation terminal 300 can transmit information indicating the gaze position of the participant to the presentation terminal 200 in the image displayed on the display 36 or the projection surface 33B.

(Gaze and gaze position detection)
FIG. 8 is a diagram illustrating the line of sight and the gaze position according to the first embodiment. In the first embodiment, when the presenter faces the display area S (the display 23 or the projection plane 203), the face of the presenter is captured by the camera 201. Similarly, when the participant faces his / her face to the display area S (display 36 or projection surface 33B), the participant's face is imaged by the camera 37. First, a method for detecting the presenter's line of sight and gaze position will be described. Cameras 201 and 37 include infrared cameras.

  The line-of-sight detection unit 51 detects the line-of-sight direction of the presenter by processing the output (image signal) from the camera 201, for example, by processing the output of the infrared camera according to the corneal reflection method. Since the corneal reflection method is a well-known technique, description thereof will not be repeated here. The line-of-sight detection unit 51 detects the parallax of the viewpoint from the stereo image signal included in the output of the camera 201, and the viewpoint position E, that is, the three-dimensional position (three-dimensional coordinates of X, Y, and Z) from the detected parallax. Is calculated.

  The position determination unit 52 determines the gaze position GP on the virtual plane (corresponding to the display area S) from the line-of-sight direction (β degrees) and the viewpoint position E. The gaze position GP corresponds to a point that intersects the display area S when a straight line is drawn from the viewpoint position E in the line-of-sight direction.

  The determination unit 53 determines an image included in a circle having a radius R with the gaze position GP as a center point in the image as a partial image C, and outputs information on the determined partial image C. Specifically, the determination unit 53 converts the gaze position GP into a position (coordinate position) on the bitmap according to a predetermined arithmetic expression in the bitmap data of the displayed image. The determination unit 53 determines the position (coordinate position) on the bitmap of the circle portion with the radius R with the gaze position GP as the center point on the calculated bitmap, and the bitmap within the circle with the determined radius R Information indicating the upper position is output as information of the partial image C. Note that the radius R of the partial image C may be variable.

  The information on the gaze position of the participant of the participant terminal 300 can also be acquired as the position (coordinate position) on the bitmap data of the image by the gaze detection unit 61 and the position determination unit 62 as in FIG. The description will not be repeated here. Participating side terminal 300 transmits information on the gaze position of the determined participant to presentation side terminal 200.

(Processing flow of the presentation side terminal 200)
FIG. 9 is a process flowchart of the display control method of presentation-side terminal 200 according to Embodiment 1. A program according to the flowchart is stored in the ROM 21 or the like. The processing is realized by the CPU 20 reading and executing the program. Here, for the sake of simplicity of explanation, the size (magnification) of the image displayed in the display area of the presentation side terminal 200 is equal to the size (magnification) of the image displayed in the display area of the participation side terminal 300. Assume that

  First, an image is displayed on the display area (display 23 or projection surface 203) by the presentation side terminal 200, and the presentation is started. This image is also displayed in the display area (display 36 or 33B) of the participating terminal 300. In the presentation-side terminal 200, during the presentation, the line-of-sight detection unit 51, the position determination unit 52, and the determination unit 53 periodically detect the information on the partial image C based on the gaze position of the presenter.

  First, the communication processing unit 41 receives information on the gaze position of the participant from the participating terminal 300 and outputs the information to the display processing unit 50 (step S3).

  The attention determination unit 54 compares the information on the gaze position of the participant received from the participant terminal 300 with the information on the area of the partial image C determined by the determination unit 53, and based on the result of the comparison, It is determined whether or not the gaze position is deviated (separated) from the partial image C by a predetermined distance (step S5). If the attention determination unit 54 determines that the predetermined distance is not deviated (NO in step S5), the process returns to step S3. As described above, in the first embodiment, a range within a predetermined distance from the partial image C is also referred to as a partial region corresponding to the partial image C. More specifically, the predetermined distance is also a distance from the gaze position GP (predetermined distance> R).

  The partial area corresponding to the partial image C may be an area occupied by the partial image C itself. In that case, the predetermined distance is equal to the radius R.

  On the other hand, when the attention determination unit 54 determines that the gaze position of the participant is not in the partial region corresponding to the partial image C (departs from the predetermined distance) (YES in step S5), the attention determination unit 54 It is determined whether or not the time continuously determined to be out of the distance is within a predetermined time (step S7). When it is determined that the duration is within the predetermined time (YES in step S7), the process returns to step S3.

  On the other hand, when it is determined that the above duration exceeds the predetermined time (NO in step S7), the position determination unit 52 uses the image matching process using the captured image of the camera 201, etc. It is determined whether or not the gaze position GP is designated by an indicator such as a pointer (step S9). If it is determined that the gaze position GP is not designated with an indicator such as a pointer (NO in step S9), the CPU 20 determines whether the presenter is viewing the display image by image matching processing using a captured image of the camera 201 or the like. It is determined whether or not (step S11). For example, it is possible to determine from the image of the presenter of the image captured by the camera 201 whether or not the presenter is looking at the display image by image matching processing.

  If it is determined that the gaze position GP is designated by an indicator such as a pointer or that the presenter is looking at the display image (YES in step S9 or YES in step S11), the position determination unit 52 determines. Information on the partial image C is determined based on the gaze position GP of the presenter (step S15).

  If it is determined that the presenter is not viewing the display image (NO in step S11), the matching unit 52A performs a matching process using the voice of the presenter's utterance, and the presenter's gaze position GP. Is determined (step S13). The determination unit 53 determines information of the partial image C based on the gaze position GP of the presenter determined by the matching process (Step S15).

  The generation unit 55 generates image data for displaying the guidance image R1 for guiding the participant's line of sight to the partial image C that the presenter gazes, and transmits the image data to the participation terminal 300 via the communication processing unit 41. (Step S17). At this time, a guidance image R1 for guiding the line of sight to the partial image C may be displayed in the display area of the presentation side terminal 200.

  The communication processing unit 41 receives gaze position information from the participating terminal 300 (step S18). The attention determination unit 54 compares the received information on the gaze position of the participant with the information on the area of the partial image C, and determines whether or not the gaze position of the participant is within the partial region corresponding to the partial image C. Is determined (step S9).

  If the attention determination unit 54 determines that the gaze position of the participant is not within the partial area corresponding to the partial image C (NO in step S19), the process returns to step S1. On the other hand, when the attention determination unit 54 determines that the gaze position of the participant is within the partial region corresponding to the partial image C (YES in step S19), the CPU 20 does not display the guidance image R1 (guidance image R1). And the non-display command for sending to the participating terminal 300 via the communication processing unit 41 (step S21).

  The CPU 20 determines whether or not the presentation has ended based on the output from the operation panel 25 (step S23). If the CPU 20 determines that the presentation is not completed (NO in step S23), the process returns to step S1, but if it is determined that the presentation is completed (YES in step S23), the series of processing ends.

  According to the processing in FIG. 9, the presentation side terminal 200 is not in the partial area corresponding to the partial image C indicated by the presenter based on the gaze position of the participant received from the participation side terminal 300, but instead to the partial image C. When it is determined that no attention has been made, the participant terminal 300 is controlled to display a guide image R1 for guiding the participant's line of sight to the partial image C. Thereby, the presenter can prompt the participant to pay attention to the information of the partial image C (C2) intended by the presenter.

(Processing flow of participating terminal 300)
FIG. 10 is a process flowchart of the display control method of participating terminal 300 according to Embodiment 1. A program according to the flowchart is stored in the ROM 31 or the like. The processing is realized by the CPU 30 reading and executing the program.

  When an image is displayed on the display area (display 36 or projection surface 33B), the gaze detection unit 61 and the position determination unit 62 obtain information on the gaze position of the participant (step S31). The communication processing unit 70 transmits the acquired gaze position information to the presentation side terminal 200 (step S33).

  The communication processing unit 70 determines whether or not information on the guidance image R1 is received from the presentation side terminal 200 (step S35). When the information on the guide image R1 is received (YES in step S35), the display processing unit 64 performs display processing of the guide image R1 (step S37). Specifically, the display processing unit 64 generates image data for displaying the received guide image R1 superimposed on the image in the display area, and outputs the image data to the display control unit 38 or the projector I / F 33. As a result, the guide image R1 for guiding the line of sight to the partial image (such as the partial image C2 in FIG. 3) corresponding to the partial image C that the presenter gazes on the image of the display area (display 36 or projection surface 33B) Is displayed.

  If the communication processing unit 70 determines that the information of the guidance image R1 is not received from the presentation side terminal 200 (NO in step S35), the communication processing unit 70 determines whether to receive a non-display instruction from the presentation side terminal 200 (step S35). S39). When the non-display command is not received (NO in step S39), the process proceeds to step S45.

  When the non-display command is received (YES in step S39), the display processing unit 64 determines whether or not the guidance image R1 is being displayed (step S41). When it is determined that the guide image R1 is being displayed (YES in step S41), the display processing unit 64 generates image data for deleting the guide image R1 and outputs the image data to the display control unit 38 or the projector I / F 33. As a result, the guide image R1 is erased from the image of the display area (display 36 or projection surface 33B) (step S43). Thereafter, the process proceeds to step S45.

  In step S45, the CPU 30 determines whether or not to end the process based on the output from the operation unit 34 (step S45). If it is determined that the process has not ended (NO in step S45), the process returns to step S31. If it is determined that the process has ended (YES in step S45), the series of processes ends.

  According to FIG. 10, by displaying the guide image R1 for guiding the line of sight, the participant is guided to the partial image C (C2) intended by the presenter, and the partial image C (C2). It becomes possible to pay attention.

  In the first embodiment, the size of the image displayed on the presenter terminal 200 of the presenter is the same as the size of the image displayed on the participant terminal 300 of the participant, but may be different. For example, the display area of the presenter image (for example, the projection plane 203) may be larger than the display area of the participant image (for example, the display 36). In that case, the conversion process is performed so that the information of the partial image C in the display image of the presentation side terminal 200 matches the size (enlargement ratio / reduction ratio) of the display image of the participation side terminal 300.

[Embodiment 2]
In the second embodiment, display / non-display of the guidance image R1 is switched based on the movement path of the gaze position of the participant from the start of the presentation by the presenter (accumulated information of movement history of the gaze position). FIGS. 11, 12, and 13 are diagrams for explaining a movement path of the gaze position according to the second embodiment.

  The route acquisition unit 56 acquires the movement route of the gaze position on the display image from the information on the gaze position of the participant. The display processing unit 50 collates the acquired travel route TR with a predetermined travel route pattern, and is a travel route pattern in which the participant loses sight of the partial image C (C2) based on the collation result. Is determined (see FIG. 11), the guidance image R1 is displayed. In FIG. 11, the participant shows a pattern in which the partial image C (C2) is lost in the display image. In this case, since the movement route does not pass through the partial area corresponding to the partial image C (C2), the guide image R1 is displayed.

  FIG. 12 shows a case where the guide image R1 is not displayed when prefetching is performed. In FIG. 12, two gaze positions C1 of the participants are shown. The display processing unit 50 analyzes the movement path TR of the lower gaze position C1, and the history of the movement path TR indicates that the gaze position C1 passes through the partial image C (C2) (prefetching). (See the movement route TR in FIG. 13), the guidance image R1 is not displayed (not displayed).

  On the other hand, when the movement path TR of the upper gaze position C1 is analyzed and it is determined that the history of the movement path TR indicates that the gaze position C1 has not yet passed through the partial image C (C2), the guidance image R1 is displayed.

[Embodiment 3]
In the third embodiment, the presenter distributes materials to be presented to the participants in an electronic file by displaying images on the presentation side terminal 200, and the participants can freely view the materials before the presentation. Will be explained. FIG. 14 is a diagram for explaining a movement path of the gaze position according to the third embodiment.

  In the participant terminal 300, when the participant is browsing a page other than the current presentation location (regardless of before and after), the display processing unit 60 displays the movement path of the gaze position C1 for each image of each page. get. When an image of a page is displayed on the presenter's presenting terminal 200, the display processing unit 60 transmits information on the movement path TR of the page to the presenting terminal 200. When the display processing unit 50 of the presenting terminal 200 analyzes the received travel route TR and determines that there is a history of gazing at the partial image C, the display processing unit 50 may stop (hide) the display of the guidance image R1. Good.

[Embodiment 4]
In each of the embodiments described above, the presenter and the participant browse different display areas, but both may view the same display area. For example, when the presenter displays and presents an image on a large screen (projection surface of a projector, etc.), each participant may view the image on the large screen without going through the participant terminal 300. The same effect as the embodiment can be obtained.

  FIG. 15 is a diagram illustrating another example of the participating terminal 300. Participating terminal 300 may be a wearable terminal as shown in FIG.

[Embodiment 5]
In the fifth embodiment, a method for confirming the presenter's result is shown. In the fifth embodiment, during the presentation, information on the movement route of the gaze position of the participant acquired by the route acquisition unit 56 and the display history of the guidance image R1 (display count, identifier of the displayed partial image, etc.) Are stored in the memory for each image. After the end of the presentation, the presenter displays the movement path for each stored image and the display history of the guide image R1 on the display 23 and confirms them. Thereby, it becomes possible to grasp | ascertain quantitatively the grade of the attention with respect to the presentation content of a participant.

[Embodiment 6]
In the sixth embodiment, a program for causing the presentation side terminal 200 or the participation side terminal 300 to execute the above-described processing of each embodiment is provided. Such a program is stored on a computer-readable recording medium such as a flexible disk, a CD-ROM (Compact Disk-Read Only Memory), a ROM, a RAM, and a memory card attached to the computer of the presentation side terminal 200 or the participation side terminal 300. And can be provided as a program product. Alternatively, the program can be provided by being recorded on a recording medium such as a hard disk built in the computer. The program can also be provided by downloading via the network NT. The program is executed by one or more processors such as the CPU 30 (CPU 20) or a combination of a processor and a circuit such as an ASIC (Application Specific Integrated Circuit).

  Note that the program may be a program module that is provided as part of an OS (Operating System) of a computer and that performs necessary processing by calling necessary modules in a predetermined arrangement at a predetermined timing. In that case, the program itself does not include the module, and the process is executed in cooperation with the OS. A program that does not include such a module can also be included in the program of the sixth embodiment.

  The provided program product is installed in a program storage unit such as a hard disk and executed. The program product includes the program itself and a recording medium on which the program is recorded.

(Effect of embodiment)
According to each of the above embodiments, when attention is not paid to the partial image intended by the presenter, the guide image R1 for guiding the participant's line of sight to the partial image is displayed. As a result, it is possible to accurately convey the presenter contents to the participants.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

23, 36 Display, 24 microphone, 25 Operation panel, 26, 38 Display controller, 27, 35 Communication controller, 28, 39 Memory interface, 29, 40 Memory card, 33A, 202 Projector, 33B, 203 Projection plane, 34 operation Unit, 37, 201 camera, 41, 70 communication processing unit, 50, 60, 64 display processing unit, 51, 61 gaze detection unit, 52, 62 position determination unit, 52A matching unit, 53 determination unit, 54 attention determination unit, 55 generation unit, 56 route acquisition unit, 63 information acquisition unit, 200 presentation side terminal, 300 participation side terminal, C, C2 partial image, C1, GP gaze position, E viewpoint position, NT network, R radius, R1 guidance image, S Display area.

Claims (12)

A determination unit that determines a gaze position at which the first user gazes in an image displayed in the display area;
An information acquisition unit for acquiring information of a partial image designated by the second user in the image;
A determination unit that determines whether or not the partial image is attracted attention by the first user based on the gaze position determined during display of the image in the display region and information on the region of the partial image; With
A display system that displays a guide image for guiding a line of sight to the partial image in the display area when it is determined that the partial image is not focused. The display area includes a first display area for displaying the image for the first user, and a second display area for displaying the image for the second user,
The determination unit determines a gaze position at which the first user gazes in the first display area,
The information acquisition unit acquires partial image information specified by a second user in an image displayed in the second display area,
The determination unit determines whether or not the partial image is noticed by the first user based on the gaze position determined during display of the image in the first display area and information on the partial image area. Judging
The display system includes:
The display system according to claim 1, wherein the guide image is displayed in the first display area when it is determined that the partial image is not focused.   The display system according to claim 1, wherein the display area includes an area on a screen for displaying the image or an area on a projection plane for projecting the image. The information acquisition unit
Of the information indicated by the image of the display area, a matching unit that detects partial information that matches the voice spoken by the second user;
A line-of-sight acquisition unit that acquires line-of-sight information of the second user,
The partial image designated by the second user is a partial image corresponding to the gaze position in the display area based on the acquired line-of-sight information, a partial image on which a pointer image is superimposed and displayed, or partial information that matches the sound. The display system of Claim 2 or 3 containing the partial image to display. A line-of-sight detection unit that detects line-of-sight information indicating the line of sight directed to the display area of the first user,
The display system according to claim 1, wherein the determination unit determines the gaze position from the detected line-of-sight information. The determination unit
6. The apparatus according to claim 1, wherein whether or not the attention is made is determined based on at least one of a distance between the gaze position and the area of the partial image or a time when the gaze position exists in the area. The display system according to item. A route acquisition unit that acquires a history of a movement route of the gaze position in the image;
The display system according to any one of claims 1 to 6, wherein the guide image is hidden when the movement path indicates a history in which the gaze position has passed the region of the partial image of the image. .   The display system according to claim 7, wherein the guidance image is displayed when the history of the movement route does not indicate that the gaze position has passed the region of the partial image.   The type of the guide image is a pop-up display of an arrow image directed toward the partial image, a cursor image moving in the direction of the partial image, an image that emphasizes the partial image, or information represented by the partial image The display system according to claim 1, comprising an image to be displayed.   The display system according to claim 9, wherein a type of the guide image to be displayed is changed based on a size of the display area. A program for causing a computer to execute a display control method,
The display control method includes:
Obtaining a gaze position at which the first user gazes in an image displayed in the display area;
Obtaining information of a partial image designated by a second user in the image;
Determining whether or not the partial image is being noticed by the first user based on the gaze position acquired during display of the image in the display region and information on the region of the partial image;
And a step of displaying an image for guiding a line of sight to the partial image in the display area when it is determined that the partial image is not focused. Obtaining a gaze position at which the first user gazes in an image displayed in the display area;
Obtaining information of a partial image designated by a second user in the image;
Determining whether or not the partial image is being noticed by the first user based on the gaze position acquired during display of the image in the display region and information on the region of the partial image;
And a step of displaying an image for inducing a line of sight to the partial image in the display area when it is determined that the partial image is not focused.

Images