JP2006268630A - Information display device and attention determination method using the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image information display device, considering, when an image to be displayed is a dynamic image, not only the time of watching the image or the number of persons watching the image, but also the timing of paying attention thereto. <P>SOLUTION: The number of persons who gaze the dynamic image during its display period is counted for every frame, a weight is assigned to each display frame of the dynamic image to set the importance for each frame, whereby the attention degree of the dynamic image considering not only the number of persons paying attention to the displayed dynamic image, but the timing of paying attention is quantitatively expressed. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image information display apparatus that displays image information, and more particularly to an apparatus having a line-of-sight detection function.

  As a conventional information display device, there is a device that detects the line of sight of a person who views an image display apparatus, displays image information at the line of sight, and counts the person who is gazing at the displayed image information (for example, , See Patent Document 1). FIG. 5 shows a configuration diagram and an outline of a conventional information display device described in Patent Document 1. In FIG.

In FIG. 5, the gaze of the person is detected by the gaze detection unit 12 from the image captured by the camera 10, and the determination unit 20 displays the image information in the presentation information storage unit on the gaze destination on the display unit 18, and the gaze of the person is While the displayed image is being watched, the number of persons watching the image is counted and recorded in the collected information storage unit 16.
Japanese Patent No. 3579218 (page 9, FIG. 1)

  However, in the conventional configuration, a person who is paying attention to the image information is measured, but it is not considered which period of the period during which the image is displayed is being watched. For this reason, in the case of an image whose display information changes over time, such as video information, there is a problem that the degree of attention cannot be measured quantitatively considering not only the number of people watching the video but also the timing of attention. Was.

  The present invention solves the above-described conventional problems. In the image display device, when the displayed image is a moving image, not only the time and the number of people watching the image but also the timing of attention is considered. An object is to provide an image information display device.

  In order to solve the conventional problems, an information display device according to the present invention includes an information display unit that displays moving image information, an image capturing unit that captures an image, and a person in the image captured by the image capturing unit. A face detection unit that detects the position of the face of the person from the image, a line of sight detection unit that detects the line of sight of the person in the face detected by the face detection unit, and a line of sight detected by the line of sight detection unit. Determines whether the information means is gazing, count means for measuring the number of people gazing, moving image storage means for storing moving image information to be displayed on the information display section, and moving image storage The moving image weight storage means for storing the frame weight obtained by quantifying the importance of each moving image frame stored in the means, the number of persons measured by the counting means, and the moving image weight storage means A control unit that calculates a frame attention score representing how much the frame of the moving image has received attention from the frame weight, calculates a moving image attention score in the entire moving image from the frame attention score, and the frame attention score and the moving image attention A score storage means for storing either or both of the scores was provided.

  With this configuration, when the displayed image is a moving image, it is possible to provide an image information display device that takes into account not only the time and number of people watching the image but also the timing of interest.

  According to the present invention, when the image displayed on the image display device is a moving image, the degree of attention of the display image in each frame from the number of people watching the image for each frame of the moving image and the importance of the moving image frame Can be expressed quantitatively. By calculating the degree of attention of the entire moving image using the degree of attention for each frame, an image information display device that takes into account not only the time and number of people watching the displayed image but also the timing of attention Can be provided.

  Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1)
FIG. 1 is a configuration diagram of an information providing apparatus according to Embodiment 1 of the present invention. In this configuration, an image information display unit 101, a photographing unit 102, a face detection unit 103, a line-of-sight detection unit 104, a counting unit 105, a moving image database 106, a moving image weight database 107, a control unit 108, and a score storage unit 109 are provided. is doing.

  In FIG. 1, an image display unit 101 receives image information stored in advance in a database and displays the image information. In the present invention, the image display unit 101 does not display image information captured by the imaging unit 102. The photographing unit 102 is installed for photographing a person who is gazing at the image information display unit 101. Ideally, the image information display unit may be installed so that the entire region where the person can gaze can be photographed. desirable. The face detection unit 103 detects a face area of a person from the photographed image photographed by the photographing unit 102. The gaze detection unit 104 detects the gaze direction of the person when the face detection unit 103 detects a person in the captured image. The count unit 105 counts the number of persons who are looking at the image information display unit 101 in the captured image.

  The moving image database 106 is a database for storing moving images to be displayed on the image information display unit 101. The moving image weight database 107 stores frame weights obtained by converting the importance set for each frame of the moving images stored in the moving image database 106 into numerical values. If the sum of the frame weights differs for each moving image, there is a problem that the evaluation criteria do not match. Therefore, the sum of the frame weights in the entire moving image is set to a constant value for normalization. The control unit 108 sends a moving image from the moving image database 106 to the image information display unit 101 for display. The control unit 108 further calculates the attention level of the person in the corresponding frame from the number of people in the captured image calculated by the counting unit 105 and the frame weight in the corresponding frame in the moving image weight DB while the moving image is displayed. A frame attention score that quantitatively expresses is calculated. The score storage unit 109 stores a frame attention score calculated by the control unit and a moving image attention score expressed from the sum of frame weights in the entire moving image.

  The operation of the information display device having the above configuration will be described with reference to FIG. First, when the operation of the information display apparatus is started (S201), the control unit 108 starts displaying a moving image on the image display unit 101 (S202). Next, the control unit 108 acquires moving image data from the moving image database, and first displays the video of the frame i (i = 1, 2, 3,..., N) in order to repeatedly display N frames of moving images. The image is sent to the image display unit 101 to start displaying a moving image (S203). Here, i is a frame number. Also, N is the number of frames of the entire moving image. At the start of display, i = 1 since it is the first video. Next, the number of people looking at the frame i is counted (S204). This is a process of counting the number of people who are gazing at the image display unit in an image taken when frame i is displayed. This is performed by the line-of-sight detection unit 104 and the count unit 105. Details of the counting process will be described later with reference to FIG.

  When the counting of the number of gazes is completed, the frame gaze score for the i-th frame is calculated (S205). Here, a value obtained by multiplying the frame weight by the number of gazes is defined as a frame gaze score.

  Next, the end of the moving image display is determined (S206). If the measurement has been completed for all frames of the moving image, i.e., i = N, the process proceeds to a process of calculating a gaze score of the entire moving image (S207). If not completed, the process repeats from S203 for the next frame.

  The calculation of the moving image gaze score indicating the gaze degree of the entire next moving image is obtained by taking the sum of the frame gaze scores for each frame for the entire moving image, that is, all the frames. When the moving image gaze score is obtained (S207), the moving image gaze score is stored in the score storage unit (S208), and the process ends (S209). Although the moving image gaze score is stored here, not only the moving image gaze score but also information for each frame, for example, the frame gaze score of each frame and the number of people counted may be stored. By storing the number of people counted in chronological order, the viewer's attention to the video can be determined, and important information can be obtained as a marketing survey.

  Next, the flow of the count process performed in S204 will be described using FIG. First, the photographing unit 102 starts photographing (S301). As described above, since the photographing unit 102 is installed to photograph a person who watches the image information display unit 101, a person who may be watching the image information display unit 101 is photographed. (S302). The photographing unit 102 may be a fixed type or a movable type. However, in order to determine whether a person's line of sight faces the information display unit 101, in the case of the movable type, the positions of the photographing unit 102 and the image information display unit 101, and the photographing It is necessary to always keep the information such as the shooting angle of the unit 102 updated.

  The photographed image is sent to the face detection unit 103, and the presence / absence of a person in the image is determined and the position of the face is detected when there is a person (S303). Is checked (S304). For example, face detection includes a method in which a frequency filter such as a Gabor filter is applied to an image to extract a frequency component, and a region having a frequency pattern similar to that of the face image is searched by template matching. In addition, color information such as a shading pattern, an edge component, and a skin color may be simply taken into consideration.

  If no face is detected in the image, the counting process is terminated (S308). If even one person is detected, line-of-sight detection is performed for all detected persons (S305). That is, a three-dimensional space in the gaze direction of the person is detected from the detected eye position in the space and the detected gaze direction, and a substance existing in the space is detected. Here, since it is desired to detect whether or not the image information display unit 101 is being watched, the count unit 105 checks whether or not the image information display unit 101 exists in the space in the line-of-sight direction (S306). If a person is gazing at the image information display unit 101, the number of people gazing is counted (S307). If the user is not gazing, the process returns to S303 and returns to detection of another person in the image. Here, with respect to the reference of counting performed by the counting unit 105, for example, there is little confirmation that the user is watching the image information display unit 101 for a moment. high. For this reason, if a person who is continuously pouring his / her line of sight into the image information display unit 101 is counted back as a number of frames, the certainty is further increased. In addition, there is a possibility that the same person gazes a plurality of times, for example, when the same person moves in the captured image to gaze, or when he / she comes to the image information display unit 101 again and starts gaze, May be counted as another person as if they are in different places in time and space, or they can be identified as the same person by face recognition or tracking, and if they need to be counted exactly, they are the same You may count as a person.

  FIG. 4 shows an example of gaze count. This is an example in the case where there are six persons A to F from the start to the end of the image. The horizontal axis is the time, and the vertical axis is the gaze information of six persons from the person A to the F. The thick line part of the arrow indicates the time of gazing, and the dotted line part indicates the part that is not gazing, such as not looking here. As described above, when a person who has watched the image information display unit 101 for a certain period of time is counted as a person to be watched and the gaze time of the person F does not satisfy the standard for a certain time, the number of people watching in the moving image at this time The count is A, B, C, D.

  Next, a calculation example of the moving image gaze score will be described with reference to FIG. In FIG. 5, in the three graphs, the horizontal axis represents time (moving image frames), the vertical axis represents the frame weight value in each frame, the number of people watching the image information display device 101 in each frame, and the frame gaze score in each frame. It is. That is, the three graphs represent time changes in the number of people watching the image display device 101, the frame weight, and the frame watching score in order from the top. Here, since the frame gaze score is the product of the frame weight and the number of gazes in the frame, the value of the frame gaze score is as shown in the bottom graph of FIG. 5, and the moving image represents the degree of attention in the entire moving image. The gaze score is the area of the hatched portion in the bottom graph of FIG.

  As described above, when a moving image is displayed on the image information display device, it is possible to quantitatively express the attention level of the moving image in consideration of not only the number of people but also the timing of attention. Further, even when the display image is a still image, since the displayed images are the same, it can be handled by making the frame weights uniform.

The information display device according to the present invention is an image information display device that displays a moving image, and sequentially counts the number of gazes during the moving image display period, and assigns a weight to each display frame of the moving image, and is important for each frame. By setting the degree, it becomes possible to quantitatively express the degree of attention of moving images that takes into account not only the number of people who are interested in the moving images to be displayed but also the timing of attention. Useful.

The block diagram which shows the structure of the information display apparatus in Embodiment 1 of this invention. Processing flow diagram of information display device in embodiment 1 of the present invention FIG. 3 is a flowchart of a person count process in the information display device in Embodiment 1 of the present invention. The figure explaining the example of moving image gaze score calculation in Embodiment 1 of this invention Block diagram showing an example of a conventional information display device

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Image display part 102 Image | photographing part 103 Face detection 104 Eye-gaze detection part 105 Count part 106 Moving image database 107 Moving image weight database 108 Control part 109 Score preservation | save part

Claims (5)

  Information display means for displaying moving image information, image photographing means for photographing an image, face detection means for detecting the position of a person's face from the image, and detecting a person's line of sight in the face detected by the face detection means A line-of-sight detecting means, a line-of-sight detected by the line-of-sight detection means, determining whether the line of sight is gazing at the information display means, a counting means for measuring the number of people gazing, and a display for displaying on the information display means Moving image storage means for storing moving image information, moving image weight storage means for storing a frame weight obtained by quantifying the importance of each moving image stored in the moving image storage means, and measurement by the counting means A frame attention score representing how much the frame of the moving image has been noticed is calculated from the number of persons received and the frame weight stored in the moving image weight storage means, and the frame attention score And a score storage means for storing one or more of the number of persons counted by the counting means, the frame attention score, and the moving image attention score in each frame. An information display device characterized by that.   2. The information display device according to claim 1, wherein, in the frame weight stored in the moving image weight storage means, the sum of the frame weights in one moving image is a constant value independent of the moving image.   2. The information display device according to claim 1, wherein the counting unit counts the number of eyes when the line of sight detected by the line of sight detection unit stays at any position of the information display unit for a predetermined time or more. .   The information display device according to claim 1, wherein when calculating the frame attention score, the control section can calculate a gaze timing by calculating a frame attention score for all frames. Display moving image information on the information display means,
Photographing an inspected individual gazing at the moving image information;
Detecting the position of a person's face from the captured image;
Detecting a person's line of sight in the detected face;
Determining whether the detected line of sight is gazing at the information display means, and counting the number of people gazing;
Calculating a frame attention score representing how much attention has been paid to the frames of the moving image from the number of people measured by the counting means and the frame weight given to each frame;
Calculating a moving image attention score in the entire moving image from the frame attention score;
Attention level determination method having

Images