JP7398854B1 - Web page viewing analysis system, web page viewing analysis method, and web page viewing analysis program - Google Patents

Abstract

The present invention provides detailed insights regarding a web page based on the viewer's behavior when viewing the web page based on biometric information. [Solution] A web page viewing analysis system that analyzes a viewer's insight into a web page acquires a plurality of gaze point data corresponding to each of a plurality of viewers, and at least some of the plurality of gaze point data. Based on the above, a gaze point visualization map is generated that visualizes the gaze point when the viewer corresponding to the gaze point data views the web page, and the gaze point visualization map is superimposed and displayed on the web page. . [Selection diagram] Figure 9

Description

Article 30, Paragraph 2 of the Patent Act applies Website publication date: May 10, 2020, Website address: https://research. viagate. com/ Site publication date: May 25, 2020, publication site address: https://prtimes. jp/main/html/rd/p/000000002.000116494. html Site publication date: May 31, 2020, publication site address: https://prtimes. jp/main/html/rd/p/000000003.000116494. html Site publication date: March 27, 2020, publication site address: https://www. wantedly. com/companies/company_7088487/about Site publication date: June 5, 2020, publication site address: https://sogyotecho. jp/news/20230605viagate/

The present disclosure relates to a web page viewing analysis system, a web page viewing analysis method, and a web page viewing analysis program.

Due to the effects of the coronavirus pandemic, most aspects of social life are now moving online (in other words, digital transformation (DX)) at an accelerating pace. In particular, in the field of marketing, digital marketing has become more mainstream than traditional marketing using paper media and the like. In the field of digital marketing, marketing using websites has become mainstream.

Patent Document 1 discloses an advertisement evaluation system for evaluating the effectiveness of digital marketing such as advertisement videos. The advertisement evaluation system includes a user terminal, an e-mail server, a video distribution server, and a management server. These elements are communicatively connected via a communications network. The advertising evaluation system determines whether or not the user has purchased a product introduced through digital marketing such as advertising videos based on the user's transaction data, and then evaluates the effectiveness of the digital marketing based on the determination result. I am evaluating it.

Patent No. 7057606

In the advertisement evaluation system disclosed in Patent Document 1, the effectiveness of digital marketing is evaluated based on sales performance, which indicates whether or not a user actually purchased a product introduced through digital marketing. For this reason, the advertising evaluation system cannot evaluate digital marketing at a stage where there is not yet actual sales performance of the advertised product. On the other hand, on the side of companies that conduct digital marketing for products, there is a need to grasp detailed insights about web pages for advertising and selling products before launching the web pages. By “insights” we mean a deep understanding of a customer’s underlying needs, emotions, motivations, values, behavior patterns and trends, presented in a sophisticated form by analyzing customer data and feedback. do. In particular, there is a need on the corporate side to obtain insights about a web page based on the biometric information (e.g., line of sight, facial expressions, etc.) of viewers viewing the web page before launching the web page. . As described above, there is room for consideration of an advertisement evaluation system that can meet the above needs.

The present disclosure provides a web page viewing analysis system, a web page viewing analysis method, and a web page viewing analysis program that can provide detailed insights regarding web pages from the viewer's behavior when viewing the web page based on biometric information. The purpose is to provide.

A web page viewing analysis system according to one aspect of the present disclosure is a web page viewing analysis system that analyzes insights of viewers regarding a web page, and acquires multiple gaze point data corresponding to each of multiple viewers. , generate a gaze point visualization map that visualizes the gaze point when the viewer corresponding to the gaze point data views the web page, based on at least some of the plurality of gaze point data; The visualization map is displayed superimposed on the web page.

According to the above configuration, since the point of interest visualization map is presented on the company terminal, the company can understand in detail the browsing behavior of viewers regarding the web page by checking the point of interest visualization map. . For example, it becomes possible for companies to understand in detail the areas of interest on a web page by viewers. In this way, it is possible to provide detailed insights regarding web pages to companies that wish to analyze web pages.

A web page viewing analysis system according to one aspect of the present disclosure is a web page viewing analysis system that analyzes insights of viewers regarding a web page, and acquires a plurality of attribute data corresponding to each of a plurality of viewers, Based on at least some of the plurality of attribute data, recommended viewers are extracted as interview destinations, and information regarding the recommended viewers is displayed.

According to the above configuration, the company can obtain information regarding recommended viewers as interviewees. In this way, it is possible to provide detailed insights regarding web pages to companies that wish to analyze web pages.

A web page viewing analysis system according to one aspect of the present disclosure is a web page viewing analysis system that analyzes insights of viewers regarding a web page, and acquires a plurality of facial expression data corresponding to each of a plurality of viewers, Generating comprehensive facial expression information indicating a comprehensive facial expression when a viewer corresponding to the facial expression data views the web page based on at least some of the plurality of facial expression data, and displaying the comprehensive facial expression information. do.

According to the above configuration, since the comprehensive facial expression information is presented to the company terminal, the company can understand in detail the viewing behavior of the viewer with respect to the web page by checking the comprehensive facial expression information. For example, it becomes possible for companies to understand in detail the facial expressions (for example, happy, sad, etc.) of viewers toward a web page. In this way, it is possible to provide detailed insights regarding web pages to companies that desire web page analysis.

A web page viewing analysis system according to one aspect of the present disclosure is a web page viewing analysis system that analyzes insights of viewers regarding a web page, and acquires a plurality of facial expression data corresponding to each of a plurality of viewers, Based on at least some of the plurality of facial expression data, comprehensive interest level information is generated that indicates the overall level of interest when a viewer corresponding to the facial expression data views the web page, and Display information.

According to the above configuration, since the comprehensive interest level information is presented on the company's terminal, the company can understand in detail the browsing behavior of viewers regarding the web page by checking the comprehensive interest level information. . For example, it becomes possible for companies to understand in detail the degree of interest in web pages by viewers. In this way, it is possible to provide detailed insights regarding web pages to companies that desire web page analysis.

Further, the web page viewing analysis system may input extraction conditions of the plurality of viewers, and extract at least some of the plurality of gaze point data based on the extraction conditions.

According to the above configuration, the company can understand in detail the browsing behavior of viewers who meet the extraction conditions, and therefore it is possible to provide the company with more detailed insight regarding web pages.

Further, the extraction conditions include age information of the plurality of viewers, gender information of the plurality of viewers, address information of the plurality of viewers, occupation information of the plurality of viewers, and The information may be associated with at least one of household number information of the viewers, marital information of the plurality of viewers, and educational background information of the plurality of viewers.

According to the above configuration, a user who matches the extraction condition associated with at least one of age information, gender information, address information, occupation information, household size information, marital information, and educational background information of multiple viewers Since it is possible to understand browsing behavior in detail, it is possible to provide companies with more detailed insights regarding web pages according to the attributes of the viewers.

Moreover, the gaze point visualization map may be a gaze point heat map.

According to the above, by checking the attention point heat map, the company can understand in detail the viewing behavior of the viewer with respect to the web page. In particular, the attention point heat map makes it possible to more clearly understand the viewer's attention area on a web page.

Moreover, the gaze point visualization map may be a gaze point heat map.

According to the above, by checking the attention point heat map, the company can understand in detail the viewing behavior of the viewer with respect to the web page. In particular, the attention point heat map makes it possible to more clearly understand the viewer's attention area on a web page.

Further, the web page viewing analysis system may display the information regarding the recommended viewers in a sorted or narrowed format.

Further, the information regarding the recommended viewer may include any one or more of the viewer's name, gender, age, length of stay when viewing the web page, whether or not a goal has been achieved, and time to achieve the goal.

Further, the comprehensive facial expression information may indicate a probability for each of the plurality of facial expression elements throughout the web page.

Further, the comprehensive facial expression information may indicate a probability transition corresponding to the time series in which the web page was viewed for each of the plurality of facial expression elements.

Further, the comprehensive interest level information may indicate a change in the interest level corresponding to a time series in which the web page was viewed.

A web page viewing analysis method according to one aspect of the present disclosure is a web page viewing analysis method for analyzing insights of viewers regarding a web page, and the method includes acquiring multiple gaze point data corresponding to each of multiple viewers. and a step of generating a gaze point visualization map that visualizes the gaze point when the viewer corresponding to the gaze point data views the web page, based on at least some of the plurality of gaze point data. , displaying the gaze point visualization map in a superimposed manner on the web page, and is executed by a computer.

A web page viewing analysis method according to an aspect of the present disclosure is a web page viewing analysis method for analyzing insights of viewers regarding a web page, and includes a step of acquiring a plurality of attribute data corresponding to each of a plurality of viewers. and a step of extracting a recommended viewer as a hearing destination based on at least some of the plurality of attribute data, and a step of displaying information about the recommended viewer, and the method is executed by a computer. .

A web page viewing analysis method according to one aspect of the present disclosure is a web page viewing analysis method for analyzing insights of viewers with respect to a web page, the method includes a step of acquiring a plurality of facial expression data corresponding to each of a plurality of viewers. a step of generating, based on at least some of the plurality of facial expression data, comprehensive facial expression information indicating a comprehensive facial expression when a viewer corresponding to the facial expression data views the web page; and displaying facial expression information, and is executed by a computer.

A web page viewing analysis method according to one aspect of the present disclosure is a web page viewing analysis method for analyzing insights of viewers regarding a web page, and includes a step of acquiring a plurality of facial expression data corresponding to each of a plurality of viewers. and a step of generating, based on at least some of the plurality of facial expression data, comprehensive interest level information indicating a comprehensive interest level when a viewer corresponding to the facial expression data views the web page; and displaying the comprehensive interest level information, and is executed by a computer.

Further, a web page viewing analysis program that causes a computer to execute the viewing analysis method may be provided.

According to the present disclosure, a web page viewing analysis system, a web page viewing analysis method, and a web page viewing analysis are capable of providing detailed insights regarding a web page from the viewer's behavior when viewing the web page based on biometric information. programs can be provided.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram illustrating an example of a configuration of a web page viewing analysis system according to an embodiment of the present disclosure (hereinafter referred to as the present embodiment). It is a diagram showing an example of the hardware configuration of a viewer terminal. It is a diagram showing an example of the hardware configuration of a server. FIG. 2 is a diagram illustrating an example of the hardware configuration of a company-side terminal. It is a sequence diagram for explaining a series of processes executed between a viewer terminal and a server in the viewing analysis system according to the present embodiment. FIG. 2 is a diagram for explaining a viewer's gaze point on a display area of a web page. FIG. 3 is a diagram for explaining facial feature points of a viewer. FIG. 2 is a sequence diagram for explaining a series of processes executed between a server and a company terminal in the web page viewing analysis system according to the present embodiment. It is a figure which shows an example of a web page browsing analysis screen. (a) is a diagram showing an example of a web page. (b) is a diagram showing an example of a gaze point heat map. (c) is a diagram showing an example of a composite web page in which a gaze point heat map is superimposed on a web page. It is a figure showing an example of an extraction condition display screen. It is a figure showing an example of a web page viewing analysis screen after extraction conditions are specified.

Hereinafter, a web page browsing analysis system 1 (hereinafter simply referred to as "browsing analysis system 1") according to the present embodiment will be described with reference to the drawings. FIG. 1 is a diagram showing an example of the configuration of a browsing analysis system 1 according to the present embodiment. As shown in FIG. 1, the viewing analysis system 1 includes viewer terminals 2a and 2b, a server 3, and a company terminal 4. These are connected to a communication network 8. Each of the viewer terminals 2 a and 2 b is communicably connected to the server 3 via a communication network 8 . The company terminal 4 is communicably connected to the server 3 via the communication network 8 . The communication network 8 is configured by at least one of a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, and a wireless core network.

The viewer terminal 2a is a terminal associated with the viewer Va, and is operated by the viewer Va. The viewer terminal 2b is a terminal associated with the viewer Vb, and is operated by the viewer Vb. In this embodiment, for convenience of explanation, the viewer terminals 2a and 2b may be collectively referred to as the viewer terminal 2. Similarly, viewers Va and Vb may be collectively referred to as viewer V. In addition, in this embodiment, a large number of viewer terminals 2 associated with a large number of viewers are provided in the viewing analysis system 1, but for convenience of explanation, two of the many viewer terminals Only terminals 2a, 2b are illustrated in FIG. The type of viewer terminal 2 is not particularly limited, and the viewer terminal 2 may be, for example, a smartphone, a personal computer, a tablet, or a wearable device (for example, a head-mounted display or an AR display).

Next, with reference to FIG. 2, the hardware configuration of the viewer terminal 2 will be described below. FIG. 2 is a diagram showing an example of the hardware configuration of the viewer terminal 2. As shown in FIG. As shown in FIG. 2, the viewer terminal 2 includes a control section 20, a storage device 21, an imaging section 22, a communication section 23, an input operation section 24, a display section 25, a speaker 26, and an RTC ( Real Time Clock) 28. These elements making up the viewer terminal 2 are connected to a communication bus 29.

The control unit 20 includes a memory and a processor. The memory is configured to store computer readable instructions (programs). For example, the memory includes a ROM (Read Only Memory) in which various programs are stored, a RAM (Random Access Memory) having a plurality of work areas in which various programs executed by a processor are stored, and the like. The processor includes, for example, at least one of a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit). The CPU may be composed of multiple CPU cores. A GPU may be configured by multiple GPU cores. The processor may be configured to load a designated program from various programs incorporated in the storage device 21 or the ROM onto the RAM, and execute various processes in cooperation with the RAM.

The storage device 21 is, for example, a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory, and is configured to store programs and various data. The imaging unit 22 is configured to acquire video data showing the surrounding environment of the viewer terminal 2 through photography. In particular, the imaging unit 22 is a camera configured to generate image data or video data showing the surrounding environment of the viewer terminal 2 through photography, and includes an image sensor (for example, a CCD sensor or a CMOS sensor), and an image sensor drive processing circuit. In this embodiment, the control unit 20 functions as a line-of-sight tracking unit that detects changes in the line of sight of the viewer V based on the video data acquired by the imaging unit 22. Further, the control unit 20 functions as a facial expression tracking unit that detects changes in the facial expression of the viewer V based on the video data.

The communication unit 23 includes a wireless communication module and/or a wired communication module for communicating with an external device connected to the communication network 8. The wireless communication module is configured to wirelessly communicate with an external device such as a base station or a wireless LAN router, and includes a transmitting/receiving antenna and a wireless transmitting/receiving circuit. The wireless communication module may be a wireless communication module compatible with short-range wireless communication standards such as Wi-Fi (registered trademark) or Bluetooth (registered trademark), or may be an X generation wireless communication module that uses a SIM (Subscriber Identity Module). It may be a wireless communication module compatible with a mobile communication system (for example, a fourth generation mobile communication system such as LTE).

The input operation section 24 is, for example, a touch panel, a mouse, and/or a keyboard arranged over the video display of the display section 25, and receives input operations from the viewer V and performs operations corresponding to the input operations. configured to generate a signal. The display unit 25 includes, for example, a video display and a video display circuit that drives and controls the video display. The display unit 25 has a display screen 125 on which a web page is displayed (see FIG. 6). The speaker 26 is configured to output sound to the outside based on the sound data. The RTC 28 is configured to obtain information indicating the current time.

In this embodiment, the viewer terminal 2 has a web browser configured to display web pages sent from the server 3. The web page is composed of HTML files, CSS files, image or video files, program files, and the like.

Returning to FIG. 1, the server 3 is communicably connected to the viewer terminal 2 and the company terminal 4 via the communication network 8. The server 3 sends a web page (for example, a landing page of a predetermined website) to each of the plurality of viewer terminals 2 via the communication network 8, and also sends the attention point heat map to the web page to the company terminal 4. Send a superimposed composite web page (described below). The server 3 may be composed of a plurality of servers. The server 3 functions as a web server configured to provide a web page viewing analysis application as a web application. In this regard, the server 3 sends data (for example, HTML files, CSS files, image and video files, program files, etc.) for displaying the web page browsing analysis screen on the web browser of the company terminal 4. It is configured. In this way, the server 3 functions as a server for providing SaaS (System as a Service). The server 3 may be constructed on-premises or may be a cloud server. Further, the server 3 functions as a data management server that manages a plurality of web pages, gaze point data of each viewer V, and facial expression data of each viewer V.

The hardware configuration of the server 3 will be described below with reference to FIG. 3. FIG. 3 is a diagram showing an example of the hardware configuration of the server 3. As shown in FIG. As shown in FIG. 3, the server 3 includes a control section 30, a storage device 31, an input/output interface 32, a communication section 33, an input operation section 34, and a display section 35. These elements making up the server 3 are connected to a communication bus 36.

The control unit 30 includes a memory and a processor. The memory is configured to store computer readable instructions. In particular, the memory may store a web page viewing analysis program for causing the processor to execute a series of processes (web page viewing analysis method) executed by the server 3 in FIGS. 5 and 8. The memory is composed of ROM and RAM. The processor includes at least one of a CPU, an MPU, and a GPU.

The storage device 31 is, for example, a storage device such as an HDD, SSD, or flash memory, and is configured to store programs and various data. The storage device 31 stores a plurality of web pages, web event data of each viewer V (event data such as clicks and scrolls), gaze point data of each viewer V, and facial expression data of each viewer V. There is. The storage device 31 also stores a viewer information table related to information of each viewer V and a user information table related to each user U who uses the web page viewing analysis application. The viewer information table includes attribute information of each viewer V. For example, the viewer information table may include at least one of identification information, gender information, age information, household number information, address information, and occupation information of each viewer V. Furthermore, the viewer information table may further include information such as the length of stay when viewing a web page, whether or not a goal is achieved, and the time at which the goal is achieved. The user information table may include identification information, attribute information, login information, etc. of each user U.

The input/output interface 32 is an interface that enables connection between an external device and the server 3, and includes an interface that conforms to a predetermined communication standard such as the USB standard or the HDMI (registered trademark) standard. The communication unit 33 may include various wired communication modules for communicating with external terminals on the communication network 8. The input operation unit 34 is, for example, a touch panel, a mouse, and/or a keyboard, and is configured to receive an input operation from an operator and to generate an operation signal according to the input operation from the operator. The display unit 35 includes, for example, a video display and a video display circuit.

The company terminal 4 is a terminal operated by a user U who uses a web page viewing analysis application provided by the server 3. In this embodiment, a plurality of company-side terminals 4 are provided in the browsing analysis system 1 (in other words, in this embodiment, a plurality of users U are using the web page browsing analysis application). For convenience of explanation, only one company terminal 4 is shown in FIG. 1.

Referring to FIG. 4, the hardware configuration of the company terminal 4 will be described below. FIG. 4 is a diagram showing an example of the hardware configuration of the company terminal 4. As shown in FIG. As shown in FIG. 4, the company terminal 4 may be, for example, a personal computer, a smartphone, a tablet, or a wearable device attached to the user U. The company terminal 4 has a WEB browser. It is assumed that the web page viewing analysis application operates on the web browser of the company terminal 4. The company terminal 4 includes a control section 40, a storage device 41, an input/output interface 42, a communication section 43, an input operation section 44, and a display section 45. These elements are connected to a communication bus 46.

The control unit 40 includes a memory and a processor. The memory is configured to store computer readable instructions (programs). For example, the memory is composed of ROM and RAM. The processor includes, for example, at least one of a CPU, an MPU, and a GPU.

The storage device 41 is, for example, a storage device such as an HDD, SSD, or flash memory, and is configured to store programs and various data. The input/output interface 42 is an interface (eg, USB, HDMI, etc.) that enables connection between an external device and the company terminal 4. The communication unit 43 is configured to connect the company terminal 4 to the communication network 8, and includes a wireless communication module and/or a wired communication module. The input operation unit 44 is, for example, a touch panel, a mouse, and/or a keyboard, and is configured to receive input operations from the user U and to generate operation signals in accordance with the input operations performed by the user U. The display section 45 includes, for example, a video display and a video display circuit. In this embodiment, a web page viewing analysis screen 70 (see FIG. 9) is displayed on the display unit 45.

Next, a series of processes executed between each of the plurality of viewer terminals 2 and the server 3 in the viewing analysis system 1 according to the present embodiment will be described with reference to FIG. 5. FIG. 5 is a sequence diagram for explaining a series of processes executed between the viewer terminal 2 and the server 3 in the viewing analysis system 1. For convenience of explanation, a series of processes executed between a single viewer terminal 2 and the server 3 is illustrated in FIG. 5, but in this embodiment, each of the plurality of viewer terminals 2 is It is assumed that a series of processes shown in 5 are executed. In addition, in the series of processes shown in FIG. 5, the viewer V, through viewing a predetermined web page provided by the server 3, transmits the gaze point data and facial expression data of the viewer V regarding the predetermined web page to the server 3. shall be provided. In this regard, the operating company that operates the server 3 and provides the web page viewing analysis application may provide the viewer V with an incentive for viewing the web page. For example, the company may award each viewer V with predetermined points or electronic money for each viewing of a web page for a predetermined period of time (for example, one minute).

Furthermore, the operating company may give each viewer V a mission to not only view the web page for a predetermined period of time but also to purchase a specific product posted on the web page. In this case, the operating company can obtain detailed insights regarding the viewing behavior (gaze and facial expressions) of each viewer V in the process of purchasing a specific product on the web page. The operating company can freely set the conditions for viewing the web page given to the viewer, depending on the nature and type of the viewer insight to be obtained.

As shown in FIG. 5, in step S1, the viewer terminal 2 sends a web page viewing request to the server to view the web page in response to the viewer V's input operation on the input operation unit 24 (for example, a touch panel). Send to 3. Next, after receiving the web page viewing request from the viewer terminal 2, the server 3 transmits the web page (HTML file, CSS file, image or video file, program file, etc.) stored in the storage device 31 to the viewer. It is transmitted to terminal 2 (step S2).

Next, the viewer terminal 2 displays the web page on the web browser (step S3). As shown in FIG. 6, the viewer terminal 2 displays the web page on the display screen 125 of the display unit 25. Furthermore, the viewer terminal 2 starts acquiring gaze point data and facial expression data of the viewer V at the same time as displaying the web page (step S4). As shown in step S5, the viewer terminal 2 continues to acquire the gaze point data and facial expression data of the viewer V until the display of the web page is finished. On the other hand, the viewer terminal 2 finishes acquiring the gaze point data and facial expression data of the viewer V when the display of the web page is finished, and then acquires the gaze point data, facial expression data, and attribute data (collected in advance). Information such as the viewer's name, gender, age, length of stay when viewing the web page, whether or not the goal was achieved, time to achieve the goal, etc.) is transmitted to the server 3 (step S6). In this regard, the display time of the web page displayed on the web browser may be set in advance to a predetermined time (for example, one minute). Further, the web page may be displayed on the web browser of the viewer terminal 2 until the viewer V achieves a specific mission (goal) on the web page. Achievement of a mission (goal) is, for example, completion of a purchase action for a specific product. Furthermore, among the attribute data of the viewer V, attribute information such as name, gender, age, etc. is acquired in response to an input operation by the viewer V on a predetermined display screen displayed on the viewer terminal 2.

<Method for acquiring gaze point data>
The viewer terminal 2 acquires video data showing the viewer V through photography using the imaging unit 22, and then acquires gaze point data of the viewer V based on the video data. In this respect, the control unit 20 of the viewer terminal 2 functions as a line-of-sight tracking unit that detects changes in the line of sight of the viewer V based on the video data acquired by the imaging unit 22. As an example of a method for acquiring gaze point data, the control unit 20 identifies the viewer V and the eyes of the viewer V shown in the predetermined frame based on a predetermined frame (still image) of the video data, and then Then, position information indicating the position of the viewer V's eyes with respect to the imaging unit 22 is acquired. Next, the control unit 20 specifies the rotation angle of the iris of the eyeball of the viewer V based on the predetermined frame, and then transmits line-of-sight information indicating the direction of the line of sight L (see FIG. 6) of the viewer V. get. Further, the control unit 20 acquires face orientation information indicating the direction of the viewer V's face based on the predetermined frame. Note that the control unit 20 may acquire line-of-sight information based on a three-dimensional eyeball model (passive method), or may acquire line-of-sight information based on a corneal reflex method (active method). When line of sight information is acquired based on the corneal reflection method, the viewer terminal 2 further includes an infrared light emitting element (for example, an infrared LED) configured to emit infrared rays and an infrared camera (infrared image sensor). Good too.

Next, the control unit 20 determines the point of gaze P, which is the point where the line of sight L of the viewer V and the display screen 125 of the viewer terminal 2 intersect, based on the position information, line of sight information, and face direction information. Get the position (X, Y) (XY coordinates). Here, the position of the gaze point P where the line of sight L and the display screen 125 intersect may be acquired as a position on the XY coordinate space set on the display screen 125. In this way, the control unit 20 can acquire the position of the gaze point P where the line of sight L and the display screen 125 intersect from a predetermined frame of video data. The control unit 20 may acquire the position of the point of interest P at an acquisition rate corresponding to the frame rate of the video data, or may acquire the position of the point of interest P at an acquisition rate different from the frame rate. For example, when the frame rate of video data is 30 fps, the acquisition rate of the gaze point P may be 30 Hz, or may be an acquisition rate lower than 30 Hz (for example, 15 Hz).

Next, the control unit 20 changes the position of the gaze point P of the viewer V from the coordinate position on the XY coordinate space set on the display screen 125 to the coordinate position on the XY coordinate space set on the display area S of the web page. Convert to In this regard, as shown in FIG. and FIG. 10(a)) slides in the +Y-axis direction. Similarly, when the viewer V performs a flick operation on the display screen 125 in the −Y-axis direction, the display area S of the web page displayed on the display screen 125 slides in the −Y-axis direction. When the viewer V performs a flick operation on the display screen 125 in the +X-axis direction, the display area S of the web page displayed on the display screen 125 slides in the +X-axis direction. Similarly, when the viewer V performs a flick operation on the display screen 125 in the -X axis direction, the display area S of the web page displayed on the display screen 125 slides in the -X axis direction.

In this way, since the display area S of the web page displayed on the display screen 125 changes according to the flick operation of the viewer V, the position of the gaze point P of the viewer V can be changed from the XY coordinate position of the display screen 125. It is necessary to convert to the XY coordinate position of the display area S. Specifically, when the display area S of the web page slides in response to a flick operation by the viewer V, the control unit 20 controls the correspondence between the XY coordinates of the display area S and the XY coordinates of the display screen 125. Update. Thereafter, when the control unit 20 acquires the XY coordinate position of the gaze point P on the display screen 125, the control unit 20 controls the display screen 125 based on the correspondence between the XY coordinates of the display area S and the XY coordinates of the display screen 125. The XY coordinate position of is converted to the XY coordinate position of the display area S.

Furthermore, the control unit 20 acquires the position of the gaze point P on the display area S as well as the viewing time of the web page when the position of the gaze point P was acquired. Here, the web page viewing time refers to the elapsed time since the web page viewing started. For example, when 10 seconds have passed since the start of viewing a web page, the viewing time is 10 seconds. If the viewing time of the web page when the position (X1, Y1) of the gaze point P1 is acquired is t1, the position (X1, Y1) of the gaze point P1 on the display area S and the viewing time t1 are different from each other. The position (X1, Y1) of the gaze point P1 and the viewing time t1 are stored in an associated state. In this manner, the control unit 20 can acquire point-of-gaze data indicating temporal changes in the position of the point of interest P on the display area S. For example, if the point of interest acquisition rate is 30 Hz and the web page viewing time is 30 seconds, the number of points of interest P included in the point of interest data is 900.

<Facial expression data acquisition method>
The viewer terminal 2 acquires video data showing the viewer V through photography using the imaging unit 22, and then acquires facial expression data of the viewer V based on the video data. In this respect, the control unit 20 of the viewer terminal 2 functions as a facial expression tracking unit that detects changes in the facial expression of the viewer V based on the video data acquired by the imaging unit 22. Here, the facial expression data is data indicating temporal changes in the facial expression information of the viewer V. More specifically, the facial expression data indicates temporal changes in probability information of viewer V's facial expression elements (in this example, five types of facial expression elements: happy, sad, neutral (no expression), angry, and surprised). It is data.

As an example of a method for acquiring facial expression data, the control unit 20 identifies the face of the viewer V shown in the predetermined frame based on a predetermined frame (still image) of the video data M, and then identifies the face of the viewer V shown in the predetermined frame. A plurality of facial feature points K included in V's face are acquired (see FIG. 7). As shown in FIG. 7, in this example, 24 facial feature points K related to each of the eyebrows, eyes, nose, and mouth are acquired. Next, the control unit 20 calculates a plurality of facial feature amounts based on the plurality of acquired facial feature points K, and then combines the calculated facial feature amounts with an expression recognition model (learning model). Based on this, probability information of each facial expression element of the viewer V is acquired. In this facial expression recognition model, the value of each unit in the input layer indicates a corresponding one of multiple facial features, and the value of each unit in the output layer indicates the probability of a corresponding one of multiple facial features. Show information. The probability information for each facial expression element may be expressed as a percentage. In particular, the probability of the expression element ``happy,'' the probability of the expression element ``sad,'' the probability of the expression element ``neutral,'' the probability of the expression element ``angry,'' and the probability of the expression element ``surprised.'' Expressed as percentage %. In this way, the control unit 20 can acquire probability information of each facial expression element of the viewer V from a predetermined frame.

The control unit 20 may acquire the facial expression information of the viewer V at an acquisition rate corresponding to the frame rate of the video data, or may acquire the facial expression information of the viewer V at a rate different from the frame rate. . For example, when the frame rate of video data is 30 fps, the acquisition rate of facial expression information may be 30 Hz, or may be an acquisition rate lower than 30 Hz (for example, 15 Hz).

The control unit 20 also acquires the facial expression information as well as the viewing time of the web page when the facial expression information was acquired. For example, if the viewing time of the web page when the probability information of each facial expression element is acquired is t1, the probability information of each facial expression element and the viewing time t1 are associated with each other. is memorized. In this way, the control unit 20 can acquire facial expression data indicating temporal changes in the facial expression information of the viewer V.

Returning to FIG. 5, in step S7, the server 3 receives the gaze point data, facial expression data, and attribute data of the viewer V regarding the web page from each viewer terminal 2, and then receives the gaze point data of each viewer V. The gaze point data, facial expression data, and attribute data of each viewer V are stored in the storage device 31 in a state in which the facial expression data and attribute data are associated with the web page to be viewed. Note that in the example shown in FIG. 5, one type of web page is transmitted to each viewer terminal 2 for convenience of explanation, but multiple types of web pages may be transmitted to each viewer terminal 2. In this case, the gaze point data and facial expression data may be associated with identification information of the viewer and identification information of the web page (eg, URL of the web page). For example, the gaze point data and facial expression data of the viewer Va transmitted from the viewer terminal 2a may be associated with the identification information of the viewer Va and the identification information of the web page targeted for viewing. Similarly, the gaze point data and facial expression data of the viewer Vb transmitted from the viewer terminal 2b may be associated with the identification information of the viewer Vb and the identification information of the web page targeted for viewing. In this way, through the series of processes shown in FIG. 5, it becomes possible to accumulate the gaze point data and facial expression data of each viewer V in the server 3.

Next, with reference to FIG. 8, a series of processes executed between the server 3 and the company terminal 4 in the browsing analysis system 1 according to this embodiment will be described below. FIG. 8 is a sequence diagram for explaining a series of processes executed between the server 3 and the company terminal 4 in the viewing analysis system 1. In this example, the user U who operates the company terminal 4 is engaged in digital marketing work related to web pages, and the web page is based on the biometric information (point of gaze and facial expressions) of multiple viewers V who viewed the web page. want to understand detailed insights. The web pages viewed by each viewer V in the series of processes shown in FIG. 5 are web pages related to user U's digital marketing business.

As shown in FIG. 8, in step S10, the company terminal 4 transmits user U's login information (eg, login ID and login password) to the server 3 through user U's input operation. In step S11, the server 3 authenticates the user U based on the login information sent from the company terminal 4. Note that the authentication method for user U is not limited to login authentication. After authenticating the user U, the server 3 identifies the web page associated with the user U by referring to the user information table stored in the storage device 31.

In step S12, the server 3 transmits data for displaying the web page viewing analysis screen 70 shown in FIG. 9 (hereinafter referred to as web page viewing analysis screen data) to the company terminal 4 via the communication network 8. The web page viewing analysis screen data is a file (for example, an HTML file, a CSS file, a program file, an image or video file, etc.) for displaying the web page viewing analysis screen 70 on the web browser of the company terminal 4. Good too. Here, the program file may be a program file executable on a web browser such as JavaScript.

As shown in FIG. 9, the web page viewing analysis screen 70 includes a web page display area 71, a viewer display area 76, an expression information display area 72, and an expression/interest level information display area 73. Further, the web page viewing analysis screen 70 includes a viewer selection button 74, a web page selection button 75, and a website selection button 78. A composite web page 63, which will be described later, is displayed in the web page display area 71. In the viewer display area 76, information regarding viewers recommended as interview destinations among the viewers of the web page is displayed. In the facial expression information display area 72, comprehensive facial expression information in a radar chart format is displayed. The facial expression/interest level information display area 73 displays comprehensive facial expression information of the time series graph and comprehensive interest level information of the time series graph.

In step S13, the company terminal 4 selects the extraction conditions for the selected web page and the viewer V displayed on the web page viewing analysis screen 70 in response to the user U's input operation on the web page viewing analysis screen 70. Specifically, the company terminal 4 selects one of the plurality of websites in response to the user U's input operation on the website selection button 78. In response to an input operation on the website selection button 78, a plurality of website candidates may be displayed on the web page viewing analysis screen 70. One or more websites that can be selected through the website selection button 78 are websites associated with user U. Thereafter, the company terminal 4 selects the selected web page displayed in the web page display area 71 from among the plurality of web pages related to the selected website in response to the user U's input operation on the web page selection button 75. select. In response to an input operation on the web page selection button 75, a plurality of web page candidates associated with the selected website may be displayed in the form of a URL on the web page display area 71. Next, in response to the user's input operation on the viewer selection button 74, extraction conditions for viewer V are selected. More specifically, as shown in FIG. 11, the company terminal 4 displays an extraction condition display screen 80 for specifying extraction conditions for the viewer V in response to the user U's input operation on the viewer selection button 74. is displayed on the web page viewing analysis screen 70.

In the extraction condition display screen 80 shown in FIG. 11, the extraction condition of the viewer V is associated with the gender information of the viewer V, the age information of the viewer V, and the information on the number of people in the viewer V's household. For example, if a viewer V in his or her 40s is selected as the extraction condition, the server 3 acquires the gaze point data of the viewer V in his or her 40s in step S15. Further, the user U can specify an extraction condition associated with at least one of gender information, age information, and household information through the extraction condition display screen 80. For example, as shown in FIG. 12, when the user U specifies the extraction condition (item name: target) regarding a woman in her 20s living in a one-person household, in step S15, the server 3 determines that Obtain gaze point data of viewer V.

Note that although FIG. 12 shows three examples of extraction conditions for viewer V: gender information, age information, and household number information, the present embodiment should not be limited to these. In this regard, the address information, occupation information, marriage information, and educational background information of the viewer V may be further associated with the extraction conditions of the viewer V. In this way, the extraction condition for viewer V may be associated with at least one of viewer V's age information, gender information, address information, occupation information, household number information, marital information, and educational background information. .

Further, the extraction conditions for the viewer V may be further associated with at least one of the viewer V's questionnaire information, survey information, and past behavior history information (for example, frequency of going to the movie theater, etc.).

Next, in step S14, the company terminal 4 requests the server 3 to transmit a composite web page in response to the user U's input operation. In this regard, in response to the user U's input operation, the company terminal 4 transmits to the server 3 a request to transmit a composite web page associated with the selected web page selected through the web page selection button 75. Further, the company terminal 4 transmits information regarding the extraction conditions of the viewer V to the server 3 along with a request to transmit the composite web page.

In step S15, the server 3 receives the synthetic web page transmission request and the information regarding the extraction conditions of the viewer V from the company terminal 4, and stores the gaze point data of the viewer V that matches the extraction conditions in the storage device. Extracted from 31. In this regard, the server 3 extracts the identification information of the viewers V that matches the extraction conditions by referring to the viewer information table related to the information of each viewer V. After that, the server 3 extracts the gaze point data associated with both the identification information of the selected web page and the extracted identification information of the viewer V. For example, as shown in FIG. 12, if the extraction conditions for viewer V are age: 20s, gender: female, and number of people in the household: one person, the server 3 selects the following: age: 20s, gender: female, In addition, after identifying the identification information of the viewer V who satisfies all the conditions of household size: one person household, the viewer V associated with both the identification information of the identified viewer V and the identification information of the selected web page. Extract the gaze point data. On the other hand, as shown in FIG. 9, when the extraction condition for viewer V is all viewers, the gaze point data of viewer V associated with the identification information of the selected web page is extracted.

In step S16, the server 3 generates a gaze point heat map (gazing point visualization An example of a map) is generated. In the gaze point heat map, temporal changes in a collection of gaze points P of a plurality of viewers V are visualized. The frame rate of the gaze point heat map may be the same as the acquisition rate of the gaze point data. For example, if the gaze points of the viewers V1 to Vn (n is an integer of 2 or more) at the viewing time t1 of the web page are P1 to Pn, the gaze points P1 to Pn are visualized in the gaze point heat map at the viewing time t1. ing. In the gaze point heat map, when there are many gaze points that overlap a predetermined area (that is, when a predetermined area is an area that is gazed at by multiple viewers V), the predetermined area is colored in a warm color such as red. May be colored with. On the other hand, if the number of gaze points that overlap a predetermined area is small (that is, if the predetermined area is an area that is not often gazed at by multiple viewers V), the predetermined area is colored in a cool color such as blue. May be colored. In this way, as the number of gaze points overlapping a predetermined region increases, the coloring of the predetermined region may gradually change from a cold color to a warm color. Furthermore, if there are no gaze points included in a predetermined area (that is, if the predetermined area is an area that is not gazed at by a plurality of viewers V at all), the predetermined area does not need to be colored. In this respect, a region containing only a gaze point treated as noise (outlier) may not be colored at all.

In addition, in this embodiment, when the extraction condition for the viewer V is all viewers, a gaze point heat map that visualizes temporal changes in the positions of the gaze points P of all viewers V is generated. . In this way, the generated gaze point heat map changes depending on the extraction conditions of the viewer V (in other words, the attributes of the viewer V).

In step S17, the server 3 generates a composite web page 63 in which the gaze point heat map 62 is superimposed on the web page 60 (see FIG. 10). In particular, the server 3 generates a composite web page 63 in which the gaze point heat map 62 is superimposed on the web page 60. In step S18, the server 3 transmits the composite web page 63 to the company terminal 4. After receiving the composite web page 63 from the server 3, the company terminal 4 displays the composite web page 63 in the web page display area 71 of the web page viewing analysis screen 70. In this way, the composite web page 63 is reproduced on the company terminal 4 (step S19).

In this embodiment, a gaze point heat map is described as an example of a gaze point visualization map that visualizes temporal changes in the gaze points of a plurality of viewers V, but the gaze point visualization map is not a gaze point heat map. It is not limited. In this regard, the line of sight map may be used as another example of the gaze point visualization map. In the gaze flow line map, flow lines indicating movement trajectories of gaze points P of a plurality of viewers V are visualized. The line-of-sight line map may be displayed in a still image format or a moving image format.

In step S20, the server 3 transmits information regarding each viewer V to the company terminal 4 based on the attribute data of each viewer V recorded in the viewer information table. The company terminal 4 displays information regarding each viewer V in the viewer display area 76 on the web page analysis screen 70 (step S21). The viewer display area 76 displays each viewer V's name, gender, age, etc., as well as information regarding each viewer V's stay time on the web page, and information regarding whether each viewer V has achieved a goal (mission). Information regarding the time required for each viewer V to achieve the goal (mission) is displayed. These pieces of information may be displayed in a sorted or narrowed format, for example, depending on the information of the item selected by the user's instruction. Furthermore, a playback button 176 for viewing the gaze point heat map 62 of each viewer V and a contact button 177 for contacting each viewer V are displayed. The contact button 177 may be associated with the viewer V's e-mail address or mobile number. By operating the playback button 176 associated with a predetermined viewer V, the user U can view a composite web page 63 in which the gaze point heat map 62 associated with the predetermined viewer V is superimposed on the web page 60. It may be displayed in the web page display area 71. More specifically, the company terminal 4 transmits information regarding a predetermined viewer V to the server 3 in response to the input operation of the playback button 176 by the user U. Thereafter, the server 3 generates a gaze point heat map 62 that visualizes temporal changes in the position of the gaze point P of the predetermined viewer V based on the received gaze point data of the predetermined viewer V. Thereafter, the server 3 transmits a composite web page 63 in which the generated gaze point heat map 62 is superimposed on the web page 60 to the company terminal 4. In this way, the updated composite web page 63 is displayed in the web page display area 71.

Next, in step S22, the server 3 extracts facial expression data of the viewer V that matches the extraction conditions from the storage device 31. In this regard, the server 3 extracts the identification information of the viewers V that matches the extraction conditions by referring to the viewer information table related to the information of each viewer V. Thereafter, the server 3 extracts facial expression data associated with both the identification information of the selected web page and the extracted identification information of the viewer V. For example, as shown in FIG. 12, if the extraction conditions for viewer V are age: 20s, gender: female, and number of people in the household: one person, the server 3 selects the following: age: 20s, gender: female, In addition, after identifying the identification information of the viewer V who satisfies all the conditions of household size: one person household, the viewer V associated with both the identification information of the identified viewer V and the identification information of the selected web page. Extract facial expression data. On the other hand, as shown in FIG. 9, when the extraction condition for viewer V is all viewers, the facial expression data of viewer V associated with the identification information of the selected web page is extracted.

In step S23, the server 3 generates comprehensive facial expression information and comprehensive interest level information based on the facial expression data of viewer V extracted in step S22. Thereafter, the server 3 transmits the generated comprehensive facial expression information and comprehensive interest level information to the company terminal 4. As for the transmission timing of these pieces of information, these pieces of information may be sent after the composite web page 63 is reproduced. As shown in FIG. 9, the company terminal 4 displays comprehensive facial expression information displayed in a radar chart format in the facial expression information display area 72 of the web page viewing analysis screen 70, and also displays it in a time series graph format. The comprehensive facial expression information is displayed in the facial expression/interest level information display area 73 of the web page viewing analysis screen 70 (step S24). Further, the company terminal 4 displays comprehensive interest level information displayed as a time series graph in the facial expression/interest level information display area 73.

The comprehensive facial expression information indicates comprehensive facial expression information of the viewer V that matches the extraction conditions, and includes comprehensive facial expression information in a radar chart format and comprehensive facial expression information in a time series graph format. The comprehensive facial expression information in the form of a radar chart indicates the average probability information (percentage %) of each comprehensive facial expression element (happy, sad, neutral, angry, surprised) of the viewer V that matches the extraction conditions. For example, the comprehensive facial expression information in the form of a radar chart includes the average probability of the facial expression element "happy" of multiple viewers V that matches the extraction conditions, the average probability of the facial expression element "sad" of the multiple viewers V, and The average probability of the facial expression element "neutral" of the plurality of viewers V, the average probability of the facial expression element "angry" of the plurality of viewers V, the average probability of the facial expression element "surprise" of the plurality of viewers V, May include.

For example, when the extracted viewers V include viewers Va, Vb, and Vc, the server 3 analyzes each facial expression element of the viewer Va based on facial expression data indicating temporal changes in each facial expression element of the viewer Va. Determine the average probability of Similarly, the server 3 determines the average probability of each facial expression element of the viewer Vb based on facial expression data indicating temporal changes in each facial expression element of the viewer Vb, and also determines the average probability of each facial expression element of the viewer Vc. The average probability of each facial expression element of the viewer Vc is determined based on the facial expression data indicating a change in the facial expression. Thereafter, the server 3 determines the overall average probability of each facial expression element of the viewers Va to Vc based on the average probability of each facial expression element of the viewers Va to Vc. In this regard, if the average probabilities of the facial expression element "happy" of viewers Va, Vb, and Vc are Ra1, Rb1, and Rc1, then the average probability of the overall facial expression element "happy" of viewers Va to Vc is Rave1. is Rave1=(Ra1+Rb1+Rc1)/3. Similarly, if the average probabilities of the facial expression element "sad" for viewers Va, Vb, and Vc are Ra2, Rb2, and Rc2, the average probability Rave2 of the overall facial expression element "sad" for viewers Va to Vc is , Rave2=(Ra2+Rb2+Rc2)/3.

The server 3 may also calculate the positive degree and negative degree of the extracted viewer V with respect to the web page 60 as a percentage based on the average probability of each comprehensive facial expression element of the extracted viewer V. good. Furthermore, the server 3 may transmit information indicating the positive degree and the negative degree to the company terminal 4. As shown in FIG. 9, information indicating the degree of positivity and degree of negativity may be displayed within the facial expression information display area 72. For example, when the average probability of the facial expression element "happy" is high, the value of the positivity level may be large. On the other hand, when the average probability of the facial expression element "anger" is high, the value of the degree of negativity may become large. The server 3 uses a relational expression or a learning model in which the overall average probability of each facial expression element is set as an input value, and the positive degree and negative degree values are set as output values. The positive degree and the negative degree may be calculated from the average probability of facial expression elements.

The comprehensive facial expression information in the form of a time-series graph indicates temporal changes in comprehensive facial expression elements (happy, sad, neutral, angry, surprised) of the viewer V that match the extraction conditions. For example, the comprehensive facial expression information in a time-series graph format includes information indicating the temporal change in the average probability of the facial expression element "happy" of a plurality of viewers V that matches the extraction conditions, and information indicating the temporal change in the average probability of the facial expression element "happy" of the plurality of viewers V who match the extraction conditions. information indicating temporal changes in the average probability of "sad", information indicating temporal changes in the average probability of the facial expression element "neutral" of the plurality of viewers V, and information indicating temporal changes of the average probability of the facial expression element "neutral" of the plurality of viewers V, and , and information indicating a temporal change in the average probability of the facial expression element "surprise" of the plurality of viewers V.

For example, when the extracted viewers V include viewers Va, Vb, and Vc, the server 3 stores information indicating the temporal change in the average probability of the facial expression element "happy" of viewers Va to Vc, and Information indicating the temporal change in the average probability of the facial expression element "sad" of ~Vc, information indicating the temporal change of the average probability of the facial expression element "neutral" of the viewers Va~Vc, and the facial expressions of the viewers Va~Vc After calculating the information indicating the temporal change in the average probability of the element "anger" and the information indicating the temporal change in the average probability of the facial expression element "surprise" of viewers Va to Vc, these pieces of information are combined in a time series. It is sent to the company terminal 4 as comprehensive facial expression information in graph format.

Further, the server 3 may calculate comprehensive interest level information in a time series graph format based on the comprehensive facial expression information in a time series graph format. The comprehensive interest level information indicates a temporal change in the overall interest level (percentage %) of the viewer V who meets the extraction conditions in the web page 60. For example, if the overall average probability of each facial expression element at the viewing time t1 is R1 to R5, the overall interest level I at the viewing time t1 may be calculated based on the average probabilities R1 to R5. In this regard, the server 3 uses a relational expression or a learning model in which the overall average probabilities R1 to R5 of each facial expression element are set as input values, and the overall interest level I is set as an output value. Then, the overall interest level I may be calculated from the overall average probabilities R1 to R5 of each facial expression element.

In addition, in this embodiment, when the extraction condition for the viewer V is all viewers V, the comprehensive facial expression information indicates the comprehensive facial expression information of all the viewers V, and the comprehensive interest level information indicates all the viewers V. 3 shows a temporal change in the overall interest level of the viewer V in the web page 60. In this way, the comprehensive expression information and the comprehensive interest level information change according to the extraction conditions of the viewer V (in other words, the attributes of the viewer V to be extracted).

Furthermore, in the present embodiment, the comprehensive interest level information of the plurality of viewers V is calculated based on the comprehensive facial expression information (in particular, the temporal change in the average probability of each facial expression element of the viewer V). The degree information may be calculated based on the gaze point data of each viewer V. Specifically, the comprehensive interest level information is calculated based on the interest level data of each viewer V, and the interest level data of the viewer V is calculated based on the viewer V's gaze point data and the viewer V's blinks. It may be calculated based on information regarding the viewer V and drowsiness information of the viewer V. The information regarding the viewer V's blinking may be information indicating the number of times the viewer V blinks or information indicating whether or not the viewer V blinks. Information regarding viewer V's blinking may be specified based on each frame of viewer V's video data. The drowsiness information of the viewer V may be information indicating whether or not the viewer V is drowsy. The sleepiness information of the viewer V may be specified based on each frame of the viewer V's video data. For example, the interest level data of the viewer V includes information regarding whether the position of the gaze point P of the viewer V exists within the display area S of the video, information regarding the presence or absence of sleepiness of the viewer V, and information regarding the presence or absence of sleepiness of the viewer V. It may be calculated based on information regarding the presence or absence of blinking. Based on the interest level data of each viewer V, it becomes possible to calculate comprehensive interest level information of a plurality of viewers V.

Furthermore, in this embodiment, the information indicating the positive degree and the negative degree of the plurality of viewers V is calculated based on the facial expression data of each viewer V, but this information is calculated based on the gaze point of each viewer V. It may be calculated based on data.

Note that in this embodiment, the order of each process shown in FIG. 8 is not particularly limited. In addition, in the examples shown in FIGS. 9 and 12, a composite web page 63 in which the gaze point heat map 62 is superimposed on the web page 60 is displayed in the web page display area 71; however, in the web page display area 71, Web page 60 may be displayed instead of composite web page 63. Even in this case, comprehensive facial expression information in the form of a radar chart is displayed in the facial expression information display area 72, and comprehensive facial expression information and comprehensive interest level information in the form of a time series graph are displayed in the facial expression/interest level information display area 73. may be displayed.

In addition, in this example, two pieces of information, the comprehensive facial expression information and the comprehensive interest level information, are displayed on the web page viewing analysis screen 70, but one of the two pieces of information, the comprehensive facial expression information and the comprehensive interest level information. Only the information may be displayed on the web page viewing analysis screen 70. Further, in this example, five types of facial expression elements, happy, sad, neutral, angry, and surprised, are employed as facial expression information, but the types of facial expression elements are not limited to these. For example, fear, happiness, enjoyment, disgust, etc. may be additionally employed as facial expression elements other than these.

Further, according to the present embodiment, the user U can select a predetermined region of the gaze point heat map 62 included in the composite web page 63 through the company terminal 4, thereby allowing the user U to Information on the viewer V associated with the viewpoint P may be acquired from the server 3. Specifically, after the company terminal 4 transmits information regarding the selected predetermined area to the server 3, the server 3 identifies a set of gaze points P included in the predetermined area based on the information regarding the predetermined area. do. Thereafter, the server 3 specifies the attribute information of the viewer V associated with the specified set of gaze points P, and then transmits the attribute information of the viewer V to the company terminal 4. In this way, attribute information regarding the viewer V who is interested in a predetermined area (for example, a specific object) of the gaze point heat map 62 can be utilized for web marketing.

According to the present embodiment, since the composite web page 63 in which the gaze point heat map 62 is superimposed on the web page 60 is presented to the company terminal 4, the user U can view the information displayed on the web page viewing analysis screen 70. By checking the composite web page 63, it becomes possible to understand in detail the viewing behavior of a plurality of viewers V with respect to the web page 60. In particular, it becomes possible for the user U to grasp in detail objects of interest (areas of interest) within the web page 60 by a plurality of viewers V. In this way, it is possible to provide detailed insights regarding the web page 60 (more specifically, detailed insights regarding the digital marketing of the web page) to companies that desire web page analysis. For example, the user U can know whether each viewer V is paying attention to an inquiry button or a purchase button arranged on the web page 60. Furthermore, the user U can take measures to improve the CVR (conversion rate) of the web page (for example, optimal placement of inquiry buttons and purchase buttons) through insights regarding the web page.

In particular, in this embodiment, based on the gaze point data of viewers V that match extraction conditions associated with age information, gender information, household size information, etc. of multiple viewers V, viewing that satisfies these attribute conditions is performed. A gaze point heat map that visualizes temporal changes in the position of the gaze point of person V is generated. In this way, it is possible to provide the company with more detailed insight regarding the web page 60 according to the attributes of the viewer.

Furthermore, according to the present embodiment, since comprehensive facial expression information and comprehensive interest level information in a radar chart format and a time series graph format are presented to the company terminal 4, the user U can view the information displayed on the web page viewing analysis screen 70. By checking these pieces of information, it is possible to understand in detail the browsing behavior of the plurality of viewers V with respect to the web page 60. In particular, the user U can grasp in detail the facial expressions (happy, sad, neutral, angry, surprised, etc.) and interest level of the viewer V toward the web page 60 for each playback time. In this way, it is possible to provide detailed insights regarding web pages (more specifically, detailed insights regarding digital marketing of web pages) to companies that desire web page analysis.

In particular, in this embodiment, based on facial expression data of viewers V who match extraction conditions associated with age information, gender information, household size information, etc. of a plurality of viewers V, viewers who meet these attribute conditions are selected. Comprehensive facial expression information indicating the probability information of each comprehensive facial expression element of V is generated, and comprehensive interest level information indicating the overall interest level of the viewer V who meets the extraction conditions in the web page 60 is generated. . In this way, through this information, it is possible to understand in detail the browsing behavior of the viewer V who meets the extraction conditions, making it possible to provide the company with more detailed insights regarding the web page 60. .

Although the embodiments of the present invention have been described above, the technical scope of the present invention should not be interpreted to be limited by the description of the embodiments. This embodiment is an example, and those skilled in the art will understand that various changes can be made within the scope of the invention as set forth in the claims. The technical scope of the present invention should be determined based on the scope of the invention described in the claims and the scope of equivalents thereof.

1: Web page viewing analysis system, 2, 2a, 2b: viewer terminal, 3: server, 4: company terminal, 8: communication network, 20: control unit, 21: storage device, 22: imaging unit, 23: Communication unit, 24: Input operation unit, 25: Display unit, 26: Speaker, RTC: 28, 30: Control unit, 31: Storage device, 32: Input/output interface, 33: Communication unit, 34: Input operation unit, 35 : Display unit, 40: Control unit, 41: Storage device, 42: Input/output interface, 43: Communication unit, 44: Input operation unit, 45: Display unit, 60: Web page, 62: Gaze point heat map, 63: Synthetic web page, 70: Web page viewing analysis screen, 71: Web page display area, 72: Facial expression information display area, 73: Interest level information display area, 74: Viewer selection button, 75: Web page selection button, 76: Viewer display area, 78: Website selection button, 80: Extraction condition display screen, 125: Display screen, 176: Play button, 177: Contact button, K: Facial feature points, L: Line of sight, M: Video data, P : Point of interest, S: Display area, U: User, V, Va, Vb: Viewer

Claims (9)

A web page browsing analysis system that analyzes insights of multiple viewers on a web page,
a plurality of viewer terminals, each of which is associated with a corresponding one of the plurality of viewers and includes a display unit;
a server communicably connected to the plurality of viewer terminals via a network;
a company-side terminal communicably connected to the server via the network and equipped with a display unit;
The web page browsing analysis system is
providing the web page to each of the plurality of viewer terminals;
Displaying the web page on the display section of each of the plurality of viewer terminals,
obtaining a plurality of facial expression data each indicating a temporal change in the probability of a plurality of facial expression elements of a corresponding one of the plurality of viewers of the web page;
receiving extraction conditions for the plurality of viewers from the company terminal;
Extracting some of the plurality of facial expression data based on the extraction conditions,
After integrating the extracted several facial expression data, a composite showing the overall facial expression in the time axis when the plurality of viewers viewed the web page based on the several integrated facial expression data. Generate facial information,
displaying the comprehensive facial expression information on a display section of the company terminal;
Web page viewing analysis system. A web page browsing analysis system that analyzes insights of multiple viewers on a web page,
a plurality of viewer terminals, each of which is associated with a corresponding one of the plurality of viewers and includes a display unit;
a server communicably connected to the plurality of viewer terminals via a network;
a company-side terminal communicably connected to the server via the network and equipped with a display unit;
The web page browsing analysis system is
providing the web page to each of the plurality of viewer terminals;
Displaying the web page on the display section of each of the plurality of viewer terminals,
obtaining a plurality of facial expression data each indicating a temporal change in the probability of a plurality of facial expression elements of a corresponding one of the plurality of viewers of the web page;
receiving extraction conditions for the plurality of viewers from the company terminal;
Extracting some of the plurality of facial expression data based on the extraction conditions,
After integrating the extracted facial expression data, the overall interest level in the time axis when the plurality of viewers viewed the web page is indicated based on the integrated facial expression data. Generate comprehensive interest information,
displaying the comprehensive interest level information on a display section of the company terminal;
Web page viewing analysis system. The extraction conditions are:
Age information of the plurality of viewers;
Gender information of the plurality of viewers;
Address information of the plurality of viewers;
Occupational information of the plurality of viewers;
Information on the number of people in the household of the plurality of viewers;
Marriage information of the plurality of viewers;
Academic background information of the plurality of viewers;
associated with at least one of
The web page viewing analysis system according to claim 1 or 2 . 2. The web page viewing analysis system according to claim 1 , wherein the comprehensive facial expression information indicates the average probability of each facial expression element of the plurality of viewers throughout the web page. 2. The web page viewing analysis system according to claim 1 , wherein the comprehensive facial expression information indicates a temporal change in the average probability of each facial expression element of the plurality of viewers along a time series in which the web page was viewed. 3. The web page viewing analysis system according to claim 2 , wherein the comprehensive interest level information indicates a temporal change in the comprehensive interest level of the plurality of viewers along the time series in which the web page was viewed. A web page viewing analysis method that analyzes insights of multiple viewers to a web page and is executed by a web page viewing analysis system, the method comprising:
The web page browsing analysis system is
a plurality of viewer terminals, each of which is associated with a corresponding one of the plurality of viewers and includes a display unit;
a server communicably connected to the plurality of viewer terminals via a network;
a company-side terminal communicably connected to the server via the network and equipped with a display unit;
The web page browsing analysis method is as follows:
providing the web page to each of the plurality of viewer terminals;
Displaying the web page on a display unit of each of the plurality of viewer terminals;
obtaining a plurality of facial expression data, each representing a temporal change in the probability of a plurality of facial expression elements for a corresponding one of the plurality of viewers of the web page;
receiving extraction conditions for the plurality of viewers from the company terminal;
extracting some of the plurality of facial expression data based on the extraction conditions;
After integrating the extracted several facial expression data, a composite showing the overall facial expression in the time axis when the plurality of viewers viewed the web page based on the several integrated facial expression data. a step of generating facial expression information;
displaying the comprehensive facial expression information on a display section of the company terminal;
Web page browsing analysis methods, including A web page viewing analysis method that analyzes insights of multiple viewers to a web page and is executed by a web page viewing analysis system, the method comprising:
The web page browsing analysis system is
a plurality of viewer terminals, each of which is associated with a corresponding one of the plurality of viewers and includes a display unit;
a server communicably connected to the plurality of viewer terminals via a network;
a company-side terminal communicably connected to the server via the network and equipped with a display unit;
The web page browsing analysis method is as follows:
providing the web page to each of the plurality of viewer terminals;
Displaying the web page on a display unit of each of the plurality of viewer terminals;
obtaining a plurality of facial expression data, each representing a temporal change in the probability of a plurality of facial expression elements for a corresponding one of the plurality of viewers of the web page;
receiving extraction conditions for the plurality of viewers from the company terminal;
extracting some of the plurality of facial expression data based on the extraction conditions;
After integrating the extracted facial expression data, the overall interest level in the time axis when the plurality of viewers viewed the web page is indicated based on the integrated facial expression data. a step of generating comprehensive interest level information;
displaying the comprehensive interest level information on a display section of the company terminal;
Web page browsing analysis methods, including A web page viewing analysis program that causes a computer to execute the web page viewing analysis method according to claim 7 or 8 .

Images