JP6586904B2 - Image display device, display control method, and display control program - Google Patents

Description

  The present invention relates to an image display device, a display control method, and a display control program.

  In recent years, there is a technique in which a user selects a product from the displayed list by displaying a list of images related to the product on a display unit, and purchases the selected product. As a method for displaying a product list for a user, there is a method that uses a user's line of sight. For example, the product being watched by the user is detected, and the detected product is displayed in another area in the product list screen.

JP 2012-22589 A

  However, in the above-described method, for example, a product list is displayed on one menu screen based on the user's line of sight, but a suitable menu screen is provided based on the user's line of sight when another menu screen is displayed. It is difficult.

  In one aspect, the present invention provides an image display device, a display control method, and a display control program that can provide a suitable menu screen.

  In one aspect, the image display device includes a reception unit, a determination unit, and a switching unit. A reception part receives operation from a user with respect to the 1st menu screen containing the image regarding goods displayed on the display part. The determination unit determines the display content of the second menu screen based on the user's line of sight and operation with respect to the first menu screen. The switching unit switches the display content of the display unit to the second menu screen on which the display content is determined.

  A suitable menu screen can be provided.

FIG. 1 is a block diagram illustrating an example of the configuration of the image display apparatus according to the first embodiment. FIG. 2 is a diagram illustrating an example of gaze detection. FIG. 3 is a diagram illustrating an example of the product DB. FIG. 4 is a diagram illustrating an example of a user operation on the first menu screen. FIG. 5 is a diagram illustrating an example of the second menu screen. FIG. 6 is a flowchart illustrating an example of image display processing according to the first embodiment. FIG. 7 is a block diagram illustrating an example of the configuration of the image display apparatus according to the second embodiment. FIG. 8 is a diagram illustrating an example of a user operation on the menu screen. FIG. 9 is a diagram illustrating another example of the user operation on the menu screen. FIG. 10 is a diagram illustrating an example of a user operation on the detail screen. FIG. 11 is a diagram illustrating an example of a menu screen. FIG. 12 is a flowchart illustrating an example of image display processing according to the second embodiment. FIG. 13 is a diagram illustrating an example of a computer that executes a display control program.

  Hereinafter, embodiments of an image display device, a display control method, and a display control program disclosed in the present application will be described in detail based on the drawings. The disclosed technology is not limited by the present embodiment. Further, the following embodiments may be appropriately combined within a consistent range.

  FIG. 1 is a block diagram illustrating an example of the configuration of the image display apparatus according to the first embodiment. The image display device 100 illustrated in FIG. 1 accepts an operation from the user for a first menu screen that is displayed on the display unit and includes an image related to a product. The image display device 100 determines the display content of the second menu screen based on the user's line of sight and operation with respect to the first menu screen. The image display device 100 switches the display content of the display unit to the second menu screen on which the display content is determined. Thereby, the image display apparatus 100 can provide a suitable menu screen.

  An image display apparatus 100 illustrated in FIG. 1 includes a line-of-sight detection unit 111, a display unit 112, a reception unit 113, a product database (product DB) 120, and a control unit 130. Note that the image display apparatus 100 may include various functional units included in a known computer, for example, functional units such as various communication devices and audio output devices, in addition to the functional units illustrated in FIG. As an example of the image display device 100, a ticket vending machine that sells meal tickets at restaurants can be adopted. The image display device 100 can employ not only the above-described ticket vending machine but also a store-installed terminal such as a ticket vending machine, a vending machine, or a multimedia terminal installed in a convenience store store. The image display device 100 can also employ a terminal that can be connected to a network such as a personal computer, a tablet terminal, or a smartphone in addition to the above-described store-installed type terminal.

  For example, the line-of-sight detection unit 111 detects the user's line of sight based on a captured image obtained by capturing the user. The captured image is, for example, an image obtained by capturing an upper body of a user irradiated with infrared rays from an infrared LED (Light Emitting Diode) with an infrared camera. The line-of-sight detection unit 111 may acquire a captured image from an infrared camera (not shown), or the line-of-sight detection unit 111 may include an infrared LED and an infrared camera (not shown). The captured image may not be an image captured by an infrared camera, and may be an image captured by a visible light camera, for example.

  The line-of-sight detection unit 111 may detect other information that can be used in place of the line of sight as the line of sight. For example, the user's face orientation may be determined based on a captured image obtained by capturing the user, and the front direction of the face may be detected as the line-of-sight direction. Further, the orientation of the user's body may be determined based on a captured image obtained by capturing the user, and the front side front direction of the trunk may be detected as the line-of-sight direction. In addition, the user's face orientation is determined using the three-dimensional position information of the user's face measured using a device capable of measuring the distance to the user's face, that is, depth information, and the front direction of the face May be detected as the line-of-sight direction.

  The line-of-sight detection unit 111 detects, for example, a position where infrared rays are reflected on the cornea of the user (corneal reflection) and a position of the user's pupil from captured images captured at predetermined intervals. The line-of-sight detection unit 111 detects the user's line of sight based on the corneal reflection and the position of the pupil. The line-of-sight detection unit 111 outputs the detected line of sight to the determination unit 140. FIG. 2 is a diagram illustrating an example of gaze detection. As shown in FIG. 2, the line-of-sight detection unit 111 detects the line-of-sight position (X1, Y1) at time t1.

  When the user's line of sight moves from time t1 to time t2, the line-of-sight detection unit 111 moves the line-of-sight position (X2, Y2) away from the line-of-sight position (X1, Y1) at the time t2. Is detected. In this case, the line segment connecting the line-of-sight positions (X1, Y1) and (X2, Y2) is the locus of movement of the user's line of sight. When the user's line of sight does not move between time t2 and time t3, the line-of-sight detection unit 111 detects the same line-of-sight position (X2, Y2) as time t2 at time t3. Here, for example, when the line of sight is detected at the same position a plurality of times within a predetermined time, or when the line of sight is detected at the same position continuously for a predetermined number of times or more, it is determined that the vehicle is stopped. it can. In this case, the line-of-sight position (X2, Y2) is the line-of-sight stop position. Note that the line-of-sight position detected when the line of sight stops is not necessarily the same line-of-sight position as the previous time, and the distance to the line-of-sight position at the previous time may be a predetermined distance or less. In this case, for example, the center of gravity of the plurality of line-of-sight positions may be set as the stop position. The line-of-sight position is, for example, a position on the XY coordinate plane including the screen displayed on the display unit 112. For example, the line-of-sight detection unit 111 associates the time with the line-of-sight position and outputs the associated time to the control unit 130.

  Returning to the description of FIG. 1, the display unit 112 is a display device for displaying various types of information. The display unit 112 is realized by, for example, a liquid crystal display as a display device. The display unit 112 displays various screens such as first and second menu screens input from the control unit 130. Here, the first and second menu screens are menu screens for displaying products provided in the store where the image display device 100 is installed. The first menu screen is a menu screen that is displayed first when the user stands in front of the image display device 100, for example. The second menu screen is a menu screen to be displayed next to the first menu screen.

  The accepting unit 113 is an input device that accepts various operations from the user. The reception unit 113 is realized by a touch panel or the like as an input device, for example. In this case, the display device of the display unit 112 and the input device of the reception unit 113 may be integrated. In addition, the reception unit 113 may be realized by a button, a keyboard, a mouse, or the like as an input device, for example. Further, for example, it may be realized by a gesture recognition device or a voice recognition device. For example, when the first menu screen is displayed on the display unit 112, the receiving unit 113 receives a transition operation to the second menu screen. In addition, the reception unit 113 receives a selection operation for selecting a product such as purchase of a product included in the first menu screen, for example. The accepting unit 113 outputs an operation input by the user to the control unit 130.

  The product DB 120 is a database that manages products provided at a store where the image display device 100 is installed. The product DB 120 stores product features for each product. FIG. 3 is a diagram illustrating an example of the product DB. As illustrated in FIG. 3, the product DB 120 includes items such as “product name” and “price”.

  The product DB 120 stores information indicating product features such as product price, raw material, presence / absence of a discount coupon, and the sale time of the product for each product A, B. In FIG. 3, the product DB 120 stores features for each item of the product, such as a product name “product A”, a price “1200 yen”, a raw material “milk, butter”, and the like. Note that the product features stored in the product DB 120 are not limited to the example illustrated in FIG. 3. For example, the product DB 120 may store information indicating whether the product genre, the product color, or the new product is a feature.

  Returning to the description of FIG. 1, the control unit 130 executes, for example, a program stored in an internal storage device using a RAM as a work area by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like. Is realized. Further, the control unit 130 may be realized by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The control unit 130 includes a determination unit 140 and a switching unit 150, and realizes or executes information processing functions and operations described below. Note that the internal configuration of the control unit 130 is not limited to the configuration illustrated in FIG. 1, and may be another configuration as long as the information processing described below is performed.

  The determination unit 140 determines display contents to be displayed on the display unit 112 based on the detection result of the line-of-sight detection unit 111. The determining unit 140 includes a specifying unit 141, an extracting unit 142, and a product determining unit 143.

  When the first menu screen is displayed on the display unit 112 and the user's line-of-sight position is input from the line-of-sight detection unit 111, the specifying unit 141 stops the user's line of sight based on the input line-of-sight position. Identify the product Here, the first menu screen is a screen on which, for example, a list of products sold in the store is displayed, and the user stands in front of the display unit 112 of the image display device 100, whereby the line-of-sight detection unit 111 is displayed. Is a screen displayed on the display unit 112 when the user's line of sight is detected.

  FIG. 4 is a diagram illustrating an example of a user operation on the first menu screen. In addition, although the example of each menu screen after FIG. 4 including the example which displays the image of goods is shown, it is not necessary to limit a display object to this. For example, the display target may be not only a still image of a product but also an image or video with animation. Note that the number of products and images corresponding to the products is not limited, and may be one or more.

  As illustrated in FIG. 4, the first menu screen 10 includes, for example, a plurality of images 11A to 11F related to a plurality of products A to F. The images 11A to 11F include the features of the corresponding products A to F. The images 11A to 11F include, for example, product names and prices of the products A to F. Note that the images 11A to 11F can include not only the above-described product name and price, but also information related to the product such as images showing the appearance of the product, raw materials, and the size of the product. Further, the first menu screen 10 includes a button 12 that receives a transition to the next menu screen.

  The straight line shown in FIG. 4 shows the locus of the line of sight when the line of sight is moving. A circle indicates a stop position when the line of sight is stopped. The size of the circle indicates the stop time of the line of sight, and the stop time is longer as the diameter of the circle is larger. In FIG. 4, the user moves the line of sight while sequentially keeping a line of sight with the product B, the product C, the product D, the product E, and the product F in order from the product A. In addition, the user stops the line of sight on the product F, moves the line of sight to the purchase button 13A, and touches the purchase button 13A to purchase the product A. After purchasing the product A, the user touches the button 12 that receives the screen transition operation.

  The specifying unit 141 specifies a product whose line of sight has stopped and has not received a product selection operation from the user. For example, the specifying unit 141 specifies the product F having the longest stop time from the products B to F that the user has not purchased. You may identify goods based on the sum total of the stop time length of the gaze measured within predetermined time. Note that the product specified by the specifying unit 141 is not limited to the product F with the longest stop time, and may be a product with a stop time equal to or longer than a specified time, for example. The specifying unit 141 outputs the specified product F to the extracting unit 142 as the specified product.

  The extraction unit 142 extracts the feature of the specific product F specified by the specification unit 141. For example, the extraction unit 142 refers to the product DB 120 and extracts features related to the specific product F. The extraction unit 142 outputs the extracted features to the product determination unit 143.

  The features extracted by the extraction unit 142 are not limited to the above features. The extraction unit 142 may extract features based on information that can be detected from the line-of-sight change state, such as the line-of-sight stop position, the number of stops, the stop time, and the trajectory with respect to the first menu screen 10. For example, as illustrated in FIG. 4, the extraction unit 142 is characterized by a keyword included in the product name “product F”, for example, when the stop time of the product name “product F” is the longest. May be extracted as

  In the example shown in FIG. 4, the line of sight is stopped on the product names and prices of the products A, B, and F, but the line of sight is not stopped on the product names of the products C to E. The line of sight is stopped. Since the line of sight is stopped at the prices of the products A to F, it is understood that the user is interested in the price of the products. In particular, because products A, B, and F are not only priced but also in the product name, they are interested in products in the same price range, which is one of the common features of products A, B, and F. It can be seen that Therefore, the extraction unit 142 may extract the prices “1200 yen” and “1100 yen” of the products A, B, and F as features.

  In addition, for example, the extraction unit 142 may extract features common to the products B and F that are not selected as purchased items although there are a plurality of stop positions. That is, you may make it extract the characteristic common to the goods which were not selected as a purchasing object, although it is a goods estimated that a user will be interested. Further, the extraction unit 142 may extract features excluding the features of the product A that is finally purchased, for example. Note that the number of features extracted by the extraction unit 142 is not limited to one. For example, the extraction unit 142 may extract a plurality of features related to a plurality of items such as price and size. In addition, when the stop time length of the line of sight for each product on the screen is substantially equal, a predetermined product previously determined as the second menu screen may be determined as the display product.

  Returning to the description of FIG. 1, the product determination unit 143 determines a display product to be displayed on the second menu screen based on the feature extracted by the extraction unit 142. For example, the product determination unit 143 inquires of the product DB 120 about a product including the keyword extracted by the extraction unit 142. Note that the extraction unit 142 may inquire the product DB 120 for products in a range including the extracted features. When the extraction unit 142 extracts the price “1200 yen” as a feature, the product determination unit 143 may inquire the product DB 120 for a product having a price of “1200 yen” or less, for example.

  The product determination unit 143 receives information on the product having the feature inquired from the product DB 120. For example, when inquiring about a product including the extracted keyword in the product DB 120, the product determination unit 143 receives information on the product G from the product DB 120. The product determination unit 143 determines the received product G as a display product. As shown in FIG. 4, the user has not purchased the specific product F, but has kept his line of sight longest. Therefore, the user does not intend to purchase the specific product F itself, or did not select the product F as a purchase target after being confused with the product A, but it is considered that the feature of the specific product F is interested. Therefore, the product determination unit 143 determines the product G including the feature of the specific product F extracted by the extraction unit 142 as a display product.

  In addition, for example, when the number of products received from the product DB 120 is larger than the number of products that can be displayed on the second menu screen, the product determination unit 143 sets the product having characteristics closer to the characteristics of the specific product as the display product. You may decide. In addition, the product determination unit 143 may determine, as a display product, a product that the store wants to sell, such as a seasonally limited product or a product that allows the use of a discount coupon, from the received product information. . Here, the product determination unit 143 determines a product different from the product included in the first menu screen as the display product.

  The product determination unit 143 may determine the display product based on information other than the movement of the line of sight. For example, even if the product determination unit 143 is a product that does not include the feature extracted by the extraction unit 142, a product that the store wants to sell, such as a seasonally limited product or a product that can use a discount coupon, is displayed as the display product. You may decide.

  The product determination unit 143 determines a product to be highlighted from among the products determined to be displayed on the second menu screen. For example, based on the product information received from the product DB 120, the product determination unit 143 determines a product that highlights a product that the store wants to sell, such as a seasonally limited product or a product that can be used with a discount coupon. In addition, the product determination unit 143 may determine a product that highlights a product having characteristics closer to those of the line-of-sight stopped product. For example, the product determination unit 143 determines a product whose characteristics more closely match the characteristics of the line-of-sight-stopped product as a product to be highlighted. The product determination unit 143 outputs information regarding the displayed product and the product to be highlighted to the switching unit 150. Note that the product determination unit 143 may determine a product to be displayed on the second menu screen and not perform highlighting.

  For example, the product determination unit 143 may determine a product to be displayed on the first menu screen 10. For example, when the user stands in front of the display unit 112 and the line-of-sight detection unit 111 starts detecting the line of sight of the user, the product determination unit 143 determines a product to be displayed on the first menu screen 10. In this case, the product determination unit 143 determines a predetermined product as a product to be displayed on the first menu screen 10.

  Returning to the description of FIG. 1, the switching unit 150 switches the screen displayed on the display unit 112. When the first menu screen 10 shown in FIG. 4 is displayed, when the user performs a touch operation on the button 12 that accepts a transition operation to the next menu screen, the switching unit 150 starts from the first menu screen 10 to the first menu screen 10. Switch to the 2 menu screen. FIG. 5 is a diagram illustrating an example of the second menu screen. As illustrated in FIG. 5, the switching unit 150 displays the second menu screen 20 including the images 11G to 11L related to the products G to L determined by the product determination unit 143 on the display unit 112. The second menu screen 20 may include a “return” button for accepting a transition to the first menu screen and a button 12 for accepting a transition to the next menu screen. As for the screen transition, the operation instruction may be received by any method as long as an instruction to switch the display from the currently displayed screen to another screen can be received. For example, instead of the button format as shown, a screen to be displayed may be selected by a tab.

  The switching unit 150 may highlight and display the product determined to be highlighted by the product determination unit 143 on the display unit 112. For example, the switching unit 150 highlights the product G by displaying a frame on the image 11G related to the product G. Note that the switching unit 150 can take various forms of highlighting, for example, change the size and color of the image related to the product, blink the image, or superimpose an image indicating additional information on the image related to the product. By doing so, the product can be highlighted. If the product to be highlighted is displayed in a way that makes it easier to attract the user's attention compared to other products on the screen, what kind of display method and how to change the display are adopted. You can also.

  Next, the operation of the image display apparatus 100 according to the first embodiment will be described. FIG. 6 is a flowchart illustrating an example of image display processing according to the first embodiment.

  When the user stands in front of the image display device 100, the line-of-sight detection unit 111 starts detecting the line of sight of the user (step S1). When the line-of-sight detection unit 111 starts line-of-sight detection, the switching unit 150 displays the first menu screen 10 on the display unit 112 (step S2). The line-of-sight detection unit 111 detects the user's line-of-sight position with respect to the first menu screen 10 (step S3). The line-of-sight detection unit 111 outputs the detected line-of-sight position to the determination unit 140.

  Based on the operation information input from the reception unit 113, the switching unit 150 determines whether or not a product to be purchased is selected by the user (step S4). If no product has been selected (No at Step S4), the switching unit 150 outputs an instruction to continue the detection of the line-of-sight position to the line-of-sight detection unit 111, and returns to Step S3. When a product is selected (step S4: affirmative), the switching unit 150 outputs a specific instruction to the determination unit 140. When a specifying instruction is input, the specifying unit 141 of the determining unit 140 specifies a product whose line of sight has stopped from among the products not selected as purchase targets based on the line-of-sight position detected by the line-of-sight detection unit 111 ( Step S5). The switching unit 150 determines whether the accepting unit 113 accepts a transition operation to the next menu screen while the display unit 112 displays the first menu screen 10 (step S6). When the receiving unit 113 has not received a transition operation (No at Step S6), the switching unit 150 waits for an operation received by the user and returns to Step S3. When the accepting unit 113 accepts the transition operation (step S6: affirmative), the switching unit 150 inquires of the determining unit 140 about a product to be displayed on the second menu screen 20.

  The extracting unit 142 of the determining unit 140 extracts the feature of the product specified by the specifying unit 141 (Step S7). The extraction unit 142 outputs the extracted features to the product determination unit 143.

  The product determination unit 143 determines a product to be displayed on the second menu screen 20 based on the feature extracted by the extraction unit 142 (step S8). The product determination unit 143 acquires information on the product including the feature extracted by the extraction unit 142 from the product DB 120. The product determination unit 143 determines a product to be displayed on the second menu screen 20 based on the acquired information regarding the product. The product determination unit 143 outputs information regarding the determined product to the switching unit 150. The switching unit 150 displays the second menu screen 20 including an image related to the product determined by the product determination unit 143 on the display unit 112 (step S9). Thereby, the image display apparatus 100 can provide a suitable menu screen.

  As described above, the image display apparatus 100 accepts an operation from the user for the first menu screen that is displayed on the display unit 112 and includes the image related to the product. The image display apparatus 100 determines the display content of the second menu screen based on the user's line of sight and operation with respect to the first menu screen. Further, the image display device 100 switches the display content of the display unit 112 to the second menu screen on which the display content is determined. As a result, a suitable menu screen can be provided.

  Further, the image display apparatus 100 identifies a product whose line of sight has stopped and has not received a product selection operation from the user. In addition, the image display device 100 extracts the characteristics of the specified product. Further, the image display device 100 determines a product based on the extracted features. Further, the image display device 100 switches to the second menu screen including an image related to the determined product. As a result, a suitable menu screen can be provided.

  The image display device 100 determines a product based on at least one of the line-of-sight stop position, stop time, and locus. As a result, a menu screen corresponding to the feature can be provided. As described above, the image display device 100 can provide a suitable menu screen. Thereby, for example, other products related to the products that the user is interested in but have not selected in the first menu screen can be included in the display of the second menu screen. Therefore, the display content of the second menu screen can be changed to a content that the user will be interested in, and the action of purchasing the product G can be expected.

  In the first embodiment, the display content of the second menu screen 20 is determined based on the user's movement with respect to the first menu screen 10, but the display content of the menu screen is determined based on the user's movement with respect to the product detail screen. You may decide. The embodiment in this case will be described as Example 2. FIG. 7 is a block diagram illustrating an example of the configuration of the image display apparatus according to the second embodiment. Note that the same components as those of the image display apparatus 100 according to the first embodiment are denoted by the same reference numerals, and the description of the overlapping configuration and operation is omitted.

  The image display device 200 according to the second embodiment includes a control unit 230 instead of the control unit 130 of the image display device 100 according to the first embodiment. The control unit 230 includes a determination unit 240 and a switching unit 250.

  The switching unit 250 switches the menu screen to the detail screen when the receiving unit 113 receives a transition operation to the detail screen indicating the detailed information of the product. FIG. 8 is a diagram illustrating an example of a user operation on the menu screen. As shown in FIG. 8, the menu screen 30 includes images 11A to 11F related to the products A to F. The images 11A to 11F include a detail button for receiving a transition operation for transitioning to the detail screen of each product in addition to the feature of the product such as a price. The example in FIG. 8 simply shows an example in which the “details” button is displayed in the product image. Naturally, for example, a “purchase” button for accepting selection as a purchase target is provided. It may be included in the product image. The user selects a product by touching the detail button. When the user touches the detail button included in the product A, the switching unit 250 switches the screen displayed on the display unit 112 from the menu screen 30 to the detail screen related to the product A corresponding to the detail button.

  The switching unit 250 switches the detail screen to the menu screen when the receiving unit 113 receives a transition operation from the detail screen to the menu screen. In this case, the switching unit 250 displays a menu screen of the display content determined by the determination unit 240 on the display unit 112.

  Returning to the description of FIG. 7, the determination unit 240 determines the display content of the menu screen based on the user's line of sight with respect to the menu screen and the detail screen. The determining unit 240 includes a specifying unit 241, an extracting unit 242, and a product determining unit 243.

  For example, based on the user's line of sight with respect to the menu screen, the specifying unit 241 specifies a product that the user has not selected and whose line of sight has stopped. FIG. 9 is a diagram illustrating another example of the user operation on the menu screen. In FIG. 9, the user moves his / her line of sight while stopping clockwise from the upper left side of the screen in the order of the products A to C and F to D. Further, the user stops the line of sight on the product D, moves the line of sight to the detail button of the product A, and touches the detail button of the product A.

  The identifying unit 241 identifies a product whose line of sight has stopped and has not received a product selection operation from the user. For example, the specifying unit 241 specifies the product F having the longest stop time from among the products B to F for which the user has not touched the detail button. Note that the product specified by the specifying unit 241 is not limited to the product F having the longest stop time, as in the first embodiment, and may be, for example, a product whose stop time is equal to or longer than a specified time. The specifying unit 241 outputs the specified product F to the extracting unit 242 as the specified product.

  Returning to the description of FIG. 7, the extraction unit 242 extracts the feature of the product based on the user's line of sight with respect to the detailed screen including the detailed information of the product. FIG. 10 is a diagram illustrating an example of a user operation on the detail screen. As illustrated in FIG. 10, the detailed screen 40 includes, for example, an image 31 indicating “product A” as a product name, information on allergic substances included in the product A, and the like. The detail screen 40 also includes a return button 32 that accepts a transition operation to the menu screen, and a purchase button 33 that accepts a purchase operation for the product A. The detailed screen may be displayed as an overlay on the first menu screen, or may be displayed as a screen different from the first menu screen. In the example of FIG. 10, the purchase button 33 is included in the detail screen 40. However, the purchase button 33 is not included in the detail screen and is provided in the product image of the first menu screen at the transition source. The purchase button may be used to accept selection of a purchase target product. In FIG. 10, the user moves from the product name included in the detailed screen 40 to the allergy information in order while stopping the line of sight. Thereafter, the user stops the line of sight on “milk” which is an allergic substance, and touches the return button 32. In this case, for example, the extraction unit 242 extracts the feature of the product A based on the stop position immediately before the return button 32 is touched. In FIG. 10, “allergen: milk” is extracted as a feature. The extraction unit 242 outputs the extracted features to the product determination unit 243.

  Further, the extraction unit 242 extracts features of the specific product F specified by the specification unit 241. For example, the extraction unit 242 refers to the product DB 120 and extracts features related to the specific product F. The extraction unit 242 outputs the extracted features to the product determination unit 243. Note that the feature extracted by the extraction unit 242 is not limited to this. The extraction unit 242 may extract the characteristics of the specific product F in the same manner as the extraction unit 142 illustrated in FIG.

  Returning to the description of FIG. 7, the product determination unit 243 determines the display product to be displayed on the menu screen based on the features extracted by the extraction unit 242. For example, the product determination unit 243 refers to the product DB 120 and determines a product that does not include the feature extracted based on the line of sight on the detail screen 40 as a display product. In FIG. 10, the user touches the return button 32 without purchasing the product A while the detailed screen 40 of the product A is displayed. In this case, it is considered that the user has stopped purchasing the product A and returned to the menu screen because the feature that was viewed just before touching the return button 32 is included. That is, it is considered that the user is interested in a product that does not include the feature noticed immediately before the return button 32 is touched. Therefore, the product determination unit 243 refers to the product DB 120 and determines a product that does not include the “milk” feature extracted by the extraction unit 242 as a display product on the menu screen. FIG. 11 is a diagram illustrating an example of a menu screen. As illustrated in FIG. 11, the product determination unit 243 determines the products B, C, G, and J to L that do not include the feature “milk” as display products. The product determination unit 243 outputs the determined display products B, C, G, J to L to the switching unit 250. When the display products B, C, G, and J to L are input from the product determination unit 243, the switching unit 250 displays the images 11B, 11C, 11G, and 11J to 11L related to the display products B, C, G, and J to L. The included menu screen 50 is displayed on the display unit 112.

  In addition, the product determination unit 243 may determine, for example, a product including features extracted by the extraction unit 242 based on a line of sight with respect to the menu screen 30 as a display product. In this case, for example, the product determination unit 243 refers to the product DB 120 and determines a product including the feature extracted based on the menu screen 30 as a display product without including the feature extracted based on the detailed screen 40.

  The product determination unit 243 may determine the display product based on information other than the movement of the line of sight. For example, even if the product determination unit 243 does not include the feature extracted by the extraction unit 242, a product that the store wants to sell, such as a seasonally limited product or a product that can be used with a discount coupon, is used as a display product. You may decide.

  In addition, the product determination unit 243 deletes, for example, a product that is included in the menu screen 30 and includes features extracted based on the detailed screen 40 from the display product, or less noticeable than other products. You may make it be a non-highlighting display. As an example of non-highlighted display, for example, other product images are in color, whereas non-highlighted product images are made monochrome or lightness is reduced. In this case, the product determination unit 243 determines the products B to E that do not include “milk” from the products A to F illustrated in FIG. In addition, for example, the product determination unit 243 refers to the product DB 120 and selects a display product in place of the products A and F removed from the display product. The product determination unit 243 determines a product including the feature extracted based on the menu screen 30 as a display product without including the feature extracted based on the detailed screen 40 as the display product instead of the products A and F.

  Next, the operation of the image display apparatus 200 according to the second embodiment will be described. FIG. 12 is a flowchart illustrating an example of image display processing according to the second embodiment. In the following description, the processing in steps S1 and S3 is the same as that in the first embodiment, and thus the description thereof is omitted.

  The image display apparatus 200 performs the following process following the process of step S1. The switching unit 250 displays a menu screen on the display unit 112 (step S21). After performing the line-of-sight detection in step S3, the specifying unit 241 specifies a product on which the line of sight has stopped based on the line of sight with respect to the menu screen (step S22). The switching unit 250 determines whether or not the user has accepted a transition operation to the product detail screen (step S23). If the switching unit 250 has not accepted the transition operation (No at Step S23), the switching unit 250 outputs an instruction to continue the detection of the line-of-sight position to the line-of-sight detection unit 111, and returns to Step S3. When the switching unit 250 receives a transition operation (Yes at Step S23), the switching unit 250 displays a detailed screen on the display unit 112 (Step S24). The line-of-sight detection unit 111 detects the user's line of sight with respect to the detailed screen (step S25).

  Based on the operation information input from the reception unit 113, the switching unit 250 determines whether or not a product to be purchased is selected by the user (step S26). When the product is selected (Step S26: Yes), the switching unit 250 ends the process. When the product is not selected (No at Step S26), the switching unit 250 determines whether the accepting unit 113 has accepted a transition operation to the menu screen (Step S27). If the switching unit 250 has not received a transition operation (No at Step S27), the switching unit 250 outputs an instruction to continue the detection of the line-of-sight position to the line-of-sight detection unit 111, and returns to Step S25. When the switching unit 250 receives a transition operation (Yes at Step S <b> 27), the switching unit 250 outputs a feature extraction instruction to the extraction unit 242. When the feature extraction instruction is input, the extraction unit 242 extracts features based on the user's line of sight with respect to the detailed screen, and extracts the features of the product identified in step S22 (step S28).

  The product determination unit 243 determines a display product to be displayed on the menu screen based on the features extracted by the extraction unit 242 (step S29). The switching unit 250 displays a menu screen including an image indicating the product determined by the product determination unit 243 on the display unit 112 (step S30). Thereby, the image display apparatus 200 can provide a suitable menu screen.

  As described above, the image display device 200 extracts the feature of the product based on the user's line of sight with respect to the detailed screen including the detailed information of the product. As a result, a menu screen reflecting the detailed information of the product can be provided.

  Further, the image display apparatus 200 determines a product that does not include the extracted feature. As a result, it is possible to provide a menu screen that reflects information indicating a negative reaction by the user.

  In each of the above embodiments, the product to be displayed on the second menu screen 20 or the menu screen 50 is determined based on the user's line of sight with respect to the product image displayed on the first menu screen 10 or the menu screen 30. It is not limited to this. For example, when an advertisement such as a recommended product is posted around the display unit 112, a product to be displayed on the second menu screen 20 or the menu screen 50 may be determined based on the line of sight of the advertisement. . In other words, the specifying units 141 and 241 may specify a plurality of products displayed on the first menu image 10 or the menu screen 30 or a product whose line of sight has stopped among products displayed in the advertisement. . Note that the advertisement does not necessarily have to be placed around the display unit 112. It is only necessary that the user's line of sight is easy to face, and the user's line of sight can be detected. That is, the advertisement may be posted around the image display devices 100 and 200 such as a bag or a poster.

  As described above, the image display devices 100 and 200 identify the product whose line of sight is stopped from the products displayed on the first menu screen 10 or the menu screen 30 and the products displayed around the display unit 112. To do. In addition, the image display devices 100 and 200 extract features based on the product whose line of sight is stopped. As a result, a suitable menu screen can be provided.

  In each of the above embodiments, the product displayed on the first menu screen 10 or the menu screen 30 is a predetermined product, but is not limited to this. For example, a product may be determined based on a product history purchased in the past, features extracted in the past, or the like.

  As described above, the image display apparatuses 100 and 200 specify the user by performing the image recognition process using the camera included in the line-of-sight detection unit 111. Note that the user can be specified by communicating with an application downloaded to a mobile terminal such as a smartphone possessed by the user. Further, the user can be specified by reading a card issued by a store such as a point card, for example.

  Further, the image display apparatuses 100 and 200 inquire and acquire a past product history of the specified user from a history DB (not shown), for example. In addition, the image display devices 100 and 200 extract features of products included in the acquired product history. Further, the image display devices 100 and 200 determine products to be displayed on the first menu screen 10 or the menu screen 30 based on the extracted features. In addition, the image display devices 100 and 200 display the first menu screen 10 or the menu screen 30 including an image related to the determined product. As a result, a suitable menu screen can be provided.

  Further, the image display devices 100 and 200 acquire the feature history used for determining the product when displaying the product for the specified user, and the first menu screen 10 or the menu screen based on the acquired feature history. A product to be displayed at 30 is determined. As a result, a suitable menu screen can be provided.

  In the second embodiment, the extracting unit 242 extracts the feature of the product specified by the specifying unit 241 and the feature based on the line of sight with respect to the detailed screen 40, and the product determining unit 243 determines the menu screen 50 based on the feature. However, the display content is not limited to this. For example, the extraction unit 242 may extract features based on the line of sight with respect to the detail screen 40. In this case, the image display device 200 may omit the function of the specifying unit 241.

  In each of the above embodiments, the first menu screen 10 or the menu screen 30 is a screen that is displayed first when the user stands in front of the image display devices 100 and 200, but is not limited thereto. For example, it may be a menu screen displayed in response to a transition operation from the user. Further, in this case, the image display devices 100 and 200 are configured so that the first menu screen 10 or the menu screen 30 is based on the user's line of sight with respect to the menu screen displayed before the transition to the first menu screen 10 or the menu screen 30. The display content may be determined.

  As described above, the image display apparatuses 100 and 200 accept an instruction to switch from the predetermined menu screen to the first menu screen 10 or the menu screen 30. Further, the image display devices 100 and 200 determine the display content of the first menu screen 10 or the menu screen 30 based on the user's line of sight with respect to a predetermined menu screen. In addition, 100 and 200 are switched from a predetermined menu screen to the first menu screen 10 or the menu screen 30 for which display contents are determined. As a result, a suitable menu screen can be provided.

  In addition, the image display devices 100 and 200 determine the display content of the second menu screen 20 or the menu screen 50 based on a predetermined menu screen and the user's line of sight with respect to the first menu screen 10 or the menu screen 30. As a result, a suitable menu screen can be provided.

  In the above description, an example of displaying an image of a product on each menu screen has been described. However, the content to be displayed need not be limited to the product. For example, a list of documents such as books and articles may be displayed on each menu screen. In addition, each component of each part illustrated does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each unit is not limited to the one shown in the figure, and all or a part thereof may be functionally or physically distributed / integrated in arbitrary units according to various loads or usage conditions. Can be configured. For example, the specifying unit 141 and the extracting unit 142 may be integrated. Further, the illustrated processes are not limited to the above-described order, and may be performed at the same time as long as the processing contents are not contradictory, or may be performed by changing the order.

  Furthermore, various processing functions performed by each device may be executed entirely or arbitrarily on a CPU (or a microcomputer such as an MPU or MCU (Micro Controller Unit)). In addition, various processing functions may be executed in whole or in any part on a program that is analyzed and executed by a CPU (or a microcomputer such as an MPU or MCU) or on hardware based on wired logic. Needless to say, it is good.

  By the way, the various processes described in the above embodiments can be realized by executing a program prepared in advance by a computer. Therefore, in the following, an example of a computer that executes a program having the same function as in the above embodiment will be described. FIG. 13 is a diagram illustrating an example of a computer that executes a display control program.

  As illustrated in FIG. 13, the computer 300 includes a CPU 301 that executes various arithmetic processes, an input device 302 that receives data input, and a monitor 303. The computer 300 also includes a medium reading device 304 that reads a program and the like from a storage medium, an interface device 305 for connecting to various devices, and a communication device 306 for connecting to other information processing devices and the like by wire or wirelessly. Have The computer 300 also includes a RAM 307 that temporarily stores various types of information and a hard disk device 308. Each device 301 to 308 is connected to a bus 309.

  The hard disk device 308 includes a display control program having the same functions as the processing units of the line-of-sight detection unit 111, the determination unit 140 or the determination unit 240, and the switching unit 150 or the switching unit 250 illustrated in FIG. Remembered. The hard disk device 308 stores various data for realizing the product DB 120 and the display control program. For example, the input device 302 receives input of various information such as operation information from a user of the computer 300. The monitor 303 displays various screens such as an output screen for the user of the computer 300, for example. The medium reader 304 reads various data from the storage medium. The interface device 305 is connected to, for example, a printing device. The communication device 306 is connected to, for example, a network (not shown) and exchanges various information with other information processing devices.

  The CPU 301 reads out each program stored in the hard disk device 308, develops it in the RAM 307, and executes it to perform various processes. In addition, these programs can cause the computer 300 to function as the line-of-sight detection unit 111, the determination unit 140 or the determination unit 240, and the switching unit 150 or the switching unit 250 illustrated in FIG.

  Note that the display control program is not necessarily stored in the hard disk device 308. For example, the computer 300 may read and execute a program stored in a storage medium readable by the computer 300. The storage medium readable by the computer 300 corresponds to, for example, a portable recording medium such as a CD-ROM, a DVD disk, or a USB memory, a semiconductor memory such as a flash memory, a hard disk drive, and the like. Alternatively, the display control program may be stored in a device connected to a public line, the Internet, a LAN, or the like, and the computer 300 may read and execute the display control program from these.

  As described above, the following supplementary notes are further disclosed regarding the embodiment including the present example.

(Additional remark 1) The reception part which receives operation from a user with respect to the 1st menu screen containing the image regarding goods displayed on the display part,
A determination unit that determines display contents of the second menu screen based on a user's line of sight and operation with respect to the first menu screen;
An image display device comprising: a switching unit that switches the display content of the display unit to the second menu screen on which the display content is determined.

(Supplementary Note 2)
A specifying unit for specifying a product in which the line of sight stops and a product selection operation from the user is not accepted;
An extraction unit for extracting features of the product specified by the specification unit;
A product determination unit that determines a product based on the feature extracted by the extraction unit;
The switching unit is
Switching to the second menu screen including an image related to the product determined by the product determination unit;
The image display device according to appendix 1, wherein

(Additional remark 3) The said extraction part extracts the characteristic of the said goods based on the user's eyes | visual_axis with respect to the detailed screen containing the detailed information of the said goods,
The image display device as set forth in appendix 2, wherein:

(Supplementary Note 4) The product determination unit determines a product that does not include the feature extracted by the extraction unit.
The image display device according to appendix 2 or 3, characterized by the above.

(Supplementary Note 5) The product determination unit determines the product based on at least one of a stop position, a stop time, and a locus of the line of sight.
The image display device according to any one of appendices 2 to 4, characterized in that:

(Appendix 6) Accepting an operation from the user for the first menu screen including an image related to the product displayed on the display unit,
Based on the user's line of sight and operation on the first menu screen, the display content of the second menu screen is determined,
Switching the display content of the display unit to the second menu screen on which the display content is determined;
A display control method, wherein a computer executes processing.

(Additional remark 7) The said line of sight stops and specifies the goods which did not receive goods selection operation from the said user,
Extracting the characteristics of the identified product,
Based on the extracted features, determine the product,
Switch to the second menu screen including an image related to the determined product.
The display control method according to appendix 6, wherein:

(Supplementary Note 8) Based on a user's line of sight with respect to a detailed screen including detailed information of the product, the feature of the product is extracted.
The display control method according to appendix 7, characterized by:

(Supplementary Note 9) Determine a product that does not include the extracted features.
The display control method according to appendix 7 or 8, characterized by the above.

(Supplementary Note 10) The product is determined based on at least one of the stop position, stop time, and locus of the line of sight.
The display control method according to any one of appendices 7 to 9, characterized in that:

(Supplementary Note 11) Accepting an operation from the user for the first menu screen displayed on the display unit and including an image relating to a product,
Based on the user's line of sight and operation on the first menu screen, the display content of the second menu screen is determined,
Switching the display content of the display unit to the second menu screen on which the display content is determined;
A display control program for causing a computer to execute processing.

(Additional remark 12) The said line of sight stops and specifies the goods which did not receive goods selection operation from the said user,
Extracting the characteristics of the identified product,
Based on the extracted features, determine the product,
Switch to the second menu screen including an image related to the determined product.
The display control program according to appendix 11, which is characterized by the above.

(Additional remark 13) Based on the user's eyes | visual_axis with respect to the detailed screen containing the detailed information of the said goods, the characteristic of the said goods is extracted.
The display control program according to supplementary note 12, characterized by:

(Supplementary Note 14) Determine a product that does not include the extracted features.
14. The display control program according to appendix 12 or 13, characterized by the above.

(Supplementary Note 15) The product is determined based on at least one of the stop position, stop time, and locus of the line of sight.
The display control program according to any one of supplementary notes 12 to 14, characterized in that:

100, 200 Image display device 111 Gaze detection unit 112 Display unit 113 Reception unit 120 Product database 130, 230 Control unit 140, 240 Determination unit 141, 241 Identification unit 142, 242 Extraction unit 143, 243 Product determination unit 150, 250 Switching unit

Claims (7)

An accepting unit that accepts an operation from a user for a first menu screen that includes an image related to a product displayed on the display unit;
Said There first based on the user's line of sight and operation on the menu screen of the line of sight is stationary, to identify the not received item selection operation from the user item, a product that does not contain the characteristics of the products identified determined by the determination unit that determines a display content of the second menu screen,
An image display device comprising: a switching unit that switches the display content of the display unit to the second menu screen on which the display content is determined. The determination unit
A specifying unit for specifying a product in which the line of sight stops and a product selection operation from the user is not accepted;
An extraction unit for extracting features of the product specified by the specification unit;
A product determination unit that determines a product based on the feature extracted by the extraction unit;
The switching unit is
Switching to the second menu screen including an image related to the product determined by the product determination unit;
The image display apparatus according to claim 1. The extraction unit extracts features of the product based on a user's line of sight with respect to a screen including detailed information of the product.
The image display device according to claim 2. The product determination unit determines a product not including the feature extracted by the extraction unit;
The image display device according to claim 2, wherein the image display device is an image display device. The product determination unit determines the product based on at least one of a stop position, a stop time, and a locus of the line of sight;
The image display device according to claim 2, wherein the image display device is an image display device. Accepting an operation from the user for the first menu screen including an image related to the product displayed on the display unit,
Said There first based on the user's line of sight and operation on the menu screen of the line of sight is stationary, to identify the not received item selection operation from the user item, a product that does not contain the characteristics of the products identified To determine the display content of the second menu screen,
Switching the display content of the display unit to the second menu screen on which the display content is determined;
A display control method, wherein a computer executes processing. Accepting an operation from the user for the first menu screen including an image related to the product displayed on the display unit,
Said There first based on the user's line of sight and operation on the menu screen of the line of sight is stationary, to identify the not received item selection operation from the user item, a product that does not contain the characteristics of the products identified To determine the display content of the second menu screen,
Switching the display content of the display unit to the second menu screen on which the display content is determined;
A display control program for causing a computer to execute processing.