JP2020115298A - Display controller and information display system and display control method and program - Google Patents

Abstract

To realize a user interface with excellent operability and convenience.SOLUTION: A control device 10 comprises a finger information acquisition unit 101 acquiring positional information indicating a position that a user's finger touches and fingerprint information indicating a fingerprint of user's finger from a touch panel 11, a display item determination unit 102 determining a display item, an image construction determination unit 103 determining an image construction that is a construction of images including the display item, a display control unit 104 displaying an image based on the image construction, a selection item specification unit 105 specifying the selection item that the user's finger has selected based on the positional information and the image construction, and a history storage unit 107 storing a selection history for each fingerprint based on the fingerprint information and the selection item. The display item determination unit 102 determines an item associated with the selection item as a display item when the selection item is specified, the image construction determination unit 103 determines an image construction based on the selection history corresponding to the fingerprint information.SELECTED DRAWING: Figure 1

Description

The present invention relates to a display control device, an information display system, a display control method and a program.

There is known a technique for realizing a user interface with excellent operability by using a touch panel display capable of not only displaying an image and detecting the position of a touched finger, but also detecting a fingerprint of the touched finger. Patent Document 1 discloses a mobile computer capable of associating a fingerprint detected by a touch panel display with an operation. According to the mobile computer, since the operation can be associated with each finger of the user, the user can perform various operations by switching the finger to be operated. Therefore, a user interface with excellent operability can be realized by the mobile computer.

JP, 2009-140390, A

The mobile computer disclosed in Patent Document 1 has a problem that the user needs to perform a registration work for associating a fingerprint with an operation. Therefore, the mobile computer of Patent Document 1 is not convenient enough.

In view of the above circumstances, an object of the present invention is to provide a display control device and the like that realizes a user interface with excellent operability and convenience.

In order to achieve the above object, the display control device according to the present invention displays an image and indicates the position touched by the user's finger from a touch panel display that detects the position touched by the finger and the fingerprint of the touched finger. Finger information acquisition means for acquiring position information and fingerprint information indicating the fingerprint of the user's finger, display item determination means for determining a display item that is an item to be displayed on the touch panel display, and an image including the display item. Of the display items based on the position information and the image configuration, an image configuration determining unit that determines an image configuration that is a configuration of, a display control unit that displays an image based on the image configuration on the touch panel display. A selection item specifying unit that specifies a selection item that is an item selected by the user's finger, and a selection history that is a history of items selected by the user, based on the fingerprint information and the selection item, for each fingerprint. And history storing means for storing the history. The display item determining means determines, when the selection item is identified by the selection item identifying means, an item associated with the selection item as the display item, and the image configuration determining means includes the fingerprint information. The image configuration is determined based on the selection history corresponding to.

According to the present invention, the selection history is stored for each fingerprint, and the image configuration that is the configuration of the image including the display item is determined based on the selection history corresponding to the acquired fingerprint information. Therefore, according to the present invention, the user does not need to perform the registration work for associating the fingerprint with the operation, which is excellent in convenience, and the image configuration based on the selection history is determined for each fingerprint, which is also excellent in operability. A user interface can be realized.

The figure which shows the terminal device which concerns on Embodiment 1 of this invention. The figure which shows an example of the change of the image displayed on the touch panel of the terminal device which concerns on Embodiment 1 of this invention. The figure which shows an example of the change of the image displayed on the touch panel of the terminal device which concerns on Embodiment 1 of this invention. The figure which shows an example of the change of the image displayed on the touch panel of the terminal device which concerns on Embodiment 1 of this invention. The figure which shows an example of the item corresponding data which concerns on Embodiment 1 of this invention. The figure which shows an example of the fingerprint table which concerns on Embodiment 1 of this invention. The figure which shows an example of the selection history table which concerns on Embodiment 1 of this invention. The figure which shows an example of the hardware constitutions of the control apparatus which concerns on Embodiment 1 of this invention. The flowchart which shows an example of operation|movement of the display control by the control apparatus which concerns on Embodiment 1 of this invention. 3 is a flowchart showing an example of an operation of saving a selection history by the control device according to the first embodiment of the present invention. The figure which shows the structure of the terminal device which concerns on Embodiment 2 of this invention. The figure which shows an example of the change of the image displayed on the touch panel of the terminal device which concerns on Embodiment 2 of this invention. The figure which shows an example of the selection history table which concerns on Embodiment 2 of this invention. Flowchart showing an example of the operation of display control by the control device according to the second embodiment of the present invention. Flowchart showing an example of operation of automatic selection by the control device according to Embodiment 2 of the present invention The figure which shows the structure of the remote control system which concerns on Embodiment 3 of this invention. The figure which shows an example of a change of the image displayed on the touch panel of the terminal device which concerns on the modification of embodiment of this invention.

Hereinafter, an embodiment in which an information display system according to the present invention is applied to a terminal device or the like will be described with reference to the drawings. In the drawings, the same or equivalent parts are designated by the same reference numerals.

(Embodiment 1)
The terminal device 1 according to the first embodiment will be described with reference to FIG. The terminal device 1 is a terminal device for remotely operating the air conditioner 2 via the network NT. The terminal device 1 is, for example, a mobile phone or a tablet terminal in which a program for remotely controlling the air conditioner 2 is installed. The network NT is, for example, the Internet.

The terminal device 1 includes a control device 10 that controls the touch panel 11 and communicates with the air conditioner 2, and a touch panel 11 that is a touch panel display that displays an image and detects a position touched by a finger and a fingerprint of the touched finger. The terminal device 1 provides the user with a user interface for remotely operating the air conditioner 2. As described below, the terminal device 1 can provide a user interface with excellent operability and convenience. The terminal device 1 is an example of the information display system according to the present invention.

The control device 10 acquires from the touch panel 11 finger information including position information indicating the contact position of the finger touching the touch panel 11 and fingerprint information indicating the fingerprint of the finger. The control device 10 controls the touch panel 11 to display an image on the touch panel 11. The control device 10 communicates with the air conditioner 2 via the Internet NT and controls the air conditioner 2. The functional configuration of the control device 10 will be described later. The control device 10 is an example of the display control device according to the present invention.

The touch panel 11 detects a position touched by a finger and a fingerprint of the finger. The touch panel 11 outputs finger information including position information indicating a position touched by the finger and fingerprint information indicating a fingerprint of the finger to the control device 10. The touch panel 11 displays an image under the control of the control device 10. The touch panel 11 is, for example, a capacitance type touch panel display capable of detecting a change in capacitance due to unevenness of a fingerprint. The touch panel 11 is an example of a touch panel display according to the present invention.

Next, in order to facilitate understanding, an example of changes in the image displayed on the touch panel 11 of the terminal device 1 will be described with reference to FIGS. 2 and 3. Items selectable by a user's touch operation are displayed in each image shown in FIGS. 2 and 3. 2 and 3 show that the image changes according to the item selected by the user's touch operation. 2 and 3, the hand icon indicates the position where the touch operation is performed, and the arrow indicates the change in the image due to the touch operation.

In the following description, it is assumed that the user has not operated the terminal device 1 before the operation shown in FIG. 2 is performed, and the operation shown in FIG. 3 is performed as the operation shown in FIG. Immediately after "Immediately" means that no other operation is performed after the operation shown in FIG. 2 is performed. Further, the operation shown in FIG. 3 is assumed to be performed by the same finger as that used in the operation shown in FIG.

FIG. 2 shows that the user selects items in the order of “air conditioner list”, “air conditioner Z”, “operation mode”, and “heating”, and the image changes each time the item is selected. When the item "heating" is selected, the operation mode of the air conditioner 2 is controlled to "heat", and the image showing the item "list of air conditioners" is displayed again.

Also in FIG. 3, the user selects an item in the order of “air conditioner list”, “air conditioner Z”, and “operation mode”, and finally an image including four items of “heating”, “cooling”, “dehumidification”, and “blowing” Is displayed. However, in FIG. 3, the display order of items is different from that in FIG. 2 except for the first image in which the “air conditioner list” is displayed. This is because the item selected by the most recent operation, that is, the operation shown in FIG. 2, is displayed at the top. By reflecting the item selected by the most recent operation in the display order, the user can easily select the most recently selected item.

When the user performs the operation shown in FIG. 2 and selects the item of “air conditioner list” with a finger different from the finger performing the operation shown in FIG. 2, the displayed image is 2 is the same as that shown in FIG. That is, when a finger different from the finger used in the operation shown in FIG. 2 is used, the items selected by the operation shown in FIG. 2 are not reflected in the display order. Therefore, it becomes easier for the user to select various items by using different fingers. For example, when the operation mode of the air conditioner Z is set to heating by the index finger of the right hand and the operation of setting the timer of the air conditioner X is performed by the thumb of the left hand, the operation mode of the air conditioner Z is set to heating. In either case of setting the X timer, selection of items becomes easy.

In the above description, the item selected most recently is reflected in the display order, but the number of selections of each item may be reflected in the display order. For example, the item with the most selection times may be displayed at the top and the item with the least selection times may be displayed at the bottom.

Further, instead of changing the display order, for example, the size of the area in which the items are displayed may be changed. For example, as shown in FIG. 4, the most recently selected item may be displayed large and the other items may be displayed small. Also, the number of selections of each item may be reflected in the size of the area instead of the item most recently selected.

In the following description, the touch operation is performed with the same finger unless otherwise specified. Also, unless otherwise specified, the item selected most recently is reflected in the display order of the item, and the size of the area in which the item is displayed does not change.

To summarize the above description, according to the terminal device 1, the item selected by the most recent user operation is reflected in the display order of the item in the next operation. The display order is reflected for each finger separately.

Next, the functional configuration of the control device 10 will be described with reference to FIG. 1 again. The control device 10 controls the control device 10 as a whole, a storage unit 110 that stores the item correspondence data 111, the fingerprint table 112, and the selection history table 113, and communication that communicates with the air conditioner 2 via the network NT. And a section 120.

The control unit 100 includes a finger information acquisition unit 101 that acquires finger information, a display item determination unit 102 that determines a display item that is an item to be displayed on the touch panel 11, and an image configuration that is an image configuration including the display item. An image configuration determination unit 103 for determining, a display control unit 104 for displaying an image based on the image configuration on the touch panel 11, and a selection item identification unit 105 for identifying a selection item that is an item selected by the user's finger among the display items. A fingerprint ID acquisition unit 106 that acquires a fingerprint ID corresponding to the fingerprint information, and a history storage unit 107 that stores a selection history that is a history of items selected by the user for each fingerprint.

The finger information acquisition unit 101 acquires from the touch panel 11 finger information including position information indicating the contact position of the finger touching the touch panel 11 and fingerprint information indicating the fingerprint of the finger. The position information is information indicating a contact position on the touch panel 11 represented by, for example, an XY coordinate system. The fingerprint information is, for example, information indicating a fingerprint image. The finger information acquisition unit 101 is an example of finger information acquisition means according to the present invention.

The display item determination unit 102 determines a display item that is an item to be displayed on the touch panel 11. When the initial screen is to be displayed on the touch panel 11, the display item determination unit 102 determines the initial display item which is the display item to be displayed on the initial screen. Explaining a series of changes in the image shown in FIG. 2 as an example, the display item determination unit 102 determines an item of “air conditioner list” as a display item. When the user selects an item and the selection item specifying unit 105, which will be described later, specifies the item as a selection item, the display item determining unit 102 refers to the item correspondence data 111 stored in the storage unit 110, and selects the selection item. The item associated with is determined as the display item. Explaining a series of changes in the image shown in FIG. 2 as an example, when the user selects the item “driving mode”, the item “driving mode” is specified as the selection item by the selection item specifying unit 105, and “driving mode” is selected. Four items of "cooling", "dehumidifying", "heating", and "blowing" associated with the item of "mode" are determined as display items. The display item determining unit 102 is an example of a display item determining unit according to the present invention.

Details of the item correspondence data 111 will be described with reference to FIG. Each item is associated with the item correspondence data 111 expressed in a tree-structured data format as shown in FIG. 5, for example. The item of "List of air conditioners" located on the route is the initial display item. The arrow in FIG. 5 indicates that when the item at the base of the arrow becomes the selection item, the item at the tip of the arrow becomes the display item. Therefore, according to FIG. 5, for example, the item of “operation mode” is associated with four items of “cooling”, “dehumidification”, “heating”, and “blowing”, and “operation mode” is a selection item. When specified, four items of "cooling", "dehumidification", "heating" and "blowing" will be determined as display items.

In FIG. 5, there are no further associated items with the terminal items (hereinafter referred to as terminal items) of the tree structure such as “cooling”, “dehumidifying”, “heating”, and “blowing”. Although details will be described later, when the terminal item is specified as the selection item, the control unit 100 controls the air conditioner 2 in accordance with the selection item, and the display item determination unit 102 sets the items of the “air conditioner list” as the initial display items. To decide.

Referring back to FIG. The image configuration determination unit 103 refers to the selection history table 113 stored in the storage unit 110, and determines the image configuration, which is the configuration of the image including the display item, based on the selection history corresponding to the fingerprint information. Details of the selection history will be described later. The image configuration determination unit 103 can specify the most recently selected item among the display items, the number of selections of each display item, and the like based on the selection history. Therefore, the image configuration determination unit 103 can determine the image configuration based on the item most recently selected, the number of selections, and the like. The image configuration determines in which area in the image the display item should be displayed. By defining the area in which the display item is to be displayed by the image configuration, for example, the display in which the user's latest operation is reflected, as shown in FIG. 3 or 4, is realized. The image configuration determination unit 103 is an example of the image configuration determination means according to the present invention.

Regarding the reference of the selection history table 113, in more detail, the image configuration determination unit 103 acquires the fingerprint ID corresponding to the fingerprint information by the fingerprint ID acquisition unit 106, and based on the selection history corresponding to the fingerprint ID. The image structure will be determined. Details of the fingerprint ID and fingerprint ID acquisition unit 106 will be described later.

The display control unit 104 creates an image based on the image configuration determined by the image configuration determination unit 103, and displays the image on the touch panel 11. The display control unit 104 is an example of display control means according to the present invention.

The selection item specifying unit 105 selects one of the display items displayed on the touch panel 11 based on the position information included in the finger information acquired by the finger information acquiring unit 101 and the image configuration determined by the image configuration determining unit 103. The selection item which is the item selected by the user's finger is specified. The selection item specifying unit 105 can determine whether or not the contact position of the user's finger touching the touch panel 11 is included in the area in which the display item is displayed, by collating the position information with the image configuration. Therefore, the selection item specifying unit 105 can specify the display item selected by the user's finger. The selection item specifying unit 105 is an example of selection item specifying means according to the present invention.

In addition, when the area of the display item displayed on the touch panel 11 is changed by an operation other than the selection of the display item such as the zoom by the pinch operation and the scroll by the flick operation, the selection item identifying unit 105 causes It is preferable to specify the selection item in consideration of the change in the image.

The fingerprint ID acquisition unit 106 refers to the fingerprint table 112 stored in the storage unit 110 and acquires the fingerprint ID corresponding to the fingerprint information acquired by the finger information acquisition unit 101. Specifically, when the fingerprint data corresponding to the fingerprint information exists in the fingerprint table 112, the fingerprint ID associated with the fingerprint data is acquired. If the fingerprint data corresponding to the fingerprint information does not exist in the fingerprint table 112, the fingerprint data is created based on the fingerprint information, the created fingerprint data is associated with the new fingerprint ID, and is saved in the fingerprint table 112. The fingerprint ID is acquired as the fingerprint ID. “Fingerprint data corresponding to fingerprint information” refers to fingerprint data indicating the same fingerprint as the fingerprint indicated by the fingerprint information. The “fingerprint ID corresponding to fingerprint information” refers to a fingerprint ID associated with the same fingerprint as the fingerprint indicated by the fingerprint information.

The fingerprint data and fingerprint ID will be described in detail with reference to an example of the fingerprint table 112 shown in FIG. The fingerprint table 112 stores the fingerprint ID and the fingerprint data in association with each other. Fingerprint data is data about a fingerprint created based on fingerprint information. In the example of FIG. 6, the data of the fingerprint image indicated by the fingerprint information is the fingerprint data, but the feature amount obtained based on the fingerprint information may be the fingerprint data, for example. The fingerprint ID is an identifier associated with the fingerprint data in order to uniquely identify the fingerprint data. By associating the fingerprint ID with the fingerprint data, the fingerprint ID and the fingerprint are associated with each other.

In the example of FIG. 6, the fingerprint ID acquisition unit 106 collates the fingerprint image represented by the fingerprint information acquired by the finger information acquisition unit 101 with the fingerprint image represented by the fingerprint data stored in the fingerprint table 112 to obtain the fingerprint information. It is determined whether or not there is fingerprint data showing the same fingerprint as the fingerprint shown by. When the fingerprint data indicating the same fingerprint as the fingerprint indicated by the fingerprint information exists, the fingerprint ID corresponding to the fingerprint data is acquired. When there is no fingerprint data showing the same fingerprint as the fingerprint indicated by the fingerprint information, fingerprint data is created based on the fingerprint image indicated by the fingerprint information, and the fingerprint data is stored in the fingerprint table 112 in association with the new fingerprint ID. ..

The history storage unit 107 associates the fingerprint ID acquired by the fingerprint ID acquisition unit 106 with the selection item identified by the selection item identification unit 105, and adds the fingerprint ID to the selection history table 113 stored in the storage unit 110. By adding the fingerprint ID and the selected item to the selection history table 113 in association with each other, the history storage unit 107 can store the selection history, which is the history of the item selected by the user, for each fingerprint. The history storage unit 107 is an example of history storage means according to the present invention.

The selection history will be described in detail with reference to an example of the selection history table 113 shown in FIG. 7. The selection history table 113 stores the items selected by the user for each fingerprint ID in time series. The example of FIG. 7 shows an example of the selection history table 113 immediately after the operation shown in FIG. 2 is performed, and the user whose fingerprint ID is aaa is "list of air conditioners", "air conditioner Z", " It indicates that the items are selected in the order of "operation mode" and "heating". With such a selection history table 113, the most recently selected item among the display items, the number of selections of each display item, and the like can be specified for each fingerprint.

In addition to the function of each of the functional units described above, when the selection item specified by the selection item specifying unit 105 is a terminal item, the control unit 100 controls the air conditioner 2 via the communication unit 120 based on the terminal item. Have a function.

The functional configuration of the control device 10 has been described above. Next, an example of the hardware configuration of the control device 10 will be described with reference to FIG. The control device 10 shown in FIG. 8 is realized by a computer such as a personal computer or a microcontroller.

The control device 10 includes a processor 1001, a memory 1002, an interface 1003, and a secondary storage device 1004, which are connected to each other via a bus 1000.

The processor 1001 is, for example, a CPU (Central Processing Unit). Each function of the control device 10 is realized by the processor 1001 reading the operation program stored in the secondary storage device 1004 into the memory 1002 and executing it.

The memory 1002 is a main storage device including, for example, a RAM (Random Access Memory). The memory 1002 stores the operation program read by the processor 1001 from the secondary storage device 1004. The memory 1002 also functions as a work memory when the processor 1001 executes the operation program.

The interface 1003 is an I/O (Input/Output) interface such as a serial port, a USB (Universal Serial Bus) port, and a network interface. The function of the communication unit 120 is realized by the interface 1003. Further, by connecting the touch panel 11 to the interface 1003, the control device 10 can control the touch panel 11.

The secondary storage device 1004 is, for example, a flash memory, a HDD (Hard Disk Drive), or an SSD (Solid State Drive). The secondary storage device 1004 stores an operation program executed by the processor 1001. The function of the storage unit 110 is realized by the secondary storage device 1004.

Next, an example of the display control operation by the control device 10 will be described with reference to FIG. 9. Further, a specific example will be described with reference to FIGS. 3, 5, and 7 as appropriate. In the specific example, the item correspondence data 111 is shown in FIG. 5, the selection history table 113 is shown in FIG. 7, the fingerprint ID corresponding to the fingerprint of the finger used for the operation is aaa, and the latest The selected item shall be displayed at the top. The operation shown in FIG. 9 is started, for example, when the user activates the remote control program installed in the terminal device 1.

In the control unit 100 of the control device 10, the display item determination unit 102 determines the initial display item, and the image configuration determination unit 103 determines the image configuration including the initial display item (step S101). For example, when the item correspondence data 111 is as shown in FIG. 5, the item “air conditioner list” is determined as the initial display item.

The display control unit 104 of the control unit 100 creates an image based on the image configuration determined in step S101 and displays the image on the touch panel 11 (step S102). For example, in the example shown in FIG. 3, when the display item is “air conditioner list”, an image including the “air conditioner list” item is displayed on the touch panel 11. Similarly, when the display items are "air conditioner X", "air conditioner Y", and "air conditioner Z", an image including the items of "air conditioner X", "air conditioner Y", and "air conditioner Z" is displayed on the touch panel 11. ..

When a finger touches the touch panel 11 by a user's operation, the finger information acquisition unit 101 of the control unit 100 outputs the finger information including the position information indicating the position touched by the finger and the fingerprint information indicating the fingerprint of the finger. (Step S103).

The selection item specifying unit 105 of the control unit 100 determines whether the user operation is an item selection operation based on the position information included in the finger information acquired in step S103 and the image configuration determined in step S101. It is determined (step S104).

When the user operation is not the item selection operation (step S104: No), the control unit 100 repeats the operation flow from step S102. Note that the process returns to step S102 because it is assumed that the user operation is an operation that changes the image displayed on the touch panel 11, such as zooming by pinch operation and scrolling by flick operation.

When the user operation is an item selection operation (step S104: Yes), the selection item specifying unit 105 specifies the selection item which is the display item selected by the user based on the position information and the image configuration ( Step S105). For example, in the example shown in FIG. 3, when the item "air conditioner list" is touched by the user's finger, the item "air conditioner list" is specified as the selection item.

The control unit 100 stores the selection history by the fingerprint ID acquisition unit 106 and the history storage unit 107 (step S106). Details of the operation of saving the selection history will be described later.

The control unit 100 determines whether or not the selection item specified in step S105 is the end item (step S107).

When the selection item is the end item (step S107: Yes), the control unit 100 controls the air conditioner 2 based on the selection item (step S108), and repeats the operation flow from step S101. For example, in the case where the item corresponding data 111 is as shown in FIG. 5, when the item “heating” is specified as a selection item, the item “heating” is an end item, so that the operation of the air conditioner 2 Control for changing the operation mode to “heating” is performed, and thereafter, an image including the item of “air conditioner list” which is an initial display item is displayed on the touch panel 11.

When the selected item is not the end item (step S107: No), the display item determination unit 102 determines the item associated with the selected item as the display item (step S109). For example, in the case where the item correspondence data 111 is as shown in FIG. 5, when the selected item is the item of “operation mode”, “cooling”, “dehumidification”, “corresponding to the item of “operation mode”, Four items of "heating" and "blowing" are determined as display items.

The image configuration determination unit 103 refers to the selection history table 113 stored in the storage unit 110, and corresponds to the display item determined in step S109 and the fingerprint information included in the finger information acquired in step S103. The image configuration is determined based on the selection history (step S110). For example, in the case where the selection history table 113 is as shown in FIG. 7, a case where the display items are the items of “air conditioner X”, “air conditioner Y”, and “air conditioner Z” will be described. Since it is specified from the selection history table 113 that the most recently selected item among these items is the item of “air conditioner Z”, the item of “air conditioner Z” as shown in FIG. The image composition for creating the displayed image is determined.

After that, the control unit 100 repeats the operation flow from step S102.

Next, an example of the operation of saving the selection history shown in step S106 of FIG. 9 will be described in detail with reference to FIG.

The fingerprint ID acquisition unit 106 determines whether fingerprint data corresponding to the fingerprint information acquired by the finger information acquisition unit 101 exists in the fingerprint table 112 stored in the storage unit 110 (step S1061).

When the fingerprint data corresponding to the fingerprint information exists (step S1061: Yes), the fingerprint ID acquisition unit 106 acquires the fingerprint ID corresponding to the fingerprint information as the fingerprint ID corresponding to the fingerprint information (step S1062). ..

When the fingerprint data corresponding to the fingerprint information does not exist (step S1061: No), the fingerprint ID acquisition unit 106 creates fingerprint data based on the fingerprint information, associates the created fingerprint data with a new fingerprint ID, and prints the fingerprint table. It is added to 112 and the new fingerprint ID is acquired as the fingerprint ID (step S1063).

The history storage unit 107 adds the fingerprint ID acquired in step S1062 or step S1063 and the selection item specified by the selection item specifying unit 105 to the selection history table 113 in association with each other (step S1064). After that, the control unit 100 ends the operation of saving the selection history and executes the operation of step S107 and subsequent steps in FIG.

The terminal device 1 according to the first embodiment has been described above. According to the terminal device 1, the selection history is stored for each fingerprint, and the image configuration, which is the configuration of the image including the display item, is determined based on the selection history corresponding to the acquired fingerprint information. Therefore, according to the terminal device 1, the user does not need to perform the registration work for associating the fingerprint and the operation with each other, which is excellent in convenience, and the image configuration based on the selection history is determined for each fingerprint, and thus the operability is improved. An excellent user interface can be realized.

For example, as in the example shown in FIG. 3, by reflecting the item selected by the most recent operation in the display order, the user can easily select the most recently selected item without requiring registration work. Further, in this case, the user can easily select various items by using different fingers. Further, even when the terminal device 1 is used by a plurality of users, the item selected by the most recent operation is reflected in the display order for each finger of each user without requiring registration work for each user. Therefore, each user can easily select the item most recently selected.

Moreover, according to the terminal device 1, a complicated operation can be simplified. For example, according to the terminal device 1, an operation in which the most recently selected item among a large number of display items is determined and selected a plurality of times is repeated, and the items displayed at the same position are continuously selected. It can be a simple operation. In particular, when the display items do not fit within one screen and it becomes necessary to select display items that are not displayed due to scrolling, flicking, or the like, the operation becomes much easier.

Moreover, according to the terminal device 1, the operation can be simplified without the user having to remember a complicated command. For example, the user can simplify the operation without learning anything by using only the finger that is the easiest to operate. In addition, for example, when it is desired to simplify the operation of the air conditioner X and the operation of the air conditioner Y by using different fingers, the user can remember only the finger used to operate the air conditioner X and the finger used to operate the air conditioner Y. Good.

(Embodiment 2)
The terminal device 1A according to the second embodiment will be described with reference to FIG. In addition to the case of the first embodiment, the second embodiment further realizes an automatic item selection operation. Hereinafter, description of points common to the first embodiment will be omitted.

Regarding the terminal device 1A, the control device 10A is different from the control device 10 of the first embodiment. Regarding the control device 10A, the control unit 100A is different from the control unit 100 of the first embodiment. The control unit 100A is different from that of the first embodiment in that it further includes an automatic selection unit 108A that extracts an item that satisfies a predetermined automatic selection condition and identifies it as a new selection item. Details of the automatic selection unit 108A will be described later. The terminal device 1A is an example of the information display system according to the present invention. The control device 10A is an example of the display control device according to the present invention.

In order to facilitate understanding, an example of changes in the image displayed on the touch panel 11 of the terminal device 1A will be described with reference to FIG. In this description, before the operation shown in FIG. 12 is performed, the operation of sequentially selecting each item of “air conditioner list”, “air conditioner Z”, “operation mode”, and “heating” is most recently performed with the same finger. It is assumed that 3 sets are continuously repeated. In this case, the selection history table 113 is as shown in FIG. 13, for example.

After the above operation is repeated 3 sets, if the user again selects the item of "air conditioner list" with the same finger, as shown in FIG. 12, the item to be displayed next to "air conditioner list" and the next item. The image including the end items of “heating”, “cooling”, “dehumidification”, and “blowing” is displayed on the touch panel 11 without displaying the items to be displayed in. This is because the item of "air conditioner Z" and the item of "operation mode" were automatically selected after selecting the item of "air conditioner list". For example, such an operation can be realized by predetermining, as the automatic selection condition, that the same item, which is not the end item and is the same as the selected item, is selected three times in a row.

With such an operation, for example, the user can simplify the operation when setting the operation mode of the air conditioner Z to heating by always using a specific finger when setting the operation mode of the air conditioner Z to heating.

The operation to select the items "Air conditioner list" and "Air conditioner Z" in that order has been performed three times in the last, but for the items after "Air conditioner Z", the same items are continuously selected. If not, the selection of "air conditioner Z" is automatically performed.

Next, the automatic selection unit 108A will be described with reference to FIG. 11 again. The automatic selection unit 108A extracts an item associated with the selection item specified by the selection item specifying unit 105. The automatic selection unit 108A refers to the selection history corresponding to the fingerprint information acquired by the finger information acquisition unit 101, which is included in the selection history table 113, and an item satisfying a predetermined automatic selection condition among the extracted items. Is extracted and the extracted item is specified as a new selection item. The automatic selection unit 108A also performs the same processing on the selection item specified by the automatic selection unit 108A itself. In other words, the automatic selection unit 108A extracts, from the items associated with the selected items, items satisfying a predetermined automatic selection condition based on the fingerprint information and the selection history, and newly extracts the extracted items. It is specified as a selection item. The automatic selection unit 108A is an example of an automatic selection unit according to the present invention.

Next, an example of a display control operation and an example of an automatic selection operation by the control device 10A will be described with reference to FIGS. 14 and 15. The operation shown in FIG. 14 is different from that of the first embodiment shown in FIG. 9 in that the operation of automatic selection shown in step S200 is executed between step S106 and step S107. An example of the automatic selection operation will be described below with reference to FIG.

The automatic selection unit 108A of the control unit 100A of the control device 10A determines whether or not the selection item is the end item (step S2001).

When the selection item is the end item (step S2001: Yes), the control unit 100A ends the operation of the automatic selection and executes the operation of step S107 and subsequent steps shown in FIG.

When the selected item is not the end item (step S2001: No), the automatic selection unit 108A refers to the item correspondence data 111 and extracts the item associated with the selected item (step S2002).

The automatic selection unit 108A refers to the selection history corresponding to the fingerprint information, and determines whether or not each item extracted in step S2002 satisfies a predetermined automatic selection condition (step S2003).

The automatic selection unit 108A determines whether or not there is an item that satisfies the automatic selection condition based on the result of the determination in step S2003 (step S2004).

When there is no item that satisfies the automatic selection condition (step S2004: No), the control unit 100A ends the automatic selection operation, and executes the operation after step S107 shown in FIG.
When there is an item that satisfies the automatic selection condition (step S2004: Yes), the automatic selection unit 108A identifies the item as a new selection item, and repeats the operation flow from step S2001.

The terminal device 1A according to the second embodiment has been described above. According to the terminal device 1A, an item satisfying the automatic selection condition is specified as a new selection item based on the fingerprint information and the selection history. Therefore, according to the terminal device 1A, the item selection operation can be automated, so that the user operation can be simplified and the operability and convenience are excellent.

(Embodiment 3)
A remote control system 1B according to the third embodiment will be described with reference to FIG. Among the configurations shown in FIG. 16, configurations having the same functions as those in the first embodiment are designated by the same names and the same reference numerals, and the description thereof will be omitted. The remote operation system 1B is a system for remotely operating the air conditioner 2 via the cloud server 4B by operating the terminal device 3B. The remote control system 1B is an example of the information display system according to the present invention.

The remote control system 1B is characterized in that each functional unit of the control unit 100 of the control device 10 according to the first embodiment is provided separately in the terminal device 3B and the cloud server 4B.

The terminal device 3B includes a touch panel 11, a control unit 300B, and a communication unit 310B.
The communication unit 310B communicates with the cloud server 4B via the network NT. The control unit 300B includes a finger information acquisition unit 101 and a display control unit 104.

The cloud server 4B includes a control unit 400B, a storage unit 110, and a communication unit 410B. The communication unit 410B communicates with the terminal device 3B and the air conditioner 2 via the network NT. The control unit 400B includes a display item determination unit 102, an image configuration determination unit 103, a selection item identification unit 105, a fingerprint ID acquisition unit 106, and a history storage unit 107.

The control unit 300B and the control unit 400B send and receive information required by the control unit 300B and the control unit 400B to and from each other via the communication unit 310B and the communication unit 410B. By transmitting and receiving each other, the control unit 300B and the control unit 400B can cooperate with each other to realize the same function as that of the control unit 100 of the first embodiment.

With the above configuration, remote operation system 1B can realize the same operation as terminal device 1 of the first embodiment, and can obtain the same effect as terminal device 1 of the first embodiment.

(Modification)
In the above description of the embodiment, the user interface for selecting an item by touch operation is exemplified, but a drum roll type user interface as shown in FIG. 17 may be used. In FIG. 17, a hand icon with an arrow indicates a flick operation in the arrow direction, not a touch operation. FIG. 17 shows that when the set temperature is selected, the item “22.0° C.” is displayed in the center in the initial state. The user can scroll the item downward by flicking the item downward. FIG. 17 shows that after the item “23.5° C.” is displayed in the center by the flick operation, the item “23.5° C.” is selected by the touch operation. Then, FIG. 17 shows that when the set temperature is set again, the most recently selected item “23.5° C.” is displayed in the center from the beginning.

In order to realize such an operation, the image configuration determination unit 103 determines the image configuration in consideration of the change due to the user's operation. For example, such an operation can be realized by including the information about the item which is not displayed in the initial state but is first displayed by the flick operation in the image configuration.

In the above embodiment, the remote operation of the air conditioner 2 is illustrated, but the present invention is applicable to other than the remote operation of the air conditioner 2. For example, the present invention can be applied to a digital pedometer that does not have a communication function.

In the third embodiment, it has been described that each functional unit of the control unit 100 of the first embodiment is provided separately in the control unit 300B of the terminal device 3B and the control unit 400B of the cloud server 4B. Separation of each functional unit is not limited to the example shown in the third embodiment. For example, the control unit 300B instead of the control unit 400B may include the image configuration determining unit 103 and the selection item specifying unit 105. Further, for example, a storage unit is also provided in the terminal device 3B, the item correspondence data 111 is stored in the storage unit of the terminal device 3B instead of the storage unit 110 of the cloud server 4B, and the control unit 300B instead of the control unit 400B determines the display item. The unit 102 may be provided.

In the hardware configuration shown in FIG. 8, the control device 10 includes a secondary storage device 1004. However, the configuration is not limited to this, and the secondary storage device 1004 may be provided outside the control device 10 and the control device 10 and the secondary storage device 1004 may be connected via the interface 1003. In this form, a removable medium such as a USB flash drive or a memory card can also be used as the secondary storage device 1004.

Further, instead of the hardware configuration shown in FIG. 8, the control device 10 may be configured by a dedicated circuit using an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like. .. Further, in the hardware configuration shown in FIG. 8, a part of the functions of the control device 10 may be realized by a dedicated circuit connected to the interface 1003, for example.

DESCRIPTION OF SYMBOLS 1,1A terminal device, 1B remote control system, 2 air conditioner, 3B terminal device, 4B cloud server, 10,10A control device, 11 touch panel, 100,100A control unit, 101 finger information acquisition unit, 102 display item determination unit, 103 Image configuration determination unit, 104 display control unit, 105 selection item specification unit, 106 fingerprint ID acquisition unit, 107 history storage unit, 108A automatic selection unit, 110 storage unit, 111 item correspondence data, 112 fingerprint table, 113 selection history table, 120 communication unit, 300B control unit, 310B communication unit, 400B control unit, 410B communication unit, 1000 bus, 1001 processor, 1002 memory, 1003 interface, 1004 secondary storage device, NT network.

Claims (7)

From a touch panel display that displays an image and detects the position touched by the finger and the fingerprint of the touched finger, position information indicating the position touched by the user's finger and fingerprint information indicating the fingerprint of the user's finger are acquired. Finger information acquisition means,
Display item determining means for determining a display item that is an item to be displayed on the touch panel display;
An image configuration determining means for determining an image configuration which is a configuration of an image including the display item,
Display control means for displaying an image based on the image configuration on the touch panel display;
Selection item specifying means for specifying a selection item which is an item selected by the user's finger among the display items based on the position information and the image configuration;
History saving means for saving, for each fingerprint, a selection history, which is a history of items selected by the user, based on the fingerprint information and the selection items.
Equipped with
The display item determination means, when the selection item is identified by the selection item identification means, determines an item associated with the selection item as the display item,
The image configuration determining means determines the image configuration based on the selection history corresponding to the fingerprint information,
Display controller. The image configuration determination means determines an area in which the display item is displayed based on the selection history,
The display control device according to claim 1. The image configuration determination means determines the display order of the display items based on the selection history,
The display control device according to claim 1. An automatic selection unit that extracts an item satisfying a predetermined automatic selection condition from the items associated with the selection item based on the fingerprint information and the selection history, and specifies the extracted item as a new selection item. Further comprising,
The display control device according to claim 1. A touch panel display that displays an image and detects the position touched by the finger and the fingerprint of the touched finger,
Finger information acquisition means for acquiring position information indicating the position of the user's finger touching the touch panel display and fingerprint information indicating the fingerprint of the user's finger;
Display item determining means for determining a display item that is an item to be displayed on the touch panel display;
An image configuration determining means for determining an image configuration which is a configuration of an image including the display item,
Display control means for displaying an image based on the image configuration on the touch panel display;
Selection item specifying means for specifying a selection item which is an item selected by the user's finger among the display items based on the position information and the image configuration;
History saving means for saving, for each fingerprint, a selection history, which is a history of items selected by the user, based on the fingerprint information and the selection items.
Equipped with
The display item determination means, when the selection item is identified by the selection item identification means, determines an item associated with the selection item as the display item,
The image configuration determining means determines the image configuration based on the selection history corresponding to the fingerprint information,
Information display system. From a touch panel display that displays an image and detects the position touched by the finger and the fingerprint of the touched finger, position information indicating the position touched by the user's finger and fingerprint information indicating the fingerprint of the user's finger are acquired. Finger information acquisition step,
A display item determining step for determining a display item that is an item to be displayed on the touch panel display;
An image configuration determining step of determining an image configuration which is a configuration of an image including the display item,
A display control step of displaying an image based on the image configuration on the touch panel display;
A selection item specifying step of specifying a selection item which is an item selected by the user's finger among the display items based on the position information and the image configuration;
A history storage step of storing a selection history, which is a history of the item selected by the user, for each fingerprint based on the fingerprint information and the selection item,
Equipped with
In the display item determination step, when the selection item is identified in the selection item identification step, the item associated with the selection item is determined as the display item,
In the image configuration determining step, the image configuration is determined based on the selection history corresponding to the fingerprint information,
Display control method. Computer,
From a touch panel display that displays an image and detects the position touched by the finger and the fingerprint of the touched finger, position information indicating the position touched by the user's finger and fingerprint information indicating the fingerprint of the user's finger are acquired. Finger information acquisition means,
Display item determining means for determining a display item that is an item to be displayed on the touch panel display,
Image configuration determining means for determining an image configuration which is a configuration of an image including the display item,
Display control means for displaying an image based on the image configuration on the touch panel display;
Selection item specifying means for specifying a selection item which is an item selected by the user's finger among the display items based on the position information and the image configuration;
History saving means for saving, for each fingerprint, a selection history, which is a history of items selected by the user, based on the fingerprint information and the selection items,
Function as
The display item determination means, when the selection item is identified by the selection item identification means, determines an item associated with the selection item as the display item,
The image configuration determining means determines the image configuration based on the selection history corresponding to the fingerprint information,
program.

Images