WO2018029738A1 - Dispositif de commande d'interface utilisateur et procédé de commande d'interface utilisateur - Google Patents

Dispositif de commande d'interface utilisateur et procédé de commande d'interface utilisateur Download PDF

Info

Publication number
WO2018029738A1
WO2018029738A1 PCT/JP2016/073250 JP2016073250W WO2018029738A1 WO 2018029738 A1 WO2018029738 A1 WO 2018029738A1 JP 2016073250 W JP2016073250 W JP 2016073250W WO 2018029738 A1 WO2018029738 A1 WO 2018029738A1
Authority
WO
WIPO (PCT)
Prior art keywords
screen
user interface
operation input
input device
display
Prior art date
Application number
PCT/JP2016/073250
Other languages
English (en)
Japanese (ja)
Inventor
彩花 田代
喬樹 八木澤
Original Assignee
三菱電機株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 三菱電機株式会社 filed Critical 三菱電機株式会社
Priority to JP2018526966A priority Critical patent/JP6377307B2/ja
Priority to PCT/JP2016/073250 priority patent/WO2018029738A1/fr
Publication of WO2018029738A1 publication Critical patent/WO2018029738A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output

Definitions

  • the present invention relates to control of a user interface (hereinafter referred to as “UI”), and more particularly to control of a UI (hereinafter referred to as “screen UI”) on a screen of a display device.
  • UI user interface
  • screen UI control of a UI
  • an operation input device of a television receiver includes a remote controller (so-called “full remote control”) capable of operating almost all the functions of the television receiver, and among the functions of the television receiver.
  • a remote controller (a so-called “simple remote controller”) capable of operating only a part of the functions, a camera for gesture input, a microphone for voice input, or the like is used.
  • a technique has been developed in which dedicated application software is installed in a portable information terminal such as a smartphone so that the portable information terminal functions as a remote controller (hereinafter referred to as “remote controller”).
  • operation input device for operation input when an operation input device used for operation input (hereinafter referred to as “operation input device for operation input”) is changed, a screen UI is displayed in accordance with the change.
  • a technique for switching has been developed (see, for example, Patent Document 1 or Patent Document 2).
  • Patent Document 1 discloses an in-vehicle navigation device having a touch panel, a remote controller, and a microphone for voice input.
  • the in-vehicle navigation device of Patent Document 1 switches the mode of the display screen according to the change when the operation input device for operation input is changed (see the summary of Patent Document 1, FIG. 1 and the like).
  • Patent Document 2 discloses an information processing apparatus having a remote controller, a mouse, and a keyboard.
  • the operation input device for operation input is changed, the information processing device of Patent Literature 2 switches a graphical user interface (Graphical User Interface, GUI) in accordance with the change (summary and diagram of Patent Literature 2). 1 etc.).
  • GUI graphical User Interface
  • the present invention has been made to solve the above-described problems.
  • the correspondence between the screen before switching and the screen after switching is visually determined. It is an object of the present invention to provide a user interface control device and a user interface control method that can be displayed in an easily understandable manner.
  • a user interface control device is a user interface control device for an electronic device that can be operated by a plurality of types of operation input devices, and the operation input device for operation input is changed from the first operation input device to the second operation input device.
  • the switching necessity determination unit and the switching necessity determination unit determine that the user interface is to be switched.
  • the intermediate screen generating unit for generating an intermediate screen between the first screen based on the first user interface and the second screen based on the second user interface, and an animation display using the intermediate screen, the screen of the display device is displayed. Switch from the first screen to the second screen Those comprising an over-user interface switching unit.
  • a user interface control method is a user interface control method for an electronic device that can be operated by a plurality of types of operation input devices.
  • the user interface control device and the user interface control method of the present invention when the screen UI is switched according to the change of the operation input device, an intermediate screen between the screen before switching and the screen after switching is generated, and the screen is displayed by animation display. Switch. As a result, the correspondence between the screen before switching and the screen after switching can be displayed in a visually easy-to-understand manner.
  • FIG. 1 It is a functional block diagram which shows the principal part of the electronic device which concerns on Embodiment 1 of this invention. It is a hardware block diagram which shows the principal part of the electronic device which concerns on Embodiment 1 of this invention. It is another hardware block diagram which shows the principal part of the electronic device which concerns on Embodiment 1 of this invention. It is a flowchart which shows operation
  • FIG. 7A is an explanatory diagram showing an example of the first screen according to Embodiment 1 of the present invention.
  • FIG. 7B is an explanatory diagram showing an example of an intermediate screen according to Embodiment 1 of the present invention.
  • FIG. 7C is an explanatory diagram showing an example of a second screen according to Embodiment 1 of the present invention.
  • It is a functional block diagram which shows the principal part of the other electronic device which concerns on Embodiment 1 of this invention.
  • It is explanatory drawing which shows an example of the 1st screen which concerns on Embodiment 2 of this invention.
  • 11A is an explanatory diagram showing an example of the first screen according to Embodiment 2 of the present invention.
  • 11B and 11C are explanatory diagrams illustrating an example of an intermediate screen according to Embodiment 2 of the present invention.
  • FIG. 11D is an explanatory diagram showing an example of a second screen according to Embodiment 2 of the present invention. It is a functional block diagram which shows the principal part of the other electronic device which concerns on Embodiment 2 of this invention. It is explanatory drawing which shows an example of the 1st screen which concerns on Embodiment 3 of this invention. It is explanatory drawing which shows an example of the 2nd screen which concerns on Embodiment 3 of this invention.
  • FIG. 15A is an explanatory diagram showing an example of a first screen according to Embodiment 3 of the present invention.
  • FIG. 15B is an explanatory diagram showing an example of an intermediate screen according to Embodiment 3 of the present invention.
  • FIG. 15C is an explanatory diagram showing an example of a second screen according to Embodiment 3 of the present invention. It is a functional block diagram which shows the principal part of the other electronic device which concerns on Embodiment 3 of this invention. It is a functional block diagram which shows the principal part of the other electronic device which concerns on Embodiment 3 of this invention.
  • FIG. 1 is a functional block diagram showing the main part of the electronic apparatus according to Embodiment 1 of the present invention.
  • FIG. 2 is a hardware configuration diagram showing a main part of the electronic device according to Embodiment 1 of the present invention.
  • FIG. 3 is another hardware configuration diagram showing the main part of the electronic apparatus according to Embodiment 1 of the present invention.
  • UI control apparatus a user interface control apparatus 100 according to the first embodiment will be described focusing on an example applied to an electronic apparatus 1 including a television receiver. .
  • the first display device 2 is configured by a display such as a liquid crystal display, an organic EL (Electro Luminescence) display, a plasma display, or a cathode ray tube display.
  • the first display device 2 may be provided integrally with the electronic device 1, or may be provided on a separate member from the electronic device 1.
  • the full remote control 3 has a plurality of physical keys including a direction key, and accepts an operation input by pressing the physical key.
  • the simple remote controller 4 has at least one physical key and accepts an operation input by pressing this physical key.
  • the gesture input device 5 has a camera for photographing an operator of the electronic device 1 (hereinafter simply referred to as “operator”).
  • the gesture input device 5 executes an image recognition process on an image captured by the camera, and receives an operation input by a gesture such as a hand sign or a hand movement.
  • the voice input device 6 has a microphone for acquiring voice uttered by the operator.
  • the voice input device 6 performs voice recognition processing on the voice acquired by the microphone and accepts voice operation input.
  • the portable information terminal 7 is configured by, for example, a smartphone or a tablet computer.
  • the portable information terminal 7 is installed with dedicated application software, and functions as a remote control for the television receiver.
  • the portable information terminal 7 includes the second display control unit 8, the second display device 9, and the touch panel 10.
  • the second display control unit 8 controls the screen of the second display device 9.
  • the second display device 9 is configured by a display such as a liquid crystal display or an organic EL display, and has a lower resolution than the first display device 2.
  • the touch panel 10 is configured integrally with the second display device 9.
  • the portable information terminal 7 receives the operation input to the touch panel 10 in a state where the second display control unit 8 displays a screen corresponding to the function of the remote control on the second display device 9, and thereby the function of the remote control is achieved. To fulfill.
  • the operation input device 11 is composed of the full remote control 3, the simple remote control 4, the gesture input device 5, the voice input device 6, and the portable information terminal 7. That is, the electronic device 1 shown in FIG. 1 has one each of five types of operation input devices.
  • a signal indicating the operation input device 11 (hereinafter referred to as an “operation start signal”) is transmitted to the electronic device by wireless communication or wired communication. 1 is transmitted.
  • Each operation input device 11 transmits a signal indicating the content of the operation (hereinafter referred to as an “operation content signal”) to the electronic apparatus 1 by wireless communication or wired communication when an operation to the operation input device 11 is input. To do.
  • the operation signal receiving unit 21 receives the operation start signal and the operation content signal transmitted by the operation input device 11.
  • the operation signal receiving unit 21 outputs the received operation start signal to the UI control device 100 and outputs the received operation content signal to the first display control unit 22.
  • the first display control unit 22 controls the screen of the first display device 2 according to the operation content indicated by the operation content signal input from the operation signal receiving unit 21.
  • the first display control unit 22 and the second display control unit 8 may be collectively referred to simply as “display control unit”.
  • the first display device 2 and the second display device 9 may be simply referred to as “display device”.
  • the user interface setting information storage unit (hereinafter referred to as “UI setting information storage unit”) 23 stores information (hereinafter referred to as “UI setting information”) used for setting the screen UI of the display devices 2 and 9. To do.
  • the UI setting information indicates, for example, the operation method of each operation input device 11, the limitation of the operation content by each operation input device 11, and the display devices 2 and 9 used for the operation of each operation input device 11. It is. These pieces of information are stored for each operation input device 11 in a table-like data structure, for example.
  • the operation method of the full remote control 3 uses a plurality of physical keys including direction keys.
  • the full remote controller 3 can perform detailed setting operations using a plurality of physical keys, and can perform detailed direction operations using direction keys.
  • the first display device 2 is used for the operation of the full remote control 3. Therefore, the UI setting information corresponding to the full remote control 3 indicates that the operation method uses a physical key, that there is no restriction on the operation content, and that the first display device 2 is used for the operation. It is.
  • the operation method of the gesture input device 5 is based on gesture input.
  • the gesture input device 5 can only perform a rough setting operation by hand sign, and can only perform a rough direction operation by hand movement.
  • the first display device 2 is used for the operation of the gesture input device 5. Therefore, the UI setting information corresponding to the gesture input device 5 is that the operation method is based on gesture input, the operation content is limited to a rough operation, and the first display device 2 is used for the operation. Is shown.
  • the operation method of the voice input device 6 is based on voice input.
  • the voice input device 6 cannot be used for a direction operation, and cannot be used for an operation that requires a high response speed.
  • the first display device 2 is used for the operation of the voice input device 6. Therefore, the UI setting information corresponding to the voice input device 6 cannot be used for the operation method by voice input, the direction operation and the operation requiring a high response speed, and the first display device 2. Is used for operation.
  • the operation method of the portable information terminal 7 uses the touch panel 10 that is configured integrally with the second display device 9.
  • the portable information terminal 7 can perform the same operation as the full remote control 3 using the second display device 9 and the touch panel 10.
  • the second display device 9 is used for the operation of the portable information terminal 7. Therefore, the UI setting information corresponding to the portable information terminal 7 indicates that the operation method is that using the touch panel 10, the operation content is not limited, and the second display device 9 is used for the operation. Is.
  • the UI setting information indicates the resolution of each of the display devices 2 and 9 in addition to the above information corresponding to each operation input device 11.
  • the UI setting information may be stored in advance when the electronic device 1 is manufactured or shipped, or may be received from the operation input device 11 after the electronic device 1 is shipped.
  • the UI control device 100 determines the operation input device 11 for operation input using the operation start signal input from the operation signal receiving unit 21.
  • the UI control device 100 acquires UI setting information corresponding to the operation input device 11 for operation input and UI setting information indicating the resolution of the display devices 2 and 9 from the UI setting information storage unit 23. is there.
  • the UI control device 100 sets the screen UI of the display devices 2 and 9 using the acquired UI setting information.
  • the UI control device 100 sets the screen UI of the first display device 2 when the operation input device 11 for operation input is the full remote control 3, the simple remote control 4, the gesture input device 5, or the voice input device 6.
  • This screen UI includes the operation method of the full remote control 3, the simple remote control 4, the gesture input device 5 or the voice input device 6, the restriction of the operation contents by the full remote control 3, the simple remote control 4, the gesture input device 5 or the voice input device 6, and It is a screen UI corresponding to the resolution of the first display device 2 and the like.
  • the UI control device 100 sets the screen UI of the second display device 9 when the operation input device 11 for operation input is the portable information terminal 7.
  • This screen UI is a screen UI corresponding to the operation method of the portable information terminal 7, the limitation of the operation content by the portable information terminal 7, the resolution of the second display device 9, and the like.
  • the first display control unit 22 controls the screen of the first display device 2 based on the screen UI set by the UI control device 100.
  • the second display control unit 8 controls the screen of the second display device 9 based on the screen UI set by the UI control device 100.
  • the UI control apparatus 100 includes a change presence / absence determination unit 24, a switching necessity determination unit 25, an intermediate screen generation unit 26, and a user interface switching unit (hereinafter referred to as “UI switching unit”) 27.
  • UI switching unit user interface switching unit
  • the change presence / absence determination unit 24 determines whether the operation input device 11 for operation input has been changed using the operation start signal input from the operation signal reception unit 21. When the operation input device 11 for operation input is changed, the change presence / absence determination unit 24 needs to switch between an operation start signal indicating the operation input device 11 before the change and an operation start signal indicating the operation input device 11 after the change. This is output to the rejection determination unit 25.
  • the operation input device 11 for operation input when the operation input device 11 for operation input is changed, the operation input device 11 before the change is referred to as a “first operation input device”, and the operation input device 11 after the change is a “second operation input device”. That's it.
  • the switching necessity determination unit 25 uses the operation start signal input from the change presence / absence determination unit 24 to switch the first operation input device and the second operation input device. Judgment.
  • the switching necessity determination unit 25 includes UI setting information corresponding to the first operation input device (hereinafter referred to as “first UI setting information”) and UI setting information corresponding to the second operation input device (hereinafter referred to as “first”. 2 UI setting information ”) from the UI setting information storage unit 23.
  • the switching necessity determination unit 25 determines whether to switch the screen UIs of the display devices 2 and 9 using the first UI setting information and the second UI setting information.
  • the switching necessity determination unit 25 compares the content of the first UI setting information with the content of the second UI setting information. When the operation method of the first operation input device is different from the operation method of the second operation input device, the switching necessity determination unit 25 limits the operation content by the first operation input device and the operation content by the second operation input device. The display UI is switched, or the display devices 2 and 9 used for the operation of the first operation input device are different from the display devices 2 and 9 used for the operation of the second operation input device. Is determined. When it is determined that the screen UI is to be switched, the switching necessity determination unit 25 outputs the first UI setting information and the second UI setting information to the intermediate screen generation unit 26.
  • first UI the screen UI before switching
  • second UI the screen UI after switching
  • the intermediate screen generation unit 26 receives the first UI setting information, the second UI setting information, and the UI setting information input from the switching necessity determination unit 25.
  • the UI setting information indicating the resolution of the display devices 2 and 9 acquired from the storage unit 23 and the information regarding the screen being displayed acquired from the display control units 8 and 22, a screen based on the first UI (hereinafter referred to as “first” One screen ”) and a screen based on the second UI (hereinafter referred to as” second screen ”) are generated (hereinafter referred to as” intermediate screen ").
  • the intermediate screen generation unit 26 outputs the generated intermediate screen to the UI switching unit 27.
  • the first display control unit 22 displays the first screen immediately before switching the screen UI. It is a screen displayed on the device 2.
  • the first UI is a method of operating the full remote controller 3, simple remote controller 4, gesture input device 5 or voice input device 6, and restriction of operation contents by the full remote controller 3, simple remote controller 4, gesture input device 5 or voice input device 6. , And a screen UI corresponding to the resolution of the first display device 2 and the like.
  • the first screen is a screen displayed on the second display device 9 by the second display control unit 8 immediately before switching the screen UI.
  • the first UI is a screen UI corresponding to the operation method of the portable information terminal 7, the limitation of the operation content by the portable information terminal 7, the resolution of the second display device 9, and the like.
  • the first display control unit 22 displays the first screen immediately after the screen UI is switched. It is a screen displayed on the apparatus 2.
  • the second UI is a method of operating the full remote control 3, simple remote control 4, gesture input device 5 or voice input device 6, and restriction of operation contents by the full remote control 3, simple remote control 4, gesture input device 5 or voice input device 6. , And a screen UI corresponding to the resolution of the first display device 2 and the like.
  • the second screen is a screen that the second display control unit 8 displays on the second display device 9 immediately after the switching of the screen UI.
  • the second UI is a screen UI corresponding to the operation method of the portable information terminal 7, the limitation of the operation content by the portable information terminal 7, the resolution of the second display device 9, and the like.
  • the intermediate screen is one or more intermediate screens in a so-called “animation display” in which the screens of the display devices 2 and 9 are smoothly switched from the first screen to the second screen.
  • the intermediate screen generation unit 26 when both the first screen and the second screen are screens displayed on the first display device 2, the intermediate screen generation unit 26 generates one or more intermediate screens for the first display device 2.
  • the intermediate screen for the first display device 2 is an intermediate screen in which the screen of the first display device 2 is smoothly switched from the first screen to the second screen.
  • the intermediate screen generation unit 26 includes the first display device.
  • One or more intermediate screens for 2 and one or more intermediate screens for the second display device 9 are respectively generated.
  • the intermediate screen for the first display device 2 is an intermediate screen in which the screen of the first display device 2 smoothly switches from the first screen to another screen (for example, a black screen).
  • the intermediate screen for the second display device 9 is an intermediate screen in which the screen of the second display device 9 is smoothly switched from another screen (for example, a black screen) to the second screen.
  • the intermediate screen generation unit 26 selects the second display device.
  • One or more intermediate screens for 9 and one or more intermediate screens for the first display device 2 are respectively generated.
  • the intermediate screen for the second display device 9 is an intermediate screen in which the screen of the second display device 9 is smoothly switched from the first screen to another screen (for example, a black screen).
  • the intermediate screen for the first display device 2 is an intermediate screen in which the screen of the first display device 2 smoothly switches from another screen (for example, a black screen) to the second screen.
  • the UI switching unit 27 switches the screens of the display devices 2 and 9 from the first screen to the second screen by animation display using the intermediate screen input from the intermediate screen generation unit 26.
  • the UI switching unit 27 sequentially displays the intermediate screen for the first display device 2 on the first display control unit 22. Output.
  • the UI switching unit 27 instructs the first display control unit 22 to sequentially display the intermediate screen for the first display device 2 on the first display device 2.
  • the screen of the first display device 2 is switched from the first screen to the second screen by animation display.
  • the UI switching unit 27 displays the first display device 2.
  • the intermediate screen for the second display device 9 is sequentially output to the first display control unit 22 and the intermediate screen for the second display device 9 is sequentially output to the second display control unit 8.
  • the UI switching unit 27 instructs the first display control unit 22 to sequentially display the intermediate screen for the first display device 2 on the first display device 2, and the second intermediate screen for the second display device 9 is displayed on the second screen.
  • the second display control unit 8 is instructed to sequentially display on the display device 9.
  • the screen of the first display device 2 is switched from the first screen to another screen by the animation display
  • the screen of the second display device 9 is switched from the other screen to the second screen by the animation display.
  • the UI switching unit 27 displays the second display device 9.
  • the intermediate screen for the first display device 2 is sequentially output to the second display control unit 8, and the intermediate screen for the first display device 2 is sequentially output to the first display control unit 22.
  • the UI switching unit 27 instructs the second display control unit 8 to sequentially display the intermediate screen for the second display device 9 on the second display device 9, and also displays the intermediate screen for the first display device 2 as the first screen.
  • the first display control unit 22 is instructed to sequentially display on the display device 2.
  • the screen of the second display device 9 is switched from the first screen to another screen by the animation display
  • the screen of the first display device 2 is switched from the other screen to the second screen by the animation display.
  • the operation signal receiving unit 21, the first display control unit 22, the UI setting information storage unit 23, and the UI control device 100 constitute a main part of the electronic device 1.
  • FIG. 2 shows an example of the hardware configuration of the main part of the electronic device 1.
  • the operation signal receiving unit 21 illustrated in FIG. 1 includes a dedicated receiver 41.
  • the UI setting information storage unit 23 illustrated in FIG. 1 includes a memory 42.
  • the memory 42 stores programs for realizing the functions of the display control unit 22, the change presence / absence determination unit 24, the switching necessity determination unit 25, the intermediate screen generation unit 26, and the UI switching unit 27 illustrated in FIG. ing.
  • the processor 43 reads and executes the program stored in the memory 42, the display control unit 22, the change presence / absence determination unit 24, the switching necessity determination unit 25, the intermediate screen generation unit 26, and the UI switching unit 27 illustrated in FIG. The function is realized.
  • the memory 42 may be, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Memory Memory), or the like.
  • An optical disk or a magneto-optical disk is used.
  • the processor 43 includes, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), a microcontroller, or a microprocessor.
  • FIG. 3 shows another example of the hardware configuration of the main part of the electronic device 1.
  • the display control unit 22, the change presence / absence determination unit 24, the switching necessity determination unit 25, the intermediate screen generation unit 26, and the UI switching unit 27 illustrated in FIG. 1 are configured by a dedicated processing circuit 44.
  • the processing circuit 44 is, for example, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a system LSI (Large-Scale Integration), or a combination thereof.
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • LSI Large-Scale Integration
  • the processing circuit 44 may be realized by the processing circuit 44.
  • the functions of the respective units may be collectively realized by the processing circuit 44. Further, some of the functions of the display control unit 22, the change presence / absence determination unit 24, the switching necessity determination unit 25, the intermediate screen generation unit 26, and the UI switching unit 27 illustrated in FIG. 43, and the remaining functions may be realized by the processing circuit 44 shown in FIG.
  • step ST1 the operation signal receiving unit 21 receives an operation start signal.
  • the operation signal receiving unit 21 outputs the received operation start signal to the UI control device 100.
  • step ST2 the change presence / absence determination unit 24 determines whether or not the operation input device 11 for operation input has been changed using the operation start signal input from the operation signal reception unit 21 in step ST1. .
  • the change presence / absence determining unit 24 operates the operation start signal input from the operation signal receiving unit 21 in the previous step ST1 and the operation start signal input from the operation signal receiving unit 21 in the previous step ST1. And compare. When the two operation start signals indicate the same operation input device 11, the change presence / absence determination unit 24 determines that the operation input device 11 for operation input has not been changed. On the other hand, when the two operation start signals indicate different operation input devices 11, the change presence / absence determination unit 24 determines that the operation input device 11 for operation input has been changed.
  • step ST2 “NO”) If the operation input device 11 for operation input has not been changed (step ST2 “NO”), the processing of the electronic device 1 returns to step ST1.
  • step ST2 “YES” when the operation input device 11 for operation input is changed (step ST2 “YES”), the change presence / absence determination unit 24 starts an operation start signal indicating the first operation input device and an operation start indicating the second operation input device. The signal is output to the switching necessity determination unit 25.
  • step ST3 the switching necessity determination unit 25 determines the first operation input device and the second operation input device using the operation start signal input from the change presence / absence determination unit 24 in step ST2.
  • the switching necessity determination unit 25 acquires the first UI setting information corresponding to the first operation input device and the second UI setting information corresponding to the second operation input device from the UI setting information storage unit 23.
  • step ST4 the switching necessity determination unit 25 determines whether to switch the screen UI of the display devices 2 and 9 using the first UI setting information and the second UI setting information acquired in step ST3. To do.
  • the switching necessity determination unit 25 compares the content of the first UI setting information with the content of the second UI setting information.
  • the switching necessity determination unit 25 limits the operation content by the first operation input device and the operation content by the second operation input device.
  • the display UI is switched, or the display devices 2 and 9 used for the operation of the first operation input device are different from the display devices 2 and 9 used for the operation of the second operation input device. Is determined.
  • the operation method of the first operation input device and the operation method of the second operation input device are the same, and the restriction of the operation content by the first operation input device and the restriction of the operation content by the second operation input device are the same.
  • the switching necessity determination unit 25 displays the screen. It is determined that the UI is not switched.
  • step ST4 “NO”) the processing of the electronic device 1 returns to step ST1.
  • step ST4 “YES” the switching necessity determination unit 25 outputs the first UI setting information and the second UI setting information to the intermediate screen generation unit 26.
  • the intermediate screen generating unit 26 acquires various information used for generating the intermediate screen. Specifically, for example, the intermediate screen generation unit 26 acquires UI setting information indicating the resolution of the display devices 2 and 9 from the UI setting information storage unit 23. Further, the intermediate screen generation unit 26 acquires information regarding the screen being displayed from the display control units 8 and 22.
  • step ST6 the intermediate screen generation unit 26 uses the first UI setting information and the second UI setting information input from the switching necessity determination unit 25 in step ST4 and the various information acquired in step ST5. Then, an intermediate image between the first screen based on the first UI and the second screen based on the second UI is generated.
  • the intermediate screen generation unit 26 outputs the generated intermediate image to the UI switching unit 27.
  • step ST7 the UI switching unit 27 uses the intermediate screen input from the intermediate screen generation unit 26 in step ST6 to change the screens of the display devices 2 and 9 from the first screen to the second screen by animation display. Switch.
  • the UI switching unit 27 sequentially displays the intermediate screen for the first display device 2 on the first display control unit 22. Output.
  • the UI switching unit 27 instructs the first display control unit 22 to sequentially display the intermediate screen for the first display device 2 on the first display device 2.
  • the UI switching unit 27 is configured to display the first display device 2. Intermediate screens are sequentially output to the first display control unit 22, and intermediate screens for the second display device 9 are sequentially output to the second display control unit 8.
  • the UI switching unit 27 instructs the first display control unit 22 to sequentially display the intermediate screen for the first display device 2 on the first display device 2, and the second intermediate screen for the second display device 9 is displayed on the second screen.
  • the second display control unit 8 is instructed to sequentially display on the display device 9.
  • the UI switching unit 27 is configured to display the second display device 9. Intermediate screens are sequentially output to the second display control unit 8 and intermediate screens for the first display device 2 are sequentially output to the first display control unit 22.
  • the UI switching unit 27 instructs the second display control unit 8 to sequentially display the intermediate screen for the second display device 9 on the second display device 9, and also displays the intermediate screen for the first display device 2 as the first screen.
  • the first display control unit 22 is instructed to sequentially display on the display device 2.
  • step ST7 the screens of the display devices 2 and 9 are switched from the first screen to the second screen by animation display.
  • the first display control unit 22 causes the first display control unit 22 to display a screen based on the second UI
  • the second display control unit 8 causes the second display control unit 8 to display a screen based on the second UI.
  • the display process is continuously executed.
  • the electronic device 1 is configured by a television receiver integrated with a hard disk recorder, and has a function of recording a received program (so-called “recording function”) and a function of playing back a recorded program (so-called “playback function”). )have.
  • the first display control unit 22 causes the first display device 2 to display a screen for selecting a program to be played back from a plurality of recorded programs.
  • the full remote control 3 is set to the operation input device 11 for operation input.
  • the operation method of the full remote controller 3 uses a plurality of physical keys including direction keys.
  • the full remote controller 3 can perform detailed setting operations using a plurality of physical keys, and can perform detailed direction operations using direction keys.
  • the first display device 2 is used for the operation of the full remote control 3.
  • the screen UI corresponding to these contents is, for example, a screen in which thumbnail images corresponding to each program are arranged in a tile shape so that any one of the thumbnail images can be selected by pressing a direction key.
  • FIG. 5 shows an example of the first screen based on the screen UI.
  • the first screen A 1 is a screen thumbnail image B corresponding to each program are arranged like tiles.
  • a direction key provided on the full remote control 3 any one of the plurality of thumbnail images B can be selected.
  • a state in which the thumbnail images B S which is arranged at the center portion of the first screen A 1 is selected and displayed by other thumbnail image B different colors.
  • the operation input device 11 for operation input is changed from the full remote control 3 to the gesture input device 5.
  • the operation method of the gesture input device 5 is based on gesture input.
  • the gesture input device 5 can only perform a rough setting operation by hand sign, and can only perform a rough direction operation by hand movement.
  • the first display device 2 is used for the operation of the gesture input device 5.
  • the screen UI corresponding to these contents is, for example, a screen on which only some thumbnail images of a plurality of thumbnail images arranged in a horizontal row are displayed, depending on the movement of the operator's hand.
  • the thumbnail image can be scrolled in the horizontal direction, and the thumbnail image arranged at the center of the screen is always selected.
  • FIG. 6 shows an example of the second screen based on the screen UI.
  • the second screen A 2 is a screen for only a part of the thumbnail images B of the plurality of thumbnail images B arranged in a row is displayed. These thumbnail images B can be scrolled horizontally in accordance with the movement of the operator's hand.
  • a state in which the thumbnail images B S which is arranged at the center portion of the second screen A 2 is always selected.
  • FIG. 7 shows an example of the animation display.
  • the intermediate screen generation unit 26 indicates an example of generating only one intermediate screen A M.
  • Figure 7A shows a first screen A 1 similar to FIG. 5.
  • Intermediate screen A M shown in FIG. 7B is made of an enlarged area C 1 including the thumbnail images B S of the first screen A 1 shown in FIG. 7A on the entire screen.
  • the second screen A 2 shown in FIG. 7C is made of an enlarged area C M including the thumbnail images B S of the intermediate screen A M shown in FIG. 7B the entire screen.
  • the area C 1 is an area having a predetermined size
  • the area CM is an area having the same size as the area C 1 .
  • the first screen A 1 and the second screen A 2 are both, is a screen for selecting a reproduction object program in reproduction function. That is, the second screen A 2 is a screen corresponding to the first screen A 1 same functions as.
  • the first screen A 1 and the second screen A 2 is a screen in a state where the same thumbnail image B S is selected. That is, the second screen A 2 is a screen corresponding to the same operating conditions as the first screen A 1.
  • the first screen is not limited to the examples shown in FIGS. 5 and 7A.
  • the first screen may be a screen corresponding to any function as long as the electronic device 1 has the function.
  • the first screen may be a screen corresponding to any operation state in the function.
  • the second screen is not limited to the examples shown in FIGS. 6 and 7C.
  • the second screen may be a screen corresponding to a function different from the first screen. In this case, it is possible to display visually that the second screen is a screen corresponding to a function different from the first screen by animation display.
  • the second screen may be a screen corresponding to an operation state different from the first screen in a screen corresponding to the same function as the first screen. In this case, it is possible to display visually that the second screen is a screen corresponding to an operation state different from the first screen by animation display.
  • the operation input device 11 is not limited to the full remote control 3, the simple remote control 4, the gesture input device 5, the voice input device 6, and the portable information terminal 7.
  • the number of operation input devices 11 is not limited to five, and the number of operation input devices 11 of each type is not limited to one.
  • the gesture input device 5 may be configured integrally with the portable information terminal 7.
  • the voice input device 6 may be configured integrally with the portable information terminal 7.
  • the voice input device 6 may be integrated with the full remote controller 3 or the simple remote controller 4.
  • the operation input device 11 may include a plurality of portable information terminals 7.
  • the resolution of the second display device 9 provided in each portable information terminal 7 may be different.
  • the operation input device 11 may include another electronic device different from the electronic device 1.
  • the other electronic device is configured by, for example, a television receiver, a personal computer, or a computer dedicated to video games.
  • the other electronic device includes a second display control unit 8 and a second display device 9 as with the portable information terminal 7.
  • the second display device 9 provided in the other electronic device may have a higher resolution than the second display device 9 provided in the portable information terminal 7.
  • the electronic device 1 is not limited to a television receiver.
  • the UI control device 100 can be applied to any electronic device 1 as long as it can be operated by a plurality of types of operation input devices 11.
  • the UI setting information corresponding to each operation input device 11 is used for the operation method of each operation input device 11, the limitation of the operation content by each operation input device 11, and the operation of each operation input device 11. Only a part of the display devices 2 and 9 may be shown.
  • the UI setting information corresponding to each operation input device 11 indicates only the operation method of each operation input device 11. Also good.
  • the first UI is a screen UI corresponding to the operation method of the first operation input device
  • the second UI is a screen UI corresponding to the operation method of the second operation input device.
  • the switching necessity determination unit 25 compares the content of the first UI setting information with the content of the second UI setting information, and the operation method of the first operation input device is different from the operation method of the second operation input device. In this case, it is determined that the screen UI is switched.
  • the UI setting information corresponding to each operation input device 11 includes the operation method of each operation input device 11 and each operation input device. 11 may only indicate the limitation of the operation content.
  • the first UI is a screen UI corresponding to the operation method of the first operation input device and the limitation of the operation content by the first operation input device
  • the second UI is the operation method of the second operation input device and the second operation input device.
  • 6 is a screen UI corresponding to the restriction of the operation content by.
  • the switching necessity determination unit 25 compares the content of the first UI setting information with the content of the second UI setting information, and the operation method of the first operation input device is different from the operation method of the second operation input device. In the case where the limitation of the operation content by the first operation input device is different from the limitation of the operation content by the second operation input device, it is determined that the screen UI is switched.
  • the UI setting information corresponding to each operation input device 11 includes the operation method of each operation input device 11 and each operation input device. Only the display devices 2 and 9 used for the operation of 11 may be shown.
  • the first UI is a screen UI corresponding to the operation method of the first operation input device and the resolution of the display devices 2 and 9 used for the operation of the first operation input device
  • the second UI is the operation of the second operation input device. It is screen UI corresponding to the resolution of the display apparatuses 2 and 9 used for operation of a method and a 2nd operation input device.
  • the switching necessity determination unit 25 compares the content of the first UI setting information with the content of the second UI setting information, and the operation method of the first operation input device is different from the operation method of the second operation input device. If the display devices 2 and 9 used for the operation of the first operation input device are different from the display devices 2 and 9 used for the operation of the second operation input device, it is determined that the screen UI is switched.
  • the data amount of the UI setting information can be reduced.
  • the content of the UI setting information corresponding to each operation input device 11 it is possible to set a screen UI that is more suitable for the operation environment of the electronic device 1.
  • FIG. 1 shows an example in which the UI control device 100 and the UI setting information storage unit 23 are provided in the electronic device 1.
  • the UI control device 100 and the UI setting information storage unit 23 are outside the electronic device 1. It may be provided.
  • the UI control device 100 and the UI setting information storage unit 23 may be provided in a server device that can communicate with the electronic device 1 and the portable information terminal 7.
  • the server apparatus having the UI control apparatus 100 communicates with the electronic device 1 and the portable information terminal 7 as appropriate, so that processing similar to the flowchart shown in FIG. 4 is executed.
  • the UI control device 100 may include an attribute detection unit 28 and an operation auxiliary unit 29 as shown in FIG.
  • the attribute detection unit 28 detects the operator's attribute using the sensor 12 provided in the electronic device 1.
  • the operation assisting unit 29 determines whether or not the operation input device 11 for operation input is set to the gesture input device 5 using the operation start signal input from the operation signal receiving unit 21.
  • the operation auxiliary unit 29 assists an operation by gesture input according to the attribute of the operator (hereinafter referred to as “operation auxiliary image”). .).
  • the operation assisting unit 29 instructs the first display control unit 22 to display the operation assisting image on the first display device 2.
  • the sensor 12 is configured by a camera that photographs the operator.
  • the attribute detection unit 28 performs image recognition processing on an image captured by the camera, and detects attributes such as whether the operator is a child or whether the operator is an elderly person.
  • the operation assisting unit 29 causes the first display device 2 to display an arrow-shaped operation assisting image indicating a direction that can be operated by a gesture input.
  • the operation auxiliary image is displayed by being superimposed on a screen such as the first screen, the second screen, or the intermediate screen.
  • the operation auxiliary unit 29 is displayed on the intermediate screen (FIG. 7B) and the second screen (FIG. 7C).
  • the operation assistance image is displayed by superimposing. Thereafter, the operation assisting unit 29 continues the process of displaying the operation assisting image superimposed on the screen based on the second UI.
  • the operation auxiliary unit 29 displays an operation auxiliary image superimposed on the screen based on the first UI.
  • the operation auxiliary image is displayed superimposed on the first screen (FIG. 7C) and the intermediate screen (FIG. 7B).
  • the second screen (FIG. 7A) is displayed on the first display device 2
  • the operation assisting unit 29 ends the process of displaying the operation assisting image.
  • the UI control apparatus 100 is the UI control apparatus 100 for the electronic apparatus 1 that can be operated by a plurality of types of operation input apparatuses 11, and the operation input apparatus 11 for operation input is the first.
  • a switching necessity determination unit 25 for determining whether or not to switch the screen UI of the display devices 2 and 9 from the first UI to the second UI, and a necessity determination for switching.
  • the intermediate screen generating unit 26 that generates an intermediate screen between the first screen based on the first UI and the second screen based on the second UI, and animation display using the intermediate screen
  • a UI switching unit 27 for switching the screens of the display devices 2 and 9 from the first screen to the second screen.
  • the second screen is a screen corresponding to the same function as the first screen. From the animation display, it is possible to display visually that the second screen is a screen corresponding to the same function as the first screen.
  • the second screen is a screen corresponding to the same operation state as the first screen. From the animation display, it is possible to display visually that the second screen is a screen corresponding to the same operation state as the first screen.
  • the first UI is a screen UI corresponding to the operation method of the first operation input device
  • the second UI is a screen UI corresponding to the operation method of the second operation input device.
  • the switching necessity determination unit 25 compares the first UI setting information indicating the operation method of the first operation input device with the second UI setting information indicating the operation method of the second operation input device, and compares the information. Whether or not to switch the screen UI is determined using the result. Thereby, based on the operation method of the operation input apparatus 11, it can be determined whether the screen UI of the display apparatuses 2 and 9 is switched.
  • the first UI is a screen UI corresponding to the restriction of the operation content by the first operation input device
  • the second UI is the screen UI corresponding to the restriction of the operation content by the second operation input device.
  • the switching necessity determination unit 25 also includes a first UI setting information indicating an operation method of the first operation input device and a restriction on an operation content by the first operation input device, an operation method of the second operation input device, and a second operation.
  • the second UI setting information indicating the limitation of the operation content by the input device is compared, and it is determined whether or not to switch the screen UI using the result of the comparison. As a result, it is possible to determine whether or not to switch the screen UI of the display devices 2 and 9 based on the operation method of the operation input device 11 and the restriction of the operation content by the operation input device 11.
  • the display devices 2 and 9 are configured by a plurality of display devices 2 and 9 having different resolutions, and the first UI corresponds to the resolution of the display devices 2 and 9 used for the operation of the first operation input device.
  • the second UI is a screen UI corresponding to the resolution of the display devices 2 and 9 used for the operation of the second operation input device. Accordingly, it is possible to provide an operation environment suitable for the resolution of the display devices 2 and 9 used for the operation of the operation input device 11 before and after switching the screen UI.
  • the switching necessity determination unit 25 includes an operation method of the first operation input device, first UI setting information indicating the display devices 2 and 9 used for the operation of the first operation input device, and an operation of the second operation input device.
  • the method and the second UI setting information indicating the display devices 2 and 9 used for the operation of the second operation input device are compared, and it is determined whether to switch the screen UI using the result of the comparison. Accordingly, it is possible to determine whether or not to switch the screen UI of the display devices 2 and 9 based on the operation method of the operation input device 11 and the display devices 2 and 9 used for the operation of the operation input device 11.
  • the operation method of the second operation input device is by gesture input.
  • the correspondence relationship between the first screen and the second screen can be visually and easily displayed by the animation display illustrated in FIG.
  • the UI control device 100 includes an operation assisting unit 29 that displays an operation assisting image that assists an operation by gesture input in a superimposed manner on the intermediate screen and the second screen according to the attribute of the operator.
  • an operation assisting unit 29 displays an operation assisting image that assists an operation by gesture input in a superimposed manner on the intermediate screen and the second screen according to the attribute of the operator.
  • the operation method of the first operation input device is by gesture input.
  • the correspondence relationship between the first screen and the second screen can be visually and easily displayed by the animation display obtained by reversing the animation display illustrated in FIG. 7 in terms of time.
  • the UI control device 100 includes an operation assisting unit 29 that displays an operation assisting image that assists an operation by gesture input in a superimposed manner on the first screen and the intermediate screen according to the attribute of the operator.
  • an operation assisting unit 29 displays an operation assisting image that assists an operation by gesture input in a superimposed manner on the first screen and the intermediate screen according to the attribute of the operator.
  • the user interface control method (hereinafter referred to as “UI control method”) according to the first embodiment is a UI control method for the electronic device 1 that can be operated by a plurality of types of operation input devices 11, and determines whether or not switching is necessary. Whether the unit 25 switches the screen UI of the display devices 2 and 9 from the first UI to the second UI when the operation input device 11 for operation input is changed from the first operation input device to the second operation input device.
  • step ST4 and the intermediate screen generation unit 26 determine that the switching UI is to be switched by the switching necessity determination unit 25, the first screen based on the first UI and the second screen based on the second UI
  • step ST6 The step of generating the intermediate screen (step ST6), and the UI switching unit 27 displays the screens of the display devices 2 and 9 from the first screen to the second screen by animation display using the intermediate screen. Switches and a step (step ST7).
  • the screen UI is switched in accordance with the change of the operation input device 11, the correspondence between the first screen and the second screen can be displayed visually and intelligibly from the animation display.
  • Embodiment 2 FIG. In the first embodiment, a specific example of the animation display when the operation input device 11 for operation input is changed from the full remote control 3 to the gesture input device 5 has been described with reference to FIGS. In the second embodiment, a specific example of animation display when the operation input device 11 for operation input is changed from the full remote control 3 to the portable information terminal 7 will be described with reference to FIGS.
  • the configurations of the electronic device 1, the first display device 2, and the operation input device 11 are the same as those described with reference to FIGS. 1 to 3 in the first embodiment.
  • the description is omitted while being used.
  • the operation of the electronic device 1 is the same as that described with reference to FIG. 4 in the first embodiment, the description is omitted while using FIG.
  • the electronic device 1 is composed of a television receiver integrated with a hard disk recorder, and has a recording function and a playback function.
  • the first display control unit 22 causes the first display device 2 to display a screen for selecting a program to be played back from a plurality of recorded programs.
  • FIG. 9 shows an example of the first screen based on the screen UI for the full remote control 3.
  • the first screen A 1 shown in FIG. 9 are the same screen as the first screen A 1 shown in FIG. 5, the description thereof is omitted.
  • the operation input device 11 for operation input is changed from the full remote control 3 to the portable information terminal 7.
  • the operation method of the portable information terminal 7 uses the touch panel 10 configured integrally with the second display device 9.
  • the portable information terminal 7 can perform the same operation as the full remote control 3 using the second display device 9 and the touch panel 10.
  • the second display device 9 is used for the operation of the portable information terminal 7.
  • the screen UI corresponding to these contents is one in which any thumbnail image can be freely selected by operating the touch panel 10 on a screen in which thumbnail images corresponding to each program are arranged in a tile shape, for example.
  • FIG. 10 shows an example of the second screen based on the screen UI.
  • the second screen A 2 is a screen thumbnail image B corresponding to each program it is arranged like tiles.
  • any one of the plurality of thumbnail images B can be selected freely.
  • a state in which the thumbnail images B S which is arranged at the center portion of the second screen A 2 is selected is displayed in another thumbnail image B with different colors.
  • the second screen A 2 shown in FIG. 10 is an enlarged view of a portion of the first screen A 1 shown in FIG.
  • the enlargement magnification is a value corresponding to the ratio of the resolution of the second display device 9 to the resolution of the first display device 2.
  • it is possible in the second screen A 2 are displayed large individual thumbnail images B, preventing selection erroneous operation of the touch panel 10.
  • FIG. 11 shows an example of the animation display.
  • the first display unit 2 are displayed first screen A 1 is, the screen of the second display device 9 is black.
  • the intermediate screen A M1 is displayed on the first display device 2 and the intermediate screen A M2 is displayed on the second display device 9.
  • Intermediate screen A M1 of the first display device 2 the first screen A 1 corresponds to the middle of the screen is slid out towards the paper downward.
  • an arrow D M1 shows a first sliding direction of the screen A 1 in the first display device 2.
  • the intermediate screen A M2 of the second display device 9, the first screen A 1 corresponds to the middle of the screen is slid in from the paper upward.
  • an arrow D M2 shows a first sliding direction of the screen A 1 in the second display device 9.
  • the screen of the first display device 2 becomes black, and the intermediate screen AM3 is displayed on the second display device 9.
  • Intermediate screen A M3 of the second display device 9 corresponds to the screen in a state where slide-in of the first screen A 1 is completed.
  • the second screen A 2 is displayed on the second display device 9.
  • the second screen A 2 is one in which the region C M including the thumbnail images B S of the intermediate screen A M3 becomes enlarged to fill the screen.
  • the magnification of the enlargement is a value corresponding to the resolution ratio of the second display device 9 to the resolution of the first display device 2.
  • the first screen A 1 and the second screen A 2 are both, is a screen for selecting a reproduction object program in reproduction function. That is, the second screen A 2 is a screen corresponding to the first screen A 1 same functions as.
  • the first screen A 1 and the second screen A 2 is a screen in a state where the same thumbnail image B S is selected. That is, the second screen A 2 is a screen corresponding to the same operating conditions as the first screen A 1.
  • the other screen after switching from the first screen or the other screen before switching to the second screen is not limited to the black screen.
  • These other screens may be screens corresponding to other functions of the electronic device 1, for example.
  • the other screen after switching from the first screen or the other screen before switching to the second screen is not limited to the black screen.
  • These other screens may be screens corresponding to other functions of the portable information terminal 7, for example.
  • the sliding direction of the screen in the animation display is not limited to the vertical direction on the paper surface shown in FIG.
  • the sliding direction may be a fixed direction set in advance, or may be set by the UI control device 100 when executing animation display.
  • a functional block diagram in this case is shown in FIG.
  • the position detection unit 30 uses the sensor 12 provided in the electronic device 1 when the portable information terminal 7 is set as the operation input device 11 for operation input, and uses the first display device 2 and the second display device 9. Is detected.
  • the sensor 12 is configured by a camera that captures an operator who has the portable information terminal 7.
  • the sensor 12 is configured by a camera that takes an image of the first display device 2 and the operator who holds the portable information terminal 7.
  • the position detection unit 30 executes an image recognition process on the image captured by the camera, and detects the positional relationship between the first display device 2 and the second display device 9.
  • the intermediate screen generator 26 sets the slide direction in the animation display according to the positional relationship detected by the position detector 30. That is, when the operation input device 11 for operation input is changed from the full remote control 3 or the simple remote control 4 to the portable information terminal 7, the intermediate screen generation unit 26 is directed from the first display device 2 to the second display device 9. Set the slide direction to. Thereby, irrespective of the positional relationship between the electronic device 1 and the portable information terminal 7, it can be displayed visually and easily that the screen of the first display device 2 has moved to the second display device 9. As a result, it is possible to prevent the operator from misunderstanding that the operation screen has disappeared.
  • the intermediate screen generation unit 26 is directed from the second display device 9 to the first display device 2.
  • the electronic device 1 according to the second embodiment can employ various modifications similar to those described in the first embodiment.
  • the UI control apparatus 100 according to the second embodiment can employ various modifications similar to those described in the first embodiment.
  • the display devices 2 and 9 include the first display device 2 provided in the electronic apparatus 1 and the second display device 9 provided in the second operation input device, and the first screen is displayed. Is a screen displayed on the first display device 2, the second screen is a screen displayed on the second display device 9, and the operation method of the second operation input device is integrated with the second display device 9.
  • the touch panel 10 configured as described above is used.
  • the second operation input device is the portable information terminal 7, the correspondence between the first screen and the second screen can be visually and easily displayed by the animation display illustrated in FIG.
  • the intermediate screen generation unit 26 generates an intermediate screen corresponding to the positional relationship between the first display device 2 and the second display device 9.
  • the slide direction can be set in the direction from the first display device 2 toward the second display device 9 regardless of the positional relationship between the electronic device 1 and the portable information terminal 7.
  • the fact that the screen of the first display device 2 has moved to the second display device 9 can be displayed in a visually easy-to-understand manner.
  • the display devices 2 and 9 include a first display device 2 provided in the electronic apparatus 1 and a second display device 9 provided in the first operation input device, and the first screen is the second screen. It is a screen displayed on the display device 9, and the second screen is a screen displayed on the first display device 2, and the operation method of the first operation input device is configured integrally with the second display device 9.
  • the touch panel 10 is used.
  • the first operation input device is the portable information terminal 7, it is easy to visually understand the correspondence between the first screen and the second screen by displaying the animation illustrated in FIG. Can be displayed.
  • the intermediate screen generation unit 26 generates an intermediate screen corresponding to the positional relationship between the first display device 2 and the second display device 9. Thereby, irrespective of the positional relationship between the electronic device 1 and the portable information terminal 7, the slide direction can be set in the direction from the second display device 9 toward the first display device 2. As a result, the fact that the screen of the second display device 9 has moved to the first display device 2 can be displayed in a visually easy-to-understand manner.
  • Embodiment 3 FIG.
  • a specific example of the animation display when the operation input device 11 for operation input is changed from the full remote control 3 to the gesture input device 5 has been described with reference to FIGS.
  • a specific example of animation display when the operation input device 11 for operation input is changed from the full remote control 3 to the voice input device 6 will be described with reference to FIGS.
  • the configurations of the electronic device 1, the first display device 2, and the operation input device 11 are the same as those described with reference to FIGS. 1 to 3 in the first embodiment.
  • the description is omitted while being used.
  • the operation of the electronic device 1 is the same as that described with reference to FIG. 4 in the first embodiment, the description is omitted while using FIG.
  • the electronic device 1 is composed of a television receiver integrated with a hard disk recorder, and has a recording function and a playback function.
  • the first display control unit 22 causes the first display device 2 to display a screen for selecting a program to be played back from a plurality of recorded programs.
  • FIG. 13 shows an example of the first screen based on the screen UI for the full remote control 3.
  • the first screen A 1 shown in FIG. 13 are the same screen as the first screen A 1 shown in FIG. 5, the description thereof is omitted.
  • the operation input device 11 for operation input is changed from the full remote control 3 to the voice input device 6.
  • the operation method of the voice input device 6 is based on voice input.
  • the voice input device 6 cannot be used for a direction operation, and cannot be used for an operation that requires a high response speed.
  • the first display device 2 is used for the operation of the voice input device 6.
  • the screen UI corresponding to these contents is, for example, any icon by voice input of a program name or the like on a screen in which icon images of a predetermined shape including characters indicating the program name or the like of each program are arranged in a tile shape. The image can be freely selected.
  • FIG. 14 shows an example of the second screen based on the screen UI.
  • the second screen A 2 is a screen characters indicating a program name of each program (ABC, DEF, ......) icon image E comprising arranged in tiles.
  • One of the plurality of icon images E can be selected by voice input of a program name or the like to the voice input device 6.
  • a state in which the second screen icon image E S disposed in a central portion of the A 2 is selected, it is displayed in a color different from the other icon image E.
  • the size and arrangement of the individual icon images E in the second screen A 2 shown in FIG. 14 is similar to the size and arrangement of the individual thumbnail images B in the first screen A 1 shown in FIG. 13.
  • Animation display from the first screen A 1 to the second screen A 2 for example, a display of changing the cross-fading individual thumbnail images B to the corresponding icon image E.
  • FIG. 15 shows an example of the animation display.
  • the intermediate screen generation unit 26 indicates an example of generating only one intermediate screen A M.
  • Figure 15A shows a first screen A 1 similar to FIG. 13.
  • Figure 15C shows a second screen A 2 similar to FIG. 14.
  • Figure 15B is first screen A 1 and shows an intermediate screen A M between the second screen A 2.
  • the intermediate screen A M is obtained by making each of the first screen A 1 and the second screen A 2 translucent and superimposing them on each other. That is, the animation display shown in FIG. 15 is a first display device 2 of the screen is switched from the first screen A 1 by the cross-fade on the second screen A 2.
  • the first screen A 1 and the second screen A 2 are both, is a screen for selecting a reproduction object program in reproduction function. That is, the second screen A 2 is a screen corresponding to the first screen A 1 same functions as.
  • the first screen A 1 and the second screen A 2 is a screen in a state where the thumbnail image B S or icon image E S corresponding to the same program is selected. That is, the second screen A 2 is a screen corresponding to the same operating conditions as the first screen A 1.
  • the UI control device 100 may include an attribute detection unit 28 and a first character mode setting unit 31 as shown in FIG.
  • the attribute detection unit 28 detects the operator's attribute using the sensor 12 provided in the electronic device 1. Specific examples of the attribute detection unit 28 and the sensor 12 are the same as those described in the first embodiment with reference to FIG.
  • the first character form setting unit 31 uses the operation start signal input from the operation signal receiving unit 21 to determine whether or not the operation input device 11 for operation input is set to the voice input device 6. is there. When the operation input device 11 for operation input is set to the voice input device 6, the first character mode setting unit 31 sets the character display mode in the icon image according to the attribute of the operator. . The first character mode setting unit 31 instructs the first display control unit 22 to display individual icon images on the first display device 2 in the set display mode.
  • the first character form setting unit 31 converts the kanji included in the icon image into hiragana and displays it. Moreover, the 1st character aspect setting part 31 expands and displays the character contained in an icon image, when an operator is elderly people.
  • the first character form setting unit 31 displays the intermediate screen (FIG. 15B) and the kanji of each icon image on the second screen (FIG. 15C) are converted into hiragana and displayed. Thereafter, the first character form setting unit 31 continues the process of converting the kanji of each icon image on the screen based on the second UI into hiragana and displaying it.
  • the first character mode setting unit 31 displays an intermediate screen ( The characters of the individual icon images in FIG. 15B) and the second screen (FIG. 15C) are enlarged and displayed. Thereafter, the first character form setting unit 31 continues the process of enlarging and displaying the characters of the individual icon images on the screen based on the second UI.
  • the first character form setting unit 31 is based on the first UI. Continuing from the process of displaying the kanji of the individual icon images on the screen as hiragana, the kanji of the individual icon images on the first screen (FIG. 15C) and the intermediate screen (FIG. 15B) is converted into hiragana and displayed. Thereafter, when the second screen (FIG. 15A) is displayed on the first display device 2, the first character form setting unit 31 ends the process.
  • the first character mode setting unit 31 sets the first UI. Continuing from the process of enlarging and displaying the characters of the individual icon images on the base screen, the characters of the individual icon images on the first screen (FIG. 15C) and the intermediate screen (FIG. 15B) are enlarged and displayed. Thereafter, when the second screen (FIG. 15A) is displayed on the first display device 2, the first character form setting unit 31 ends the process.
  • the UI control device 100 may include a distance detection unit 32 and a second character mode setting unit 33 as shown in FIG.
  • the distance detection unit 32 detects the distance between the first display device 2 and the operator using the sensor 12 provided in the electronic device 1. Specifically, for example, when the first display device 2 is provided integrally with the electronic device 1, the sensor 12 is configured by a camera that photographs the operator. On the other hand, when the first display device 2 is provided as a separate member from the electronic device 1, the sensor 12 is configured by an operator and a camera that photographs the first display device 2. The distance detection unit 32 performs image recognition processing on the image captured by the camera, and detects the distance between the first display device 2 and the operator.
  • the second character mode setting unit 33 determines whether or not the operation input device 11 for operation input is set to the voice input device 6 using the operation start signal input from the operation signal receiving unit 21. is there. When the operation input device 11 for operation input is set to the voice input device 6, the second character mode setting unit 33 displays the character display mode in each icon image according to the distance detected by the distance detection unit 32. Is set.
  • the second character mode setting unit 33 changes the character size according to the distance. That is, when the operation input device 11 for operation input is changed from the full remote controller 3 or the simple remote controller 4 to the voice input device 6, the second character mode setting unit 33 displays the intermediate screen (FIG. 15B) and the second screen (FIG. 15). The character size of each icon image in 15C) is changed according to the distance. Thereafter, the second character mode setting unit 33 changes the character size of each icon image on the screen based on the second UI according to the distance.
  • the second character mode setting unit 33 displays the characters of the individual icon images on the screen based on the first UI. Continuing from the process of changing the size of the icon according to the distance, the size of the characters of the individual icon images on the first screen (FIG. 15C) and the intermediate screen (FIG. 15B) is changed according to the distance. Thereafter, when the second screen (FIG. 15A) is displayed on the first display device 2, the second character form setting unit 33 ends the process.
  • the readability of the character can be improved.
  • the electronic device 1 according to the third embodiment can employ various modifications similar to those described in the first embodiment.
  • the UI control apparatus 100 according to the third embodiment can employ various modifications similar to those described in the first embodiment.
  • the operation method of the second operation input device is by voice input.
  • the second operation input device is the voice input device 6
  • the correspondence between the first screen and the second screen can be displayed visually and easily by the animation display illustrated in FIG. 15.
  • the intermediate screen and the second screen are screens including character display
  • the UI control device 100 includes a first character mode setting unit 31 that sets a character display mode in the character display according to the attribute of the operator. . Thereby, the readability of the characters on the intermediate screen and the second screen can be improved.
  • the intermediate screen and the second screen are screens including character display, and the UI control device 100 sets the character display mode in the character display according to the distance between the display device 2 and the operator.
  • a setting unit 33 is provided. Thereby, the readability of the characters on the intermediate screen and the second screen can be improved.
  • the operation method of the first operation input device is by voice input.
  • the first operation input device is the voice input device 6
  • the correspondence between the first screen and the second screen can be visually easily understood by displaying the animation illustrated in FIG. Can be displayed.
  • the first screen and the intermediate screen are screens including character display
  • the UI control device 100 includes a first character mode setting unit 31 that sets a character display mode in the character display according to the attribute of the operator. . Thereby, the readability of the characters on the first screen and the intermediate screen can be improved.
  • the first screen and the intermediate screen are screens including character display
  • the UI control device 100 sets the character display mode in the character display according to the distance between the display device 2 and the operator.
  • a setting unit 33 is provided. Thereby, the readability of the characters on the first screen and the intermediate screen can be improved.
  • the UI control device of the present invention can be used for control of a screen UI in an electronic device having a plurality of types of operation input devices, for example, a television receiver.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

L'invention concerne un dispositif de commande (100) d'interface utilisateur (UI) qui est un dispositif de commande (100) d'UI destiné à un appareil électronique (1) qui peut être actionné librement à l'aide d'une pluralité de types de dispositifs d'entrée d'actionnement (11). Ce dispositif de commande d'UI (100) comporte : une unité de détermination de nécessité de commutation (25) qui détermine s'il faut commuter une UI d'écran de chaque dispositif d'affichage (2, 9), d'une première UI à une seconde UI, lorsqu'un dispositif d'entrée d'actionnement (11) destiné à entrer des actionnements est changé en un second dispositif d'entrée d'actionnement à partir d'un premier dispositif d'entrée d'actionnement ; une unité de génération d'écran intermédiaire (26) qui génère un écran intermédiaire entre le premier écran, sur la base de la première UI, et le second écran, sur la base de la seconde UI, dans les cas où l'unité de détermination de nécessité de commutation (25) détermine que l'UI d'écran doit être commutée ; et une unité de commutation d'UI (27) qui commute chaque écran des dispositifs d'affichage (2, 9) du premier écran au second écran au moyen d'un affichage d'animation à l'aide de l'écran intermédiaire.
PCT/JP2016/073250 2016-08-08 2016-08-08 Dispositif de commande d'interface utilisateur et procédé de commande d'interface utilisateur WO2018029738A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2018526966A JP6377307B2 (ja) 2016-08-08 2016-08-08 ユーザーインターフェース制御装置及びユーザーインターフェース制御方法
PCT/JP2016/073250 WO2018029738A1 (fr) 2016-08-08 2016-08-08 Dispositif de commande d'interface utilisateur et procédé de commande d'interface utilisateur

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2016/073250 WO2018029738A1 (fr) 2016-08-08 2016-08-08 Dispositif de commande d'interface utilisateur et procédé de commande d'interface utilisateur

Publications (1)

Publication Number Publication Date
WO2018029738A1 true WO2018029738A1 (fr) 2018-02-15

Family

ID=61161805

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2016/073250 WO2018029738A1 (fr) 2016-08-08 2016-08-08 Dispositif de commande d'interface utilisateur et procédé de commande d'interface utilisateur

Country Status (2)

Country Link
JP (1) JP6377307B2 (fr)
WO (1) WO2018029738A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009107589A1 (fr) * 2008-02-27 2009-09-03 京セラ株式会社 Dispositif de génération d'interface utilisateur
JP2013037689A (ja) * 2011-08-05 2013-02-21 Samsung Electronics Co Ltd 電子装置及びその制御方法
JP2015088086A (ja) * 2013-11-01 2015-05-07 ソニー株式会社 情報処理装置および情報処理方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009107589A1 (fr) * 2008-02-27 2009-09-03 京セラ株式会社 Dispositif de génération d'interface utilisateur
JP2013037689A (ja) * 2011-08-05 2013-02-21 Samsung Electronics Co Ltd 電子装置及びその制御方法
JP2015088086A (ja) * 2013-11-01 2015-05-07 ソニー株式会社 情報処理装置および情報処理方法

Also Published As

Publication number Publication date
JP6377307B2 (ja) 2018-08-22
JPWO2018029738A1 (ja) 2018-08-16

Similar Documents

Publication Publication Date Title
JP5912014B2 (ja) ３ｄリモートコントローラ用のｇｕｉアプリケーション
JP4102827B2 (ja) 電子装置のディスプレイ部でメニューを検索する方法及び装置
KR101493603B1 (ko) 외부 표시 장치와 접속가능한 표시 단말 장치 및 방법
US20110138275A1 (en) Method for selecting functional icons on touch screen
JP5900161B2 (ja) 情報処理システム、方法およびコンピュータ読み取り可能な記録媒体
TW200930096A (en) User interface for set top box
JP2013152693A (ja) 情報処理プログラム、情報処理装置、画像表示方法及び画像表示システム
US20060197751A1 (en) Display control apparatus and control method thereof
US20130162501A1 (en) Method for controlling multiple displays
JP6082190B2 (ja) プログラム、情報処理装置、画像表示方法及び表示システム
KR100703289B1 (ko) 문자 효과 아이템 입력 방법 및 장치
JP5113218B2 (ja) 携帯端末装置、表示制御方法、及び表示制御プログラム
JP2016038728A (ja) 画像表示装置、画像表示装置の制御方法、及びプログラム
JP2013250772A (ja) プログラム、情報処理装置、画像表示方法及び表示システム
US20070109261A1 (en) Information processing method and information processing apparatus
US8887101B2 (en) Method for moving a cursor and display apparatus using the same
JP6377307B2 (ja) ユーザーインターフェース制御装置及びユーザーインターフェース制御方法
US7594190B2 (en) Apparatus and method for user interfacing
JP4803951B2 (ja) 文字入力装置およびその処理方法、記録媒体、並びにプログラム
JP6058900B2 (ja) 情報処理システム、制御装置、画像表示方法及び情報処理プログラム
JP6587553B2 (ja) 撮像装置及び表示方法
US20200257396A1 (en) Electronic device and control method therefor
JP2013033499A (ja) 携帯端末装置、表示制御方法、及び表示制御プログラム
WO2019102885A1 (fr) Dispositif électronique avec section d'affichage d'image modifiable
JP2018106597A (ja) 表示制御装置およびその制御方法

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2018526966

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16912622

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16912622

Country of ref document: EP

Kind code of ref document: A1