JP2007066099A - Gui constitution system, gui constitution method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To constitute a GUI with feeling of unification by using UI data corresponding to the same functional category as a control target appliance, even if the control appliance does not have the UI data for constituting the GUI corresponding to the control target appliance. <P>SOLUTION: An easy-to-use UI for a user is provided by acquiring a category of a control target appliance on a network 106 and applying an optimum UI based on this. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a system for controlling personal computers, peripheral devices, AV devices, home appliances, and the like connected on a network.

  Conventionally, a concept of connecting household devices such as digitized home appliances to a network is known. For example, if a protocol such as Universal Plug & Play (hereinafter abbreviated as UPnP) is used, a plurality of devices can be directly connected to various networks and can be exchanged between devices. You can also know the existence and function of the equipment.

  In UPnP, there are “devices” that provide “services” that are some functions, and “control points” that discover and control the services. When a device is connected to the network, a control point on the network automatically detects that the device is connected. Of course, even if the control point is connected to the network to which the device is connected first, the control point automatically recognizes the presence of the device. In addition, the control point can know the service that the device has and can also execute it.

  For example, if a computer and audio amplifier are connected via a network, music data on the Internet is downloaded on the computer, and the audio amplifier power operation, music data designation operation, and playback instruction operations are performed by the controller displayed on the computer. By doing so, music data can be played back with an audio amplifier. In this example, the audio amplifier is a device operated as a device, and the personal computer functions as a device to be operated, that is, a control point. In this way, unlike the prior art, there is no one-to-one correspondence between the device to be operated and the device to be operated. Furthermore, if not only an audio amplifier but also a CD player, a DVD player, and the like exist on the network, there are a plurality of devices even if only devices capable of reproducing music data are present. Therefore, a device is also required for the GUI for controlling a plurality of devices. If the music data is reproduced as described above, it is desirable that it can be operated on the same control screen whether it is operated from a personal computer, a mobile phone, or from some multi-controller. In addition, it is preferable that the same GUI can be used for the operation of music data reproduction regardless of whether a CD player, a DVD player, or any other device is used.

  Conventionally, as a technology for controlling multiple multimedia devices connected to the network, when a device is connected, the control panel necessary for operation can be automatically displayed and the input / output relationship between the devices can be specified. Have been proposed (see, for example, Patent Document 1).

  In addition, there is a GUI that provides a GUI familiar to the user by preparing a GUI of a representative device used by the user in the DB and selecting and reusing the GUI from the DB when developing a new device (for example, Patent Document 2). reference).

JP 07-044477 A Japanese Patent Laid-Open No. 10-301761

  However, according to the technique of the above-mentioned patent document 1 by the present applicant, UI data such as AV equipment, home appliances, and the like cannot be provided with a category. For example, a GUI having an inorganic image for AV equipment. The design of the GUI could not be switched according to the field of use, such as unifying with a GUI that has a warm image for home appliances.

  Of the above-described conventional techniques, in the technique of Patent Document 2, when developing a UI for a new device, the UI developer needs to select UI data corresponding to the new function from the DB and embed it in the new device. was there. Furthermore, in order to deal with users with different GUI preferences, UI data for functions, such as layout and component information, can be diverted by selecting from the DB according to various preferences and combining them. A dedicated UI had to be designed for each different user.

  The present invention has been made in view of such problems, and relates to a case where a GUI of a control device is generated when a plurality of control target devices and a control device for performing the control are connected via a network. When the UI data necessary for generating the GUI is classified and stored by the function category of the control target device and the GUI of the control target device newly connected to the network is configured, the UI classified by the same function category is stored. An object is to provide a GUI familiar to the user by using the data.

  In addition, when a function that can be used by the user is added to the control target device, a GUI having a shape and layout suitable for the GUI being used is automatically configured to configure the control target device. The goal is to have a unified feeling in the GUI. In other words, since the control device side I do not know in advance what the control target device is connected, but not be prepared only a limited GUI at the time of development, by using the function category, even GUI of features that were not anticipated when the development It becomes possible to configure in accordance with other GUIs. Further, another object is to use a different GUI for each function category, so that a GUI with the same function category has a sense of unity, and a GUI with a different function category can be easily distinguished.

  In the GUI configuration system of the present invention, a control device for controlling a plurality of control target devices and the control target device operated by the control device are connected to a network, and the control device and the control are connected via the network. A GUI configuration system in which a target device can transmit and receive data including functions of each other, wherein the control device includes a GUI generation unit that configures a GUI from UI data describing a GUI component and layout, and the UI A UI data storage unit with category for storing data classified according to a category of functions of the control target device; and a function category acquisition unit for acquiring the function of the control target device, wherein the GUI generation unit includes the function The above-mentioned category-added UI data storage corresponding to the category acquired by the category acquisition means To generate a GUI by using the UI data of the stored category by stage.

  The GUI configuration method of the present invention includes a function category acquisition step of acquiring the function of the control target device, and a step of generating a GUI using UI data corresponding to the acquired category.

  According to the GUI configuration system of the present invention, even if the control device does not have the UI data for configuring the GUI corresponding to the control target device, the UI data corresponding to the same function category as the control target device is used. Thus, a unified GUI can be configured.

Conversely, when the function categories are different, even if the GUI performs the same control, the GUI shape and layout can be changed for each function category.
In other words, when controlling a plurality of electronic devices, if the function categories of the various control target devices are the same, the GUI can be configured using the same category-added UI data familiar to the user. Since the GUI can be configured using UI data of different categories so as not to confuse the devices, it is possible to provide a user-friendly GUI.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In FIG. 1, reference numeral 101 denotes a control device, and reference numeral 102 denotes a category-attached UI for holding category-attached UI data in which functional category information to which each GUI belongs is associated with UI data such as component information and layout information related to the GUI of the control target device. UI data storage means. Reference numerals 103 to 105 are control targets of the control device 101, which are device A, device B, and device C, respectively. A network 106 connects the control target devices of the control apparatus 101, device A103, device B104, and device C105. In addition, the control apparatus 101 includes an operation unit 107 that receives a user instruction for controlling each device. The operation unit 107 may be a physical button, but displays a GUI configured using different UI data for each device on a liquid crystal screen or the like so that the number of buttons can be flexibly changed depending on the device. It is preferable to be able to select with a pointing device such as a mouse. Further, each device has a function category 110 that represents a function held by each device.

  The category-added UI data stored in the category-added UI storage unit 102 of the control device 101 includes categories as shown in FIG. 2 in addition to GUI information such as size and layout. FIG. 2 is an example of a conceptual diagram showing the contents of data held in the category-added UI data storage unit 102. Examples of the GUI 201 include a category 202 of AV equipment such as a video and CD player, a DVD player, a kitchen category 203 used in a kitchen such as a microwave oven, an oven, and a refrigerator, and a bath category 204 used in bath and toilet management. Can be considered. Furthermore, in the AV category 202, a UI Play 205, a stop function Stop 206, a pause Pause 207, and the like relating to a playback function common to all devices of video, CD, and DVD are prepared. In this example, the functions are shown following each category. For example, following the AV category 202, the control target devices belonging to the AV category such as a CD player and a DVD player are held so that all the functions for all the control target devices are provided. You may make it hold | maintain.

  The data stored in the category-added UI data storage means 102 includes not only the function categories but also the design of GUI component types, sizes, colors such as buttons, sliders, and dials, and the Stop function beside the Play function. The GUI layout information is also associated. Of course, descriptions related to device control are also linked. If it is the Play function, it is a reproduction operation of the designated data, and if it is the Stop function, a data reproduction stop operation is linked. For the description of these data, for example, an XML language can be used, and exchange between devices, definition of GUI, and connection of functions can be performed.

  Further, the category-added UI data stored by the category-added UI data storage unit 102 may be stored for several types if the control device is for personal use and for the number of users used if the control device is used for sharing. Since it is only held as data, it can be increased as necessary, the entire GUI data 201 can be replaced, or the categories such as 202 and 203 can be replaced.

  On the other hand, each device holds a function category 110 as information on a category to which a function held by each device belongs. FIG. 3 shows an example in which the function category is indicated by a path. In the case of having a function category, AV 302 is set next to GUI 301, and in order to configure the GUI of a device having this function category, the UI data stored in category-equipped UI data storage unit 102 of the control device is included. From this, it can be understood that UI data with an AV category should be selected. Further, as shown in FIG. 4, a place where a GUI called Play 403 is required may be set after AV 402. In either case, the UI data corresponding to the Play button can be acquired from the UI data of the corresponding category using the category of the control target device obtained by the function category acquisition unit 108 of the control device.

  Here, regarding the function category acquisition means 108, the exchange of data with other devices on the network by the device can be used. Through this exchange, the device can notify its own function or inquire about the function from the control device side. Therefore, the corresponding category may be determined from the result. For example, when using the above-described protocol such as UPnP, the communication is performed as shown in FIG. FIG. 5 shows an example in which a device is added later. When a new device is connected, the control point of the control device is notified by Advertise (501). The control point of the advertised control device performs a search in 502 to obtain a device description that is information such as the model name and model number of the device and a service description that is information such as the type of operation to be controlled for the function of the device. Since the function category can be obtained, the function category can be obtained.

  As described above, since the function category is determined by the device, UI data corresponding to the obtained function category can be selected from the category-added UI data storage unit 102, and the GUI generation unit 109 can configure the GUI. As illustrated in FIG. 6, the category-added UI data 603 referred to by the control device 601 becomes a GUI 604 that is actually displayed by the control device by the GUI generation unit 605. Therefore, if the device 602 has the function category 606, the category to which the device belongs can be made to correspond to the category-added UI data of the control device, and the GUI for the actual device can be configured from the corresponding UI data.

Hereinafter, an example of this embodiment will be described more specifically.
(First embodiment)
In FIG. 1, a device A103 is a CD player, and a device B104 is a newly connected DVD player.
First, when only the CD player 103 and the control device 101 exist on the network, the GUI as shown in FIG. 7 is displayed on the control device 101, so that the user operates the CD player 103 from the operation unit 107 of the control device 101. can do.

  FIG. 8 is a flowchart illustrating an example of processing performed by the control device 101 when the DVD player 104 is added. First, in step 801, it is detected that a DVD player is connected. If it is not detected, the answer is No and the process waits until it is detected. However, if connection is detected, the answer is Yes and the process proceeds to Step 802. In step 802, the function category of the newly connected device is acquired using the function category acquisition means 108 having the mechanism as shown in FIG. Since the connected device can be identified as a DVD player from the function category obtained in step 802, based on the information, in step 803, it is checked whether or not the GUI of the DVD player already exists. As shown in FIG. 6, it is only necessary to check whether the GUI of the target DVD player has already been generated in the GUI 604 created by the GUI generation means 605 from the UI data 603. In the example, since the DVD player is newly connected, the GUI of the CD player is prepared in the control device 101 which is a control device for operating the CD player and the DVD player, but the GUI of the DVD player is not yet available, and the step becomes No. Go to 804. In step 804, categories are associated. Since the function category possessed by the DVD player is acquired, the control device associates it with the UI data possessed by the UI data storage means with category 108.

  Here, for example, the category data of the category-added UI data and the acquired function category data are compared and searched using character strings, and the acquired function category is used from the UI data stored by the category-added UI data storage unit 108. Any method may be used as long as desired UI data can be obtained. Since a DVD player is an AV device, a GUI having the same operational feeling as that of a CD player can be configured by using UI data in the AV category. In step 805, the GUI generating means 109 uses the AV category UI data to configure the GUI of the DVD player.

  An example of the GUI configured as described above is shown in FIG. Compared with the GUI of the CD player shown in FIG. 7, it can be seen that a unified GUI is configured. In step 806, in order to indicate to the user the presence of the control target device detected in step 801, a DVD player icon or the like is displayed according to the function category obtained in step 802, and the GUI generated in step 805 is displayed on the control apparatus 101. To finish the process.

  If the process proceeds to No in step 803, the DVD player may have been used, so that it may not be detected when something that has been removed from the network is reconnected or when the main power is off. If it is detected that the power was turned on and the GUI configured at the previous use can be reused, the process proceeds to step 806 and the last used icon and GUI are displayed and finished. To do. It may not be an icon to be displayed when the connection of the control target device is detected, a dedicated GUI is displayed at the top of the screen, a tab is displayed so as not to interfere with the currently displayed screen, or a user instruction It is only necessary to be able to control the device to be controlled even if it is not displayed until there is.

  The GUI is prepared as shown in FIG. That is, the newly connected DVD player has Play, Stop, Next, Back, VolumeUP, and VolumeDown as functions as the CD player. In addition, the CD player of this example has no function, and is not necessary. It also has a function. Since the functions listed here are general AV categories, a GUI prepared in advance in the control device 101 can be used. The same GUI that the user is currently using as the AV category GUI can be used as the GUI of the DVD player in the same manner as the CD player. The AV category GUI is held in the UI display device 101 so that a CD player or a DVD player can be displayed. The GUI display for each device may be arranged in a tab display and switched according to the user's selection, or may be automatically displayed depending on whether the device is powered on or the user is approaching the device. Also good.

  Here, there may be a case where the device also holds UI data with a category, such as when preparing UI data that can be used by a device development manufacturer from a control device. In this case, if it is not necessary to unify the GUI, a GUI dedicated to the device to be controlled may be created from the prepared UI data. However, for the purpose of the present invention, it is preferable to use the UI data on the control device side in order to unify the GUI. For example, the DVD player gives a sense of unity even if the labels shown in “” such as playback “PLAY”, stop “STOP”, advance “→”, and return “←” are given to the function. For this purpose, the label name of the category-attached UI data of the AV category held by the control device, which is also used in the CD player, is used. Even if only the DVD player has a dubbing function, a dubbing button labeled “●” is generated from the GUI data held by the UI display device 101 which is the control device in the present embodiment.

  As the UI data of the device, a part design, a size, a label name such as “Play” and the like can be given, but only necessary UI data can be used. For UI data, in addition to shape and layout information, for example, for a button, it is possible to combine only the button label and the action when the button is pressed, so only the UI data such as the label and action is used. Even with buttons of functions that are not prepared for the control device, it is possible to generate a GUI with a sense of unity by using the UI data of the control device for the shape and layout.

  For example, when the UI display device as the control device does not have the bilingual function when the DVD player is replaced with another DVD player having the bilingual playback function in the previous example, the DVD player A GUI as shown in FIG. 10 is configured by combining the operation of the bilingual function of the player and only the label “Billingual” with the UI data of the default button from the UI data with the AV category. be able to. In this case, it is not necessary for the control device to know the operation of the bilingual function, and the GUI is configured with data as shown in FIG. 12, for example. As shown in 1201, regarding the exception function, the category-equipped UI data storage unit of the control device may be held as Function A or the like. Since labels and actual functions are prepared by the device side, only buttons having a sense of unity are prepared as the Function A, and the GUI generation unit configures the GUI as shown in FIG. 10 using only the “Billingual” label from the UI data of the device.

  Here, it is considered that there are many devices having a stop function that expresses a GUI with the label “STOP” not only in the AV category described above but also in the home appliance category, for example. However, even if there is a device having a stop function, the GUI can be configured with an expression for the home appliance category as long as the device can be recognized as the home appliance category. As a result, even with the same function of stopping, GUIs with different shapes and layouts can be configured, and the proper use of GUIs by category can be made clearer.

  As described above, according to the present embodiment, it is possible to provide a unified GUI by associating and holding categories in the UI data for configuring the GUI used by the control device. In particular, in the case of a control target device that generates a common operation such as reproduction of multimedia content data, the same GUI is configured by combining the function categories of the control target device. The user can create the same GUI as that used by other controlled devices, and can provide a user familiar GUI. Further, even if there is no UI data of the control device used by the user corresponding to the function of the control target device, the GUI is automatically used by using UI data having a shape and layout suitable for the category to which the control target device belongs. By configuring, it is possible to give the GUI a sense of unity. Furthermore, since the control device side I do not know in advance what kind of device is connected, but not be prepared only a limited GUI at the time of development, the use of the category, in other GUI even GUI that were not anticipated when the development It can also be configured.

  In this embodiment, the UI data is assumed to be held by the control device or the control target device, but may be downloaded from the network. For example, a manufacturer who has released a new device can prepare data for the user.

(Second Embodiment)
In this embodiment, a GUI configuration system that constructs a category-added UI data unit in a server on a network will be described.
In FIG. 13, reference numeral 1301 denotes a control device, and 1302 denotes a server that stores UI data. Reference numerals 1303 to 1305 denote device A, device B, and device C, respectively. Reference numeral 1306 denotes a network, which connects the control device 1301, the server 1302, the device A 1303, the device B 1304, and the device C 1305. Further, the server 1302 includes a category-added UI data storage unit 1311.

  Here, it is assumed that the device C is a newly connected control target device. FIG. 14 is a flowchart showing the processing flow. First, in step 1401, it is detected that the device C is connected. Since it has been detected, the determination becomes Yes and the process proceeds to step 1402. In step 1402, the function category of the device C of the control target device newly connected is acquired using the mechanism shown in FIG. Based on the function category obtained in step 1402, it is checked in step 1403 whether the corresponding GUI already exists. If the result of step 1403 is Yes and the GUI already exists, the GUI configuration is unnecessary. Therefore, in step 1406, the device information obtained in step 102 is used to display the icon of the corresponding device, and the process ends. If No in step 1403, UI data corresponding to the category is obtained from the category-added UI data held in the server in step 1404.

  For example, as shown in FIG. 15, the server returns UI data with categories in response to a UI data request 1501 sent from the control device. Alternatively, even when a newly connected control target device holds UI data, only a part of the UI data or a function category is sent from the control device, and the server returns UI data corresponding to the sent information. The protocol is not particularly limited as long as UI data necessary for configuring a unified GUI can be obtained. By configuring the GUI in step 1405 using the UI data obtained in step 1404, the GUI of device C can be prepared.

  As described above, according to the present embodiment, in addition to the effects described in the first embodiment, there is no need to prepare a category-added UI data storage unit on the control device side. Leading to In addition, data can be easily used from a plurality of control devices, and management is also facilitated.

(Third embodiment)
In the present embodiment, a description will be given of a system that displays a GUI by dividing a taste for each category when the device to be controlled has a plurality of categories.
For example, if you have a bath hot water system that comes with a radio so that you can listen to the radio in the bathroom, it will be related to radio operations such as tuning the radio, determining the volume of the speaker for the radio, and hot water for the bath, such as the start of hot water supply and additional cooking. Operations are mixed. Therefore, the hot water supply system with a radio belongs to both the AV category 202 and the Bath category 204 in the category of FIG.

  The connection detection of the hot water supply system, the mechanism of GUI generation, and the like are the same as those in the above-described example, and are omitted. FIG. A portion indicated by 1601 is a GUI related to radio belonging to the AV category, and a portion indicated by 1602 is a GUI related to hot water supply belonging to the Bath category. Here, both the buttons 1603 and 1604 are labeled “+”, but 1603 is a plus button related to the AV volume, and 1604 is a plus button related to the temperature of the Bath. Composed. The radio GUI 1601 can display the same devices as other AV category control target devices such as CD players and DVD players, and the hot water supply GUI 1602 can display the same hot water supply GUI used in other places, so that the user is familiar with the GUI. Can be used.

  As described above, according to the present embodiment, in addition to the effects described in the first embodiment and the second embodiment, even if the function of the device is spread over a plurality of categories, the GUI that is used to each is used. Can be expressed as

  Note that the program code in the flowcharts of FIGS. 8 and 14 can be realized by operating a program stored in a RAM or ROM of a computer. This program and a computer-readable storage medium storing the program are included in the embodiment of the present invention.

  Specifically, the program is recorded on a recording medium such as a CD-ROM or provided to a computer via various transmission media. As a recording medium for recording the program, besides a CD-ROM, a floppy (registered trademark) disk, a hard disk, a magnetic tape, a magneto-optical disk, a nonvolatile memory card, or the like can be used. On the other hand, as the transmission medium of the program, a communication medium (wired line such as an optical fiber or wireless) in a computer network (LAN, WAN such as the Internet, wireless communication network, etc.) system for propagating and supplying program information as a carrier wave. Line).

  In addition, the functions of the above-described embodiments are realized by executing a program supplied by a computer, and the program is used in cooperation with an OS (operating system) or other application software running on the computer. When the functions of the above-described embodiment are realized, or when all or part of the processing of the supplied program is performed by a function expansion board or a function expansion unit of the computer, the function of the above-described embodiment is realized. Such a program is included in the embodiment of the present invention.

  For example, FIG. 17 is a schematic diagram illustrating an internal configuration of a personal user terminal device. In FIG. 17, reference numeral 1200 denotes a computer PC. The PC 1200 includes a CPU 1201 and executes device control software stored in a ROM 1202 or a hard disk (HD) 1211 or supplied from a floppy (registered trademark) disk drive (FD) 1212 and connected to the system bus 1204. Control the device as a whole.

  The program stored in the CPU 1201, the ROM 1202 or the hard disk (HD) 1211 of the PC 1200, the procedure of steps S801 to S806 in FIG. 8 of the first embodiment, the procedure of S1401 to S1406 in FIG. 14 of the second embodiment, etc. Is realized.

  A RAM 1203 functions as a main memory, work area, and the like for the CPU 1201. A keyboard controller (KBC) 1205 controls instruction input from a keyboard (KB) 1209, a device not shown, or the like.

  Reference numeral 1206 denotes a CRT controller (CRTC) which controls display on a CRT display (CRT) 1210. A disk controller (DKC) 1207 is a hard disk (boot program (start program: a program that starts execution (operation) of personal computer hardware and software)), a plurality of applications, editing files, user files, a network management program, and the like. HD) 1211 and floppy (registered trademark) disk (FD) 1212 are controlled.

  Reference numeral 1208 denotes a network interface card (NIC) that performs bidirectional data exchange with a network printer, other network devices, or other PCs via the LAN 1220.

It is a block diagram showing an example of hardware constitutions concerning this embodiment. It is a conceptual diagram which shows an example of UI category. It is a figure in case UI category is shown by a path. It is a figure in case UI category is shown by a path. It is a sequence diagram which shows an example of an apparatus control protocol. It is a block diagram which shows an example of the relationship between the function category which concerns on this embodiment, UI data with a category, GUI data, and a GUI production | generation means. It is a figure of GUI of the CD player which concerns on this embodiment. It is a flowchart which shows a series of operation examples of this embodiment. It is a figure of GUI of the DVD player which concerns on this embodiment. It is a figure of GUI of the DVD player which concerns on this embodiment. It is a figure of the GUI structure data which concerns on this embodiment. It is a figure of the GUI structure data which concerns on this embodiment. It is a block diagram showing an example of hardware constitutions concerning this embodiment. It is a sequence diagram which shows an example of exchange of UI category information. It is a flowchart which shows a series of operation examples of this embodiment. It is a figure of GUI of the hot water supply system with a radio function concerning this embodiment. It is a schematic diagram which shows the internal structure of a personal user terminal device.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Control apparatus 102 UI data preservation | save means 103-105 with a category Device 106 to be controlled Network 107 Operation part 108 Function category acquisition means 109 GUI production | generation means

Claims (5)

A control device for controlling a plurality of control target devices and the control target device operated by the control device are connected to a network, and the control device and the control target device include each other's functions via the network. A GUI configuration system capable of transmitting and receiving data,
The control device is
GUI generation means for configuring the GUI from UI data describing GUI parts and layout,
Category-added UI data storage means for classifying and holding the UI data according to the category of functions of the control target device;
A function category acquisition means for acquiring the function of the device to be controlled;
The GUI generating system generates a GUI using UI data of a category stored by the category-added UI data storing unit corresponding to the category acquired by the functional category acquiring unit. .   2. The GUI configuration system according to claim 1, wherein the category-added UI data storage unit is constructed on a server accessible from the control device.   The GUI generation unit generates a single GUI by combining the GUIs generated for each acquired function category when there are a plurality of function categories acquired by the function category acquisition unit. Item 3. The GUI configuration system according to Item 1 or 2. A function category acquisition step for acquiring the function of the control target device;
And a step of generating a GUI using UI data corresponding to the acquired category.   A program for causing a computer to execute each step of the method of claim 4.

Images