JP2023056660A - Device management device, device management system, input/output device, device management method, and program - Google Patents

Abstract

To easily select a desired device from a plurality of available devices.SOLUTION: A device management device includes a display processing part for causing a display part to display a display screen where display elements indicating each device are arranged at different positions, and changing at least one of the positions of the display elements, and a range specification figure indicating a selection region constituting a part of the display screen according to operation, and a device management part for selecting a device corresponding to the display element in which a positional relation between the range specification figure and the display element varies as an object device, and controlling an operation state of the object device, wherein the display processing part determines a display mode of the display element of the object device as a display mode corresponding to the operation state of the object device.SELECTED DRAWING: Figure 2

Description

The present invention relates to a device management apparatus, device management system, input/output device, device management method, and program. For example, the present invention relates to a method of providing a graphical user interface for selecting a device to be actually used for reproduction from reproducible devices capable of reproducing information distributed by media.

Devices capable of acquiring information distributed using media such as broadcasting and the Internet are diversifying. For example, conventionally, radio receivers for receiving radio broadcasts and television receivers for receiving television broadcasts have been widely used, but smartphones have recently spread remarkably. Also, smart watches, smart speakers, and other IoT (Internet of Things) devices capable of presenting various types of information are expected to spread. According to this trend, the diversification of devices used by viewers is expected to continue in the future.

Along with the diversification of devices, not only the presentation of information using a single device, but also the forms of presenting information by linking various devices are diversifying. For example, playing music with earphones connected wirelessly to a smartphone, or linking a television receiver connected to a home network (LAN: Local Area Network) with a smartphone, and playing videos on the smartphone to be played on a television receiver from the middle.

Therefore, a technique for selecting a device to be used for presenting information from various devices is expected. For example, the information processing apparatus described in Patent Document 1 grasps the user's privacy level based on the information about the user, and based on the privacy level of the preset output form, the preset output form is not suitable. If so, the optimum output form is selected from among the plurality of output forms based on the privacy level of the user and the privacy levels corresponding to the plurality of output forms.
The control method described in Patent Document 2 includes a step of performing control corresponding to a second predetermined operation related to control of a smart hardware device when a first predetermined operation is performed on a control key on a screen lock screen of a terminal device. include.

The information processing apparatus described in Patent Document 3 performs a process of guiding the user's line of sight to the location of the device to be controlled, and a process of controlling the execution timing of the operation command in the device to be controlled according to the timing of guiding the user's line of sight. conduct.
In the control method described in Patent Document 4, in the step of adding a chat room with a bot for interlocking the user and the IoT server to the messenger, the control signal of the IoT device is transmitted to the IoT device according to the control input entered by the user in the chat room. It includes controlling the IoT device by transmitting to the server.

JP 2016-91221 A Japanese Patent Publication No. 2017-509277 Japanese Unexamined Patent Application Publication No. 2019-113928 JP 2020-61131 A

However, according to the above method, it is not considered to grasp the entirety of devices that can be used by the user, and only a predetermined device is selected according to objective conditions. Therefore, the device desired by the user is not always used for presenting information.

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and aims to provide a device management apparatus, a device management system, an input/output device, a device management method, and a program that enable easy selection of a desired device. aim.

The present invention was made to solve the above problems,
[1] According to one aspect of the present invention, a display screen in which display elements indicating individual devices are arranged at different positions is displayed on a display unit, and the positions of the display elements and a selection range forming a part of the display screen are displayed. A display processing unit that changes at least one of the range-specifying figures indicating , according to an operation, and a device corresponding to a display element whose positional relationship with the range-specifying figure has changed as a target device, and the target device a device management unit that controls an operating state, wherein the display processing unit determines a display mode of display elements of the target device to a display mode corresponding to the operating state of the target device.
According to the configuration [1], the operating state of the target device can be controlled by changing the positional relationship between the display element indicating the target device and the range designating figure by an operation, and the display elements can be displayed in a display mode corresponding to the operating state. is displayed. The user can easily select the target device whose operating state is to be controlled, and can grasp the operating state of the device while looking at the display screen.

[2] An aspect of the present invention is the device management system described above, in which a user display element, which is a display element indicating a user, is located at a different position from a device display element, which is a display element indicating the device, on the display screen. , the range specifying graphic is displayed in an area including the user display element, and the device management unit corresponds to a device display element entering the selection range or a device display element exiting the selection range. A device to be used may be selected as the target device.
According to the configuration [2], by operating the user display element to enter or leave the range designation graphic, the operating state of the device corresponding to the user display element is intuitively controlled. can be done.

[3] An aspect of the present invention is the device management system described above, wherein the display processing unit displays the range designation graphic in the selection range instructed in response to an operation, and the device management unit displays the selection range. Devices corresponding to display elements included in the range may be selected as target devices.
According to the configuration [3], it is possible to intuitively select a device whose operating state is to be controlled from the selection range defined by the operation using the range designation graphic.

[4] An aspect of the present invention is the device management system described above, wherein the display processing unit indicates the type of information that can be presented for each device when the type of information is instructed according to an operation. A device capable of referring to data and presenting the indicated type of information is specified as a presentable device, and a display element of the presentable device is displayed in a different display mode from a display element of a device separate from the presentable device. may be displayed.
According to the configuration [4], by specifying the type of information that can be presented, a device that can present the specified type of information is selected. Therefore, by designating the type of information, it is possible to narrow down the devices that can be selected and to easily distinguish them from other devices.

[5] An aspect of the present invention is the device management system described above, wherein the display processing unit refers to device management data indicating the installation location of each device when the installation location is instructed in response to an operation. , a device that can be used at the designated location is specified as a usable device, and the display elements of the usable device are displayed in a different display mode from the display elements of the device that is separate from the usable device. good.
According to the configuration [5], by specifying the installation location, a device that can be used at the specified installation location is selected. Therefore, by designating the installation location, it is possible to narrow down the devices that can be selected and to easily distinguish them from other devices.

[6] An aspect of the present invention is the device management system described above, wherein the display processing unit refers to device management data indicating other connectable devices for each device, and for each device, , a figure indicating the possibility of connection with the other device is displayed between the display elements respectively corresponding to the one device and the other device.
According to the configuration [6], the user can easily understand that the devices corresponding to the display elements shown at both ends of the figure indicating the connectability can be connected to each other.

[7] An aspect of the present invention is a device management apparatus that causes a display unit to display a display screen in which display elements indicating individual devices are arranged at different positions, and displays the positions of the display elements and the display screen. The target device is a display processing unit that changes at least one of the range designation figures indicating the selection range forming part of the a device management unit that selects and controls an operating state of the target device, wherein the display processing unit determines a display mode of display elements of the target device to a display mode corresponding to the operating state of the target device It is a management device.
According to the configuration [7], the operating state of the target device can be controlled by changing the positional relationship between the display element indicating the target device and the range designating figure by an operation, and the display element can be displayed in a display mode corresponding to the operating state. is displayed. The user can immediately select the target device whose operating state is to be controlled, and can grasp the operating state of the device while looking at the display screen.

[8] An aspect of the present invention is an input/output device that causes a display unit to display a display screen on which display elements indicating individual devices are arranged at different positions, and a display processing unit that changes, according to an operation, at least one of range designation figures indicating a selection range forming a part of the display element whose positional relationship with the range designation figure has changed may be selected as the target device, and the display mode of the display elements of the target device may be set to a display mode corresponding to the operating state of the target device.
According to the configuration [8], the detected operating device becomes a candidate for the providing device for providing the distribution information, and the providing device is excluded from the undetected devices. Therefore, it is possible to efficiently select a providing device from devices in operation, and to avoid unnecessary processing for providing distribution information to devices that are not detected.

[9] One aspect of the present invention may be a program for causing a computer to function as the device management device of [7] or the input/output device of [8].
According to the configuration [9], the operating state of the target device can be controlled by changing the positional relationship between the display element indicating the target device and the range designating figure by an operation, and the display element can be displayed in a display mode corresponding to the operating state. is displayed. The user can immediately select the target device whose operating state is to be controlled, and can grasp the operating state of the device while looking at the display screen.

[10] One aspect of the present invention is a first step in which a display processing unit causes a display unit to display a display screen in which display elements indicating individual devices are arranged at different positions; a second step of changing, according to an operation, at least one of the position of a display element and a range designation figure indicating a selection range forming a part of the display screen; a third step of selecting, as a target device, a device corresponding to the display element of which the change has occurred, and controlling the operating state of the target device; and a fourth step of defining a display mode corresponding to the operating state of the target device.
According to the configuration [10], the operating state of the target device can be controlled by changing the positional relationship between the display element indicating the target device and the range designating figure by an operation, and the display element can be displayed in a display mode corresponding to the operating state. is displayed. The user can immediately select the target device whose operating state is to be controlled, and can grasp the operating state of the device while looking at the display screen.

According to the present invention, a desired device can be easily selected from a plurality of available devices.

1 is an explanatory diagram showing an outline of a device management system according to a first embodiment; FIG. 1 is a schematic block diagram showing a functional configuration example of a device management system according to a first embodiment; FIG. FIG. 4 is a diagram showing a first display example of a display screen according to the first embodiment; FIG. 10 is a diagram showing a second display example of the display screen according to the first embodiment; 4 is a diagram showing a first example of device management data according to the first embodiment; FIG. 6 shows an example of user selection range data indicating the display range of the range designation graphic according to the first embodiment; FIG. 7 is a diagram showing a second example of device management data according to the first embodiment; FIG. 10 is a diagram showing a third example of device management data according to the first embodiment; FIG. 4 is a sequence diagram showing an example of device selection processing according to the first embodiment; FIG. FIG. 11 is an explanatory diagram showing an overview of a device management system according to a second embodiment; FIG. FIG. 11 is a schematic block diagram showing an example functional configuration of a device management system according to a second embodiment; FIG. 10 is a diagram showing a first example of device management data according to the second embodiment; FIG. FIG. 10 is a diagram showing an example of user selection range data according to the second embodiment; FIG. FIG. 10 is a diagram showing a first display example of a display screen according to the second embodiment; FIG. 11 is a diagram showing a second display example of the display screen according to the second embodiment; FIG. 13 is a diagram showing a third display example of the display screen according to the second embodiment; FIG. 14 is a diagram showing a fourth display example of the display screen according to the second embodiment; FIG. 15 is a diagram showing a fifth display example of the display screen according to the second embodiment; FIG. 14 is a diagram showing a sixth display example of the display screen according to the second embodiment; FIG. 12 is a diagram showing a seventh display example of the display screen according to the second embodiment; FIG. 14 is a diagram showing an eighth display example of the display screen according to the second embodiment; FIG. 10 is a diagram showing a second example of device management data according to the second embodiment; FIG. FIG. 22 is a diagram showing a ninth display example of the display screen according to the second embodiment; FIG. 13 is a diagram showing a third example of device management data according to the second embodiment; FIG. FIG. 11 is a sequence diagram showing a first example of device selection processing according to the second embodiment; FIG. 11 is a sequence diagram showing a second example of device selection processing according to the second embodiment; FIG. 11 is a sequence diagram showing a third example of device selection processing according to the second embodiment; FIG. 14 is a sequence diagram showing a fourth example of device selection processing according to the second embodiment; FIG. 11 is a sequence diagram showing an example of device deselection processing according to the second embodiment;

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
<First Embodiment>
FIG. 1 is an explanatory diagram showing an outline of a device management system 1 according to the first embodiment.
The device management system 1 includes a control device 100 and an input/output device 200 . In the illustrated example, the device management system 1 is installed in an indoor living space. Here, the IoT gateway corresponds to the control device 100 and the tablet PC (Personal Computer) corresponds to the input/output device 200 .

The control device 100 detects devices installed in a predetermined living space, and manages the functions and operating states of the detected devices. The control device 100 outputs the detected operating states of the devices to the input/output device 200 . A device to be detected is a device having a function of presenting distribution information distributed to itself. In the illustrated example, in addition to tablet PCs, smartphones, smart speakers, and TVs (Television sets, television receivers) can be detection targets. In this application, a device having a function of presenting distribution information may be called a "presentation device". The input/output device 200 may also have a configuration for presenting information as a presentation device. The number of presentation devices can be two or more.

The input/output device 200 receives the operating state of the device from the control device 100 . The input/output device 200 generates a display screen in which device icons are arranged at different positions as examples of display elements indicating individual devices, and causes the display unit to display the generated display screen. In this embodiment, on the display screen, a user icon indicating the user of the input/output device 200 is arranged at a position different from that of the device icon, and the range designation figure is arranged at a position where the region includes the user icon. In the example of FIG. 1, the device icons of the smart speaker, the tablet PC (the input/output device 200 itself), the TV, and the smartphone are represented by circles. The user icon is a circle mark placed near the character string "you".

In the input/output device 200, the circle centered on the user icon corresponds to a range designation figure indicating a selection range for selecting device icons. The input/output device 200 accepts a user's operation and enables the position of each device icon to be changed according to the operation. The input/output device 200 notifies the control device 100 of operation information indicating the position of the designated device icon.

The control device 100 selects, as a target device, a device corresponding to the device icon whose positional relationship with the range specifying graphic has changed, and controls the operating state of the target device. The control device 100 selects a device corresponding to a device icon that enters the selection range from outside the selection range as a target device, and causes the selected target device to start presenting distribution information. The input/output device 200 displays the device icon of the target device to which the distribution information is presented in a darker color than before the distribution information is presented. The input/output device 200 selects, as a target device, a device corresponding to a device icon that exits from within the selection range to outside the selection range. The control device 100 selects a device corresponding to a device icon that moves from within the selection range to outside the selection range as a target device, and stops presenting the distribution information to the selected target device. The input/output device 200 displays the device icon of the target device whose presentation of the distribution information is stopped brighter than before the stop. The illustrated example shows a state in which the TV and the smartphone are activated, and the operation of the smart speaker and the tablet PC (information presentation function) is stopped. Thereby, the user can immediately check the presentation state of the distribution information by each device, and can adjust the presentation state by a simple operation. In the present application, a device having a function of presenting distributed information may be called a "presentation device". In the example shown in FIG. 1, smart speakers and TVs correspond to presentation devices. The input/output device 200 (smartphone) also functions as a presentation device.

Next, a functional configuration example of the device management system 1 according to this embodiment will be described. FIG. 2 is a schematic block diagram showing a functional configuration example of the device management system 1 according to this embodiment. The device management system 1 includes a control device 100 and an input/output device 200 . The control device 100 and the input/output device 200 are connected via a network NW so as to be able to input and output various information. The network NW may be either wireless or wired. The network NW is, for example, a LAN capable of wirelessly transmitting information using a predetermined communication method such as IEEE802.11 or IEEE802.15. A device separate from the control device 100 and the input/output device 200, such as the presentation device 300, can also be connected to the network NW. The number of presentation devices 300 is not limited to one, but usually two or more.

The control device 100 detects a presentation device in a predetermined management area and controls the operating state of the detected presentation device. The control device 100 may be a dedicated device having device management as its main function, or may be a device having other functions as its main function. The control device 100 may be configured as other types of gateway devices (e.g., router devices), meters (e.g., power meters, gas meters, etc.), etc., and presentation devices (e.g., television receivers, recording device), and may be managed by one or both of the device itself and the input/output device 200 . In the example shown in FIG. 1, the management area is a residence in which one or more persons reside, and the resident is the user who receives the distribution information and can be the user of the input/output device 200 and the control device 100 . One management area does not necessarily correspond to one building. A managed area may be, for example, one or more dwelling units that are part of an apartment complex.

The control device 100 includes a storage section 122 , a device management section 124 and a communication section 126 .
The storage unit 122 temporarily or permanently stores various data. The storage unit 122 includes storage media such as RAM (Random Access Memory) and ROM (Read Only Memory). The stored data includes, for example, device management data and user selection range data. Device management data includes various types of information for each presentation device. Device management data may include presentable information and operating state information for each presentation device. Device management data may include detection information for each presentation device. The user selection range data is data indicating a range designation figure for the user to select an icon for presenting information. A specific example of device management data will be described later.

The device management unit 124 manages information on presentation devices located within the management area. The device management unit 124 may detect an energized presentation device connected to its own device at predetermined detection intervals (for example, several seconds to several minutes). When detecting a presentation device, the device management unit 124, for example, causes the communication unit 126 to broadcast a notification signal (beacon) to the network NW connected to itself according to a predetermined communication method. When the device management unit 124 receives a response signal to the notification signal from any of the presentation devices within a predetermined detection time after transmission of the notification signal, the device management unit 124 determines detection of the presentation device that is the transmission source of the response signal. can be done. When the presentation device does not receive a response signal to the notification signal or receives a negative response signal to the notification signal, it can be determined that the presentation device is not detected.
The device management unit 124 can extract the identification information of the presentation device from the response signal, and use the extracted identification information to specify the presentation device that is the transmission source of the response signal. Even if it is installed in the management area, a managed device that is not receiving power and has stopped operating is not detected. Therefore, the managed devices in operation may fluctuate from time to time.

The device management unit 124 associates the detected device information indicating the detection of the newly specified device with the presented device and stores it in the storage unit 122 . This updates the device management data. The device management unit 124 determines that the presentation device is not detected when the presentation device whose detection information is stored in the storage unit 122 is not transmitted by the input detected device information. The device management unit 124 stores detection information indicating non-detection of the presentation device in association with the presentation device. This updates the device management data.

The device management unit 124 monitors the operation information received from the input/output device 200 using the communication unit 126, and monitors the position of the device icon indicated by the operation information and the selection range defined by the predetermined range selection graphic. Determine the positional relationship. The device management unit 124 identifies a device icon that has entered the selection range from outside the selection range, and causes the presentation device corresponding to the identified device icon to start presenting information. The device management unit 124 uses the communication unit 126 to transmit a control signal indicating the start of information presentation to the presentation device. When an acknowledgment to the control signal is input within a predetermined time after transmission of the control signal, the device management unit 124 associates the operation state information indicating that information is presented for the presentation device with the presentation device, and stores the information in the storage unit 122. memorize to The device management unit 124 transmits device management data including the updated operating state information to the input/output device 200 using the communication unit 126 .

The device management unit 124 identifies a device icon that has moved out of the selection range from within the selection range, and causes the presentation device corresponding to the identified device icon to stop presenting information. The device management unit 124 uses the communication unit 126 to transmit a control signal indicating information presentation stop to the presentation device. When an acknowledgment (ACK) for the control signal is input within a predetermined time from the transmission of the control signal, the operating state information indicating that the presentation device does not present information is stored in the storage unit 122 in association with the presentation device. do. The device management unit 124 transmits device management data including the updated operating state information to the input/output device 200 using the communication unit 126 .
Note that when an acknowledgment for the control signal is not input within a predetermined time after the transmission of the control signal, or when a negative acknowledgment (NACK) for the control signal is input, the device management unit 124 updates the device management data. Do not update. This is because it is presumed that the change of the operating state instructed by the control signal has failed.

The communication unit 126 enables wireless or wired communication with other devices using a predetermined communication method. The communication unit 126 includes, for example, a communication interface. The input/output device 200 and the presentation device 300 are included as other devices that are communication partners.

The input/output device 200 displays a display screen on which device icons of each presentation device are arranged, and receives a user's operation to indicate the operating state of any one of the presentation devices. The input/output device 200 may be any form of device as long as it can visibly display information and accept a user's operation. The input/output device 200 is not limited to an information communication device such as a general-purpose tablet PC or smart phone, and may be a dedicated controller (remote control).

The input/output device 200 includes a storage unit 222 , a communication unit 226 , a display processing unit 228 , a display unit 234 and an operation unit 236 .
The storage unit 222 stores various data. The storage unit 222 includes storage media such as RAM and ROM. The data stored in the storage unit 222 will be described together with the functions of the display processing unit 228. FIG.
The communication unit 226 enables wireless or wired communication with another device using a predetermined communication method. The communication unit 226 includes, for example, a communication interface. The control device 100 is included in other devices that are communication partners.

The display processing unit 228 generates a display screen and causes the display unit 234 to display the generated display screen. A range specifying figure is arranged on the display screen. A user icon indicating the user is arranged at a representative point (for example, center, center of gravity, etc.) of the range specifying graphic. The display range of the range designating figure is set as a selection range, device icons of presentation devices that present information are displayed within the selection range, and device icons of presentation devices that do not present information are displayed outside the selection range. The display processing unit 228 refers to the device management data received from the control device 100, identifies the presentation device detected by the control device 100, and determines the identified presentation device as a device for which a device icon is to be arranged (FIG. 3). , see FIG. 5). The display processing unit 228 describes, in the device management data, the coordinates indicating the display position of the user icon on the display screen for each presentation device.

The display processing unit 228 may randomly determine the coordinates of the initial display position of the device icon within or outside the display range using pseudorandom numbers, or may use predetermined coordinate values. The display processing unit 228 may refer to the latest device management data stored in the storage unit 222 to identify the coordinates of the initial display position of the device icon of the presentation device. The display processing unit 228 may set the display position of the presentation device closer to the user icon as the installation location of the presentation device is closer to the input/output device 200 . The display processing unit 228 may refer to the device management data (FIG. 12) to identify the installation location of each presentation device.

The display processing unit 228 changes the display position of the device icon indicated according to a predetermined mode of operation (for example, drag operation). A drag operation is an operation for spatially continuously moving a pressed position while pressing on the display screen. In this application, "pressing" may mean not only actually pressing a certain area or position, but also inputting operation information indicating that area or position. The display processing unit 228 sequentially updates the device icon indicated by the operation information input from the operation unit 236 to the position on the display screen indicated by the operation information. In other words, the device icon (sometimes referred to as “operation icon”) and the display position of the presentation device to be operated are indicated by the operation information. The display processing unit 228 transmits the input operation information to the control device 100 using the communication unit 226 .

The display processing unit 228 monitors whether the device management data received from the control device 100 using the communication unit 226 is updated. The display processing unit 228 can detect the addition or removal of the presentation device by comparing the latest device management data and the immediately preceding device management data. The display processing unit 228 adds a device icon indicating the added presentation device to the display screen. The display processing unit 228 erases the device icon indicating the removed presentation device from the display screen.

The display processing unit 228 can detect a change in the operating state of the maintained presentation device by comparing the latest device management data and the immediately preceding device management data. The display processing unit 228 may change the display mode of the device icon of the presentation device whose operating state has changed from no information presentation to information presentation to a display mode corresponding to information presentation. Any one or a combination of color, brightness, shape, pattern, etc. is used as a display mode. For a presentation device whose operating state does not change to information presentation even though the display position of the device icon has entered the selection range from outside the selection range, the display processing unit 228 moves the display position of the device icon to the selection range. May be returned to outside position. This is because it is presumed that the change in operating state for such a presentation device has failed.

The display processing unit 228 may change the display mode of the device icon of the presentation device whose operating state has changed from information presentation to no information presentation to a display mode corresponding to no information presentation. For a presentation device whose operating state does not change without information presentation even though the display position of the device icon has moved from within the selection range to outside the selection range, the display processing unit 228 moves the display position of the device icon to the selection range. You can put it back inside.

The display processing unit 228 displays the display screen reflecting the processing each time the device icon is changed, that is, added, deleted, changed in position, or changed in display mode, or any combination thereof is performed. The display data shown is output to the display unit 234 . The display unit 234 displays a display screen that reflects changes in device icons according to operations, detection states of presentation devices, operating states, and the like.

The display unit 234 displays various display information based on display data input from the display processing unit 228 . The display unit 234 may be, for example, a liquid crystal display, an organic electroluminescence display, or the like.
The operation unit 236 receives a user's operation, generates operation information indicating information instructed by the received operation, and outputs the generated operation information to the display processing unit 228 . The operation unit 236 may have general-purpose members such as a touch sensor, mouse, slider, and joystick, or may have dedicated members such as buttons and knobs. The touch sensor functioning as the operation unit 236 and the display functioning as the display unit 234 may be integrally overlapped to form a touch panel.

The presentation device 300 includes a communication section 326 , a presentation information processing section 332 and an information presentation section 334 .
The communication unit 326 enables wireless or wired communication with other devices using a predetermined communication method. The communication unit 326 includes, for example, a communication interface. The control device 100 is included in other devices that are communication partners. When receiving a detection signal (beacon) from the control device 100, the communication unit 326 transmits a response signal to the control device 100 as a response to the detection signal. The communication unit 326 can extract the identification information of the control device 100 from the detection signal and specify the control device 100 that is the transmission source based on the extracted identification information.

The communication unit 326 may have a function as an example of a distribution information acquisition unit that receives distribution information to be presented via a communication transmission line or a broadcast transmission line. A communication transmission line is a transmission line that enables both transmission and reception of various data by the presentation device. The communication transmission line includes the Internet, public network, LAN, leased line, etc., or a combination thereof. A broadcast channel is a channel that allows the presentation device to receive data, but not to transmit it. The communication unit 326 outputs the acquired distribution information to the presentation information processing unit 332 . The presentation device 300 may include a delivery information acquisition section separately from the communication section 326 . In that case, the function of acquiring distribution information may be omitted from the communication unit 326 .

The presentation information processing unit 332 executes processing related to presentation of distribution information. The presentation information processing unit 332 monitors control signals input from the control device 100 using the communication unit 326 . The presentation information processing unit 332 starts presenting the distribution information when the input control signal indicates the start of information presentation. At this time, the presentation information processing unit 332 starts receiving the distribution information instructed by the control signal using the communication unit 326 . The presentation information processing unit 332 generates presentation data for presenting distribution information. The processing for generating presentation data can depend on the type of distribution information and the encoding scheme used to encode the distribution information. The presentation information processing section 332 outputs the generated presentation data to the information presentation section 334 .
The presentation information processing section 332 transmits an acknowledgment to the control device 100 when the acquisition of the distribution information and the output of the presentation data are successful. On the other hand, the presentation information processing section 332 transmits a negative acknowledgment to the control device 100 when acquisition of distribution information or output of presentation data fails.

When the distribution information is video or auxiliary video, the presentation information processing unit 332 generates video data as presentation data using a video decoding method corresponding to the video encoding method used for encoding the distribution information. When the distribution information is audio or auxiliary audio, the presentation information processing unit 332 generates audio data as presentation data using a decoding method corresponding to the audio encoding method used for encoding the distribution information. When the distribution information is notification information, the presentation information processing section 332 generates, as presentation data, a driving signal according to the notification pattern indicating the distribution information. Notification information is information represented by an information transmission medium other than audio or video. Such information transmission media include light emission, vibration, warning sound, and the like. Device notification information is typically information that conveys non-verbal information such as cautions, warnings, and cues. The notification pattern is represented by presence/absence of presentation, or a time series consisting of repetition of the presence/absence of presentation at predetermined time intervals, or the like. The drive signal for presenting the notification information may use the power supplied to the members involved in presenting the information. Drive signals for light emission, vibration, and warning sound are represented by presence/absence or repetition of power supply to light emitters (eg, light emitting diodes), vibrators, and sounding bodies (eg, speakers).

Auxiliary video is a video that shows content (for example, accompanying, complementary, or alternative content) related to the main video (hereinafter referred to as "main video" and may be distinguished from the auxiliary video). . For example, when the main video is a drama video. Characters, commentary videos or texts of historical backgrounds, subtitles in other languages, etc. can be supplementary images. Auxiliary audio refers to audio that indicates content (for example, incidental, complementary, or alternative content) related to the main audio (hereinafter referred to as "main audio" and may be distinguished from auxiliary audio). is. For example, if the main audio is music, the auxiliary audio can be a specific instrument, singing a specific part, commentary for the visually impaired, commentary in other languages, or the like.

Note that the presentation information processing unit 332 stops presenting the distribution information when the input control signal indicates to stop presenting the information. At this time, the presentation information processing section 332 stops acquiring the distribution information and outputting it to the information presentation section 334 .
The presentation information processing unit 332 transmits an acknowledgment to the control device 100 when it succeeds in acquiring the distribution information or stopping the output of the presentation data. On the other hand, presentation information processing section 332 transmits a negative acknowledgment to control device 100 when acquisition of distribution information and stop of presentation data output fail.

The information presentation section 334 presents presentation information based on the presentation data input from the presentation information processing section 332 . The information presentation unit 334 includes a member capable of presenting one type or multiple types of information. The information presentation unit 334 includes, for example, a display for displaying video or auxiliary video based on video data as part or all of the presentation data. The information presentation unit 334 may include a speaker for reproducing sound or auxiliary sound based on audio data as part or all of the presentation data. The information presenting unit 334 may include a light emitter, a vibrator, a speaker, or a combination thereof for presenting notification information in response to a drive signal.

Note that when the input/output device 200 has a function as a presentation device, it is provided with components corresponding to the presentation information processing section 332 and the information presentation section 334 . For example, the tablet PC used as the input/output device 200 may include a tuner capable of receiving presentation information distributed by television broadcasting. The tablet PC may include a light emitter and/or a vibrator for presenting notification information in conjunction with or in place of the tuner.

Next, display examples of the display screen displayed on the display unit 234 of the input/output device 200 will be described. FIG. 3 is a diagram showing a first display example of the display screen according to this embodiment. In the example of FIG. 3, four device icons and one user icon are arranged at different positions. Characters indicating the name of the corresponding presentation device (for example, "smart speaker") are displayed near each device icon. The letters "you" are displayed near the user icon. Display of these characters may be omitted. A circle centered at the position of the user icon is an example of a range designation figure that occupies the selection range. The inside of the range specifying figure is painted with bright gradation as a mode different from the outside.

In the example of FIG. 3, the TV is the information presenting device that is presenting information (for example, video and audio of a broadcast program), and the other devices are not presenting information. The TV device icon is painted in a darker gradation as a display mode different from other device icons. Also, the user icon is painted with a darker gradation than its surroundings. As a result, it can be confirmed that the TV corresponding to the device icon displayed within the display area of the range designation figure displayed based on the position of the user icon is presenting information, and the other devices are not presenting information. It is possible to make the user understand immediately. In the following description, a display mode corresponding to the presentation of presentation information, such as a device icon of TV, may be referred to as a display during presentation. The display during presentation corresponds to active display indicating that presentation information is being presented.

FIG. 5 is a diagram showing an example of device management data according to this embodiment. The device management data illustrated in FIG. 5 indicates the operating state of the presentation device displayed on the display screen illustrated in FIG. 3 and the display position of the device icon or user icon. The device management data includes and associates the type, presentable information, detection information, operating state information, and display position for each presentation device. In the presentation device item, characters to be displayed accompanying individual presentation icons and user icons are described. The type indicates whether the type of information described in the presentation device item is presentation device or user. The item of presentable information indicates types of presentation information that can be presented. The detection information indicates whether or not the device is detected by the control device 100 . Circles indicate detected devices. Although not shown in FIG. 5, X marks indicate devices that have not been detected. The operating state information indicates whether or not information is presented. The ○ mark indicates that information is provided. The x mark indicates no information presentation. The display position indicates two-dimensional coordinates for displaying the device icon or user icon on the display screen. In addition, - mark indicates not applicable.

FIG. 6 shows an example of user selection range data indicating the display range of the range designation graphic according to this embodiment. In the example of FIG. 6, the two-dimensional coordinates of the center of the circle and the radius of the circle used as the range designation figure are shown. Note that the shape of the range specifying figure is not necessarily limited to a circle, and may be any size and shape that can be displayed within the display range so that a plurality of device icons and a single user icon do not overlap. is.

Here, it is assumed that a device icon representing a smart phone is moved into a selection range of a figure by dragging the display screen illustrated in FIG. 3 . FIG. 5 is an example of a display screen displayed in that case. In the example of FIG. 5, in addition to the TV, the smart phone also provides information presentation. The display mode of the device icon of the smart phone is also displayed during presentation, and is filled with a darker gradation than the device icons of the tablet PC and the smart speaker that are not presenting information.
The device management data illustrated in FIG. 7 indicates the operating state information of the presentation device, the display position of the device icon, and the display position of the user device displayed on the display screen illustrated in FIG. That is, it is different from the example in FIG. 5 in that a circle is described in the item of the operating state information of the smartphone, and coordinates of the device icon after movement are described in the item of the display position.

The data format of the device management data illustrated in FIGS. 5 and 7 is a table format, but is not limited to this. Any format that expresses the relationship between information elements for each presentation device is applicable. For example, the data format of the device management data may be the RDF (Resource Description Framework) format, as illustrated in FIG. RDF is one of the W3C (World Wide Web Consortium) recommendations, and in RDF, relationships between resources that are information elements are expressed by combinations of three elements (triples): subject, predicate, and object. . The subject corresponds to the resource to be described. A predicate indicates the characteristics of a subject or the relationship between a subject and an object. The object indicates what is related to the subject or the value of the predicate. A set of triples can be represented with nodes that indicate the subject, nodes that indicate the object, and edges (arrows) that indicate the relationship between the subject and the object.

The device management data illustrated in FIG. 8 shows substantially the same information as the tabular device management data illustrated in FIG. More specifically, each node of the presentation devices ``smart speaker'', ``tablet PC'', ``TV'', and ``smartphone'' represents the coordinates of each display position and the corresponding option of the presentable information `` "video", "audio", and "notification", and any of the corresponding options of the operating status information, ie, "o" or "x". The user "you" node is associated with display position coordinates, center coordinates, and the appropriate choice of type "user". A selection shape (user selection) node is associated with a center coordinate and a radius. According to RDF, by associating different element information, which is the subject, with element information, which is the predicate, the relationship between the element information is expressed just enough.

Next, an example of device selection processing in the device management system 1 according to this embodiment will be described. FIG. 9 is a sequence diagram showing an example of device selection processing according to this embodiment. However, as exemplified in FIGS. 3 and 4, it is assumed that the device icon of the presentation device that is not presenting information is moved into the selection range of the range designation figure by a drag operation.
(Step S102) The device management unit 124 of the control device 100 detects a presentation device in operation connected to its own device.
(Step S104) The device management unit 124 stores detection information indicating the detection of the newly detected presentation device in the storage unit 122 of its own device in association with the presentation device. The device management unit 124 transmits detection information to the input/output device 200 . The storage unit 222 of the input/output device 200 stores the detection information received from the control device 100 in association with the presentation device. This updates the device management data.

(Step S<b>106 ) The display processing unit 228 of the input/output device 200 reads the updated device management data and user selection range data from the storage unit 222 .
(Step S108) The display processing unit 228 identifies the detected presentation device based on the device management data, arranges the device icon at a different position for each presentation device, and places the user icon at the position indicated by the device management data. Generates a display screen that represents the selection shape.
(Step S<b>110 ) The display processing unit 228 outputs display data indicating the generated display screen to the display unit 234 . A display screen based on the display data input from the display processing unit 228 is displayed on the display unit 234 .

(Step S<b>114 ) The operation unit 236 receives the user's drag operation, and outputs operation information indicating the position indicated by the drag operation to the display processing unit 228 . The display processing unit 228 identifies the display position and the device icon after the operation indicated by the position indicated by the operation information input from the operation unit 236 as the operation icon.
(Step S116) The display processing unit 238 outputs to the control device 100 operation information indicating the coordinates of the display position of the specified operation icon.
(Step S118) The display processing unit 238 reflects the drag operation on the display screen by updating so that the operation icon is moved to the specified display position.
(Step S<b>120 ) The display processing unit 238 outputs display data indicating the updated display screen to the display unit 234 . The display unit 234 displays the updated display screen.

(Step S122) Based on the operation information input from the input/output device 200, the device management unit 124 of the control device 100 refers to the user selection range data and displays the operation icon moved from outside the selection range to within the selection range. To detect. The device management unit 124 transmits a control signal indicating the start of information presentation to the presentation device corresponding to the detected operation icon (presentation device control).
(Step S124) When an acknowledgment to the control signal is input, the device management unit 124 displays operation state information indicating that information is being presented and a display position after movement, which is operation state information related to the presentation device. The location information is associated with the specified presentation device and stored in the storage unit 222 . This updates the device management data.
(Step S<b>126 ) The device management section 124 outputs the updated device management data to the input/output device 200 .

(Step S<b>128 ) Device management data is input from the control device 100 to the display processing unit 228 of the input/output device 200 , and the input device management data is stored in the storage unit 222 . The display processing unit 228 identifies the presentation device corresponding to the identified operation icon, refers to the device management data, and identifies the operating state information of the identified presentation device. The display processing unit 228 reflects, on the display screen, the display mode of the operation icon corresponding to the specified presentation device, which corresponds to the operating state indicated by the specified operating state information, which corresponds to the presence of information presentation.
(Step S<b>130 ) The display processing unit 228 outputs display data indicating the updated display screen to the display unit 234 . The display unit 234 displays a display screen showing the operation icon after movement whose display mode has been changed.

As described above, the device management system 1 according to this embodiment includes the display processing section 228 and the device management section 124 . The display processing unit 228 displays a display screen in which device icons as display elements indicating individual presentation devices and user icons as display elements indicating users are arranged at different positions, and a range designation figure is arranged in an area including the user icons. 234 to change the position of the device icon according to the operation. The device management unit 124 selects, as a target device, a device corresponding to a device icon that enters the selection range occupying the range designation graphic or a device icon that exits the selection range, and controls the operating state of the target device. The display processing unit 228 sets the display mode of the device icon of the target device to a display mode corresponding to the operating state of the target device.
According to this configuration, the user can intuitively control the operating state of the device corresponding to the device icon by operating the device icon to enter or leave the range specifying graphic.

<Second embodiment>
Next, a second embodiment will be described. The following description mainly focuses on the differences from the first embodiment, and unless otherwise specified, the same reference numerals are given to the processing and configuration common to those of the first embodiment, and the description thereof is used. FIG. 10 is an explanatory diagram showing an overview of the device management system 1 according to the second embodiment. A notebook PC (which may be referred to as a “notebook PC” in this application) corresponds to the input/output device 200 . Illustration of devices corresponding to the control device 100 is omitted.

In the example of FIG. 10, the management area is divided into a plurality of installation locations (rooms), and the location of the presentation device is managed for each installation location. As presentation devices, a bedroom TV and an IoT light are installed in the bedroom, and a TV, smart speaker, additional speaker, headset, smart phone and notebook PC are installed in the living room. The display screen displayed on the input/output device 200 displays a device icon for each presentation device and four edges (also called "links"). A device icon is coupled to each end of each edge. Each edge indicates that presentation information can be input or output between presentation devices indicated by two device icons that are combined. In the example of FIG. 10, a TV and an additional speaker, a TV and a headset, a TV and a smart phone, and a smart phone and a headset can be connected. This differs from the first embodiment in that the user icon and range selection graphic are not displayed on the display screen.

In this embodiment, the display processing unit 228 of the input/output device 200 displays the range designation graphic in the selection range instructed according to the operation. The device management unit 124 of the control device 100 selects the presentation device corresponding to the device icon arranged within the designated selection range as a target device whose operating state is to be controlled. When the installation location is instructed according to the operation, the display processing unit 228 displays the device icon of the presentation device installed at the instructed installation location in a display mode different from the device icons of the other presentation devices, It is determined as a candidate for the target device. The display processing unit 228 determines other presentation devices as non-target device candidates. When the type of information is instructed in response to an operation, the display processing unit 228 identifies a device capable of presenting the instructed type of information as a presentable device, and changes the device icon of the identified presentable device to other devices. Displayed in a display mode different from that of the device icon of the presentation device, it is determined as a target device candidate. The display processing unit 228 determines other presentation devices as non-target device candidates.

Next, a functional configuration example of the device management system 1 according to this embodiment will be described. FIG. 11 is a schematic block diagram showing a functional configuration example of the device management system 1 according to this embodiment. A device management system 1 according to this embodiment includes a control device 100 and an input/output device 200 . In the illustrated example, the control device 100 is connected via the Internet and a router 400 to the input/output device 200 and presentation device 300 installed in the management area. The control device 100 may be configured as a server device such as a web server or database server. The device management unit 124 of the control device 100 can detect the operating input/output device 200 and the presentation device 300 detected in the management area based on the detected device information received from the router 400 .

The router 400 connects to the Internet and LAN, and relays data transmitted between the control device 100 connected to the Internet and the input/output device 200 or presentation device 300 connected to the LAN. Also, the router 400 relays data transmitted between the input/output device 200 and the presentation device 300 . Note that the router 400 does not necessarily have to be equipment independent of the input/output device 200 or the presentation device 300 . At least one of the input/output device 200 and the presentation device 300 may have the same function as the router 400, and the router 400 may be omitted.

Next, an example of device management data according to this embodiment will be described. FIG. 12 shows a first example of device management data according to this embodiment. The device management data according to this embodiment includes presentable information, detection information, operating state information, connectable device information, installation location information, and display position for each presentation device, and associates them. The operation state information indicates presence/absence of presentation for each type of presentation information. ○ mark indicates that the information is presented, and no mark indicates that the information is not presented. - indicates not applicable. The connectable device information is information indicating other connectable devices. The example of FIG. 12 shows that none of the presentation devices is presenting delivery information. The connectable device describes the information of the presentation device that is the transmission destination of the presentation information acquired by the self device or the presentation device that is the source of the presentation information provided to the self device. The display processing unit 228 of the input/output device 200 refers to the presentable information, identifies the connectable device indicated by the connectable device information of the detected presentation device, and displays the device corresponding to the detected presentation device. Add a connecting edge to the display screen between the icon and the device icon corresponding to the identified connectable device.

The installation location information indicates the installation location where the corresponding presentation device is installed. However, in the example shown in FIG. 12, for a movable presentation device whose installation location is not fixed, all possible locations are described as installation location information. Specifically, the living room and the bedroom are described for smartphones, notebook PCs, and headsets. The item of installation location information includes the installation location of the router 400 that is directly connected to the presentation device, or the installation location of another gateway device (for example, an access point) that is directly connected to the presentation device via the router 400. May contain information. In that case, it is not necessary to preliminarily describe all possible locations as installation location information. Therefore, the router 400 may include in the device detection information the information on the installation location of its own device or the gateway device directly connected to each presentation device and transmit the information to the control device 100 . The device management unit 124 of the control device 100 describes information indicating the detection location included in the device detection information received from the router 400 as installation location information in association with the presentation device.

Next, an example of user-designated range data according to this embodiment will be described. FIG. 13 shows an example of user selection range data according to this embodiment. FIG. 13 shows an example in which the range specifying graphic is a polygon having three or more vertices. The user selection range data indicates display positions of individual vertices forming a polygon. The display processing unit 228 of the input/output device 200 determines the display position of each vertex based on the operation information input from the operation unit 236, and stores user-specified range data indicating the display position determined for each vertex in the storage unit 222. is transmitted to the control device 100 together. The display processing unit 228 causes the display unit 234 to display the display screen including the range specifying figure by adding a side that sequentially connects the two vertices to the display screen. When the display processing unit 228 receives the device management data in response to the user-specified range data, it compares it with the immediately preceding device management data to identify the presentation device whose operating state information has been updated and the updated operating state information. The display processing unit 228 changes the display mode of the device icon associated with the specified presentation device to a display mode corresponding to the operating state (whether or not information is presented) indicated by the specified operating state information. Note that the display processing unit 228 erases the range selection graphic from the display screen regardless of whether the operation state information in the received device management data is updated. Note that the range designation graphic is not limited to a polygon, and may be a curve passing through a plurality of positions indicated by operation information. As a predetermined function that gives coordinate points forming such a curve, for example, a Bernstein polynomial that gives a Bezier curve, a spline function that gives a spline curve, and the like can be applied. The display processing unit 228 may specify a position on the display screen indicated by the operation information as a control point, and determine parameters of the function so as to give a curve passing through the specified control point.

Next, display examples of the display screen according to the present embodiment will be described. FIG. 14 is a diagram showing a first display example of the display screen according to this embodiment. In the example of FIG. 14, in addition to nine device icons, a presentation information selection button is arranged near the upper right corner of the display screen, and a location selection button is arranged near the lower right corner of the display screen. The presentation information selection button is a button for selecting the type of presentation information by pressing. The characters "notification" displayed on the presentation information selection button indicate a state in which notification information is selected as the type of presentation information. The display processing unit 228 switches the type of presentation information in a predetermined order each time it detects that the presentation information selection button is pressed based on the operation information input from the operation unit 236 . The display processing unit 228 refers to the device management data and determines whether or not the selected presentation information type is included in the presentation information types indicated in the presentable information for each presentation device. It is possible to determine whether or not it is the type of presentation information. The display processing unit 228 may display the display mode of the device icon corresponding to the presentation device capable of presenting the selected type of presentation information more clearly than the display mode of the device icon corresponding to the other presentation devices. In the following description, the display mode of a device icon that accepts selection based on operation information, such as device icons corresponding to other presentation devices, may be referred to as presentable display. The presentable display corresponds to an active display indicating that it can be selected according to the operation information. Then, the display processing unit 228 may accept selections of device icons corresponding to presentation devices capable of presenting the selected type of presentation information, and may not accept selections of device icons corresponding to other presentation devices. In the following description, the display mode for device icons that do not accept selection based on operation information, such as device icons corresponding to other presentation devices, may be referred to as non-presentation display. In the example shown in FIG. 14, all of the nine presentation devices can present the notification information, so the display mode of the corresponding device icon is a presentation possible display.

The location selection button is a button for selecting the location where the presentation device is installed by pressing. The text "Room selection" displayed on the location selection button indicates that a room (installation location) can be selected by pressing the button. In the example of FIG. 14, no location is specified, and any presentation device detected in the management area is selectable. Functions and processing related to the location selection button will be described later.

FIG. 15 is a diagram showing a second display example of the display screen according to this embodiment. The example of FIG. 15 shows a state in which video is selected as the type of presentation information by pressing the presentation information selection button. In this state, the characters "video" are displayed on the presentation information selection button. The display modes of the device icons corresponding to the smart phone, the bedroom TV, the TV, and the notebook PC are displayed in a presentation-enabled display, and operations can be accepted. These device icons are outlined in solid lines. This display indicates that the smart phone, bedroom TV, TV, and notebook PC can be selected as candidates for presentation devices capable of displaying images. On the other hand, the display mode of the device icons corresponding to the headset and the additional speaker is a non-presentable display, in which selection is not accepted. These device icons are outlined in dashed lines. This display expresses that the headset and additional speakers cannot be selected as presentation device candidates capable of displaying images. Therefore, when the display processing unit 228 switches the characters indicating the type of presentation information on the presentation information selection button (i.e., “video”, etc.) in response to pressing, the display mode of the presentation information selection button is changed to the presentation information. You may switch to the display mode corresponding to a kind.

FIG. 16 is a diagram showing a third display example of the display screen according to this embodiment. In the example of FIG. 16, a pointer pt and a trajectory tr are also displayed on the display screen illustrated in FIG. In the example of FIG. 16, the pointer pt is used to indicate a polygon as a selection designating figure according to the operation. For example, when detecting that any position on the display screen is pressed, the display processing unit 228 places the pointer pt at that position. The display processing unit 228 moves the display position of the pointer pt to the position indicated by the drag operation. The display processing unit 228 displays the trajectory tr of the position instructed according to the drag operation on the display screen. The display processing unit 228 determines, for example, positions indicated at predetermined time intervals from the start of the drag operation to the end of the drag operation as positions of the vertices of the polygon, and stores data indicating the series of positions determined in that order as the user range. Can be generated as designated data. The display processing unit 228 may stop displaying the pointer pt and the trajectory tr on the display screen when the drag operation ends. The display processing unit 228 transmits the generated user range designation data to the control device 100 . The device management unit 124 of the control device 100 refers to the device management data, and based on the user range designation data received from the input/output device 200, presents corresponding device icons whose display positions are included in the selection range surrounded by the trajectory tr. Identify TV as the device.

The device management unit 124 transmits a control signal for instructing the display of the image to the TV that is not displaying the image at that time (presentation device control). When an acknowledgment for the control signal is input from the TV, the device management unit 124 stores the operating state information indicating that information for video is presented in the storage unit 122 in association with the TV. By inputting an acknowledgment, it is confirmed that the TV is displaying the image. The device management unit 124 transmits device management data including updated operating state information to the input/output device 200 . The display processing unit 228 of the input/output device 200 detects an update of the operating state information from the device management data received from the control device 100, and changes the display mode of the device icon of the TV to indicate that video is being presented. In the example shown in FIG. 16, the display mode of the device icon of the TV is set to display during presentation, and the device icons of the other presentation devices are painted and displayed with a darker gradation. The display processing unit 228 may determine the display mode related to the display during presentation according to the type of presentation information. Different display modes are preset in the display processing unit 228 according to the types of presentation information. The display mode used for distinguishing the presentation information may be any one of color tone, fill pattern, other pattern, etc., or a combination thereof. A fill pattern refers to a pattern applied inside the outline of a figure, such as hatching, lattice, diagonal lines, and the like. The display processing unit 228 may share the display mode for the display during presentation and the display mode for the presentation information selection button. For example, in the example of FIG. 16, the presentation information selection button indicating "audio" and the TV device icon may be filled with the same color. As a result, a user viewing the device icon can immediately recognize the type of information presented by the presentation device.

FIG. 17 is a diagram showing a fourth display example of the display screen according to this embodiment. The example of FIG. 17 shows a state in which voice is selected as the type of presentation information by pressing the presentation information selection button. In this state, the characters "Voice" are displayed on the presentation information selection button. The display modes of the device icons corresponding to the smart phone, bedroom TV, headset, TV, notebook PC, additional speaker, and smart speaker are presentable display and selectable. A pointer pt is also displayed on the display screen. In the example of FIG. 17, the pointer pt is also used as a range specifying figure to indicate the position of the tip of itself. For example, when detecting that any position on the display screen is pressed, the display processing unit 228 places the pointer pt at that position. The pointer pt can be moved to an indicated position according to a drag operation. The display processing unit 228 displays the pointer pt at the position indicated according to the drag operation without displaying the trajectory. For example, when the tip of the pointer pt is included in the display area of one of the device icons and a press due to a click operation is detected, the display processing unit 228 specifies a user range indicating the press position (selection range) where the press was detected. Data is sent to the control device 100 . The pointer pt can also be regarded as one aspect of the range specifying figure corresponding to the click operation. The device management unit 124 of the control device 100 refers to the device management data, and identifies the additional speaker as the presentation device corresponding to the device icon whose display area includes the pressed position indicated by the user range designation data received from the input/output device 200. do.

The device management unit 124 transmits a control signal for instructing the additional speaker, which is not reproducing the auxiliary sound at that time, to reproduce the auxiliary sound (presentation device control). When an acknowledgment to the control signal is input from the additional speaker, the device management unit 124 stores the operating state information indicating that the information for the auxiliary sound is presented in the storage unit 122 in association with the additional speaker. However, if the additional speaker does not have the function of acquiring the auxiliary audio, which is the distribution information, the device management unit 124 can acquire the auxiliary audio by referring to the device management data and can be connected to the additional speaker. Identify the TV as a device that is The device management unit 124 transmits a control signal to instruct the TV, which is the specified device, to acquire auxiliary audio and transmit it to an additional speaker (presentation device control). Further, when a confirmation response to the control signal is input from the TV, the device management unit 124 associates the operation state information indicating that the information for the auxiliary sound is presented with the additional speaker and stores it in the storage unit 122. .

The device management unit 124 transmits device management data including updated operating state information to the input/output device 200 . The display processing unit 228 of the input/output device 200 detects an update of the operating state information from the device management data received from the control device 100, and changes the display mode of the device icon of the additional speaker to indicate that the auxiliary sound is presented. . In the example of FIG. 17, the display mode of the device icon of the additional speaker is also displayed during presentation, and the device icon of the TV and the device icon of the other presentation devices are painted with a darker gradation than the device icons of the other presentation devices.

FIG. 18 is a diagram showing a fifth display example of the display screen according to this embodiment. The example of FIG. 18 shows a state in which an auxiliary image is selected as the type of presentation information by pressing the presentation information selection button. In this state, the characters "auxiliary image" are displayed on the presentation information selection button. The display modes of the device icons corresponding to the smart phone, the bedroom TV, the TV, and the notebook PC are presentable display and selectable. The display mode of the device icon corresponding to each of the headset and the additional speaker is a non-presentable display and cannot be selected. The outline of the device icon displayed as non-presentable is represented by a dashed line. The dark filling of the display during presentation of the device icon of the additional speaker indicates that there is information presentation for any type of distribution information.

In FIG. 18, a rectangle is shown as another example of the range designation figure by the drag operation. The display processing unit 228 of the input/output device 200 defines a rectangle having a diagonal line segment connecting the start point and the end point of one drag operation as the range designation figure. The display processing unit 228 causes the display unit 234 to display a display screen on which the determined rectangles are superimposed. The display processing unit 228 generates, as user range specification data, data indicating the coordinates of the vertices of the determined rectangle in the order in which the sides are connected. The display processing unit 228 transmits the generated user range designation data to the control device 100 . The device management unit 124 of the control device 100 refers to the device management data, and based on the user range designation data received from the input/output device 200, the device icon corresponding to the smartphone is a device icon whose display position is included in the selection range surrounded by a rectangle. Identify the device icon.

The device management unit 124 transmits a control signal for instructing playback of the auxiliary video to the smartphone that is not presenting the auxiliary video at that time (presentation device control). When an acknowledgment to the control signal is input from the smartphone, the device management unit 124 associates the operating state information indicating that information on the auxiliary video is presented with the additional speaker and stores the operation state information in the storage unit 122 . However, if the smartphone does not have a function of acquiring the auxiliary video that is the distribution information, the device management unit 124 refers to the device management data to acquire the auxiliary video and is connectable to the smartphone. Identify the TV as The device management unit 124 transmits a control signal for instructing the TV, which is the specified device, to acquire the auxiliary video and transmit it to the smartphone (presentation device control). Further, when an acknowledgment to the control signal is input from the TV, the device management unit 124 associates the smartphone with the operating state information indicating that the information for the auxiliary video is presented, and stores the operation state information in the storage unit 122 .

The device management unit 124 transmits device management data including updated operating state information to the input/output device 200 . The display processing unit 228 of the input/output device 200 detects an update of the operating state information from the device management data received from the control device 100, and changes the display mode of the device icon of the smartphone to indicate that the auxiliary video is presented. In the example of FIG. 18, the display mode of the device icon of the smart phone is displayed during presentation, and together with the device icon of the TV, the device icons of the other presentation devices are painted in a darker color.

FIG. 19 is a diagram showing a sixth display example of the display screen according to this embodiment. The example of FIG. 19 shows a state in which auxiliary audio is selected as the type of presentation information by pressing the presentation information selection button. In this state, the characters "auxiliary voice" are displayed on the presentation information selection button. The display modes of the device icons corresponding to the smart phone, bedroom TV, headset, TV, additional speaker, and notebook PC are presentable and selectable. Among them, the display mode of the device icon of the headset is distinguished by being darkly painted as being displayed during presentation. This indicates that the headset is playing auxiliary audio.

In addition, a pull-down menu pd is displayed adjacent to the location selection button (room selection) at the lower right corner of the display screen. The pull-down menu pd is displayed in response to pressing of the location selection button, and includes “living room” and “bedroom” as options for the installation location, and one of the options is selected by the pointer pt.

When the location selection button is instructed based on the input operation information, the display processing unit 228 of the input/output device 200 refers to the device management data, specifies the installation location indicated in the installation location information as an option, A pull-down menu with the selected options arranged at the upper limit is placed adjacent to the location selection button. The display processing unit 228 makes it possible to move the display position of the pointer pt to the position indicated by the operation information. Choose an installation location. After that, the display processing unit 228 refers to the device management data to identify the presentation device installed at the selected installation location. The display processing unit 228 maintains the display mode of the device icon corresponding to the specified presentation device on the display screen as the presentable display, and changes the display mode of the device icon corresponding to the other presentation devices to the non-presentable display. . The display processing unit 228 transmits presentation information type information indicating the type of the selected presentation information and installation location information indicating the installation location to the control device 100 . The device management unit 124 of the control device 100 receives the presentation information type information and the installation location information received from the input/output device 200 . The device management unit 124 refers to the device management data, and among the presentation devices that can present the type of presentation information indicated by the presentation information type information and that can be installed or used at the installation location indicated by the installation location information, Identify headsets, laptops, and smart speakers as presentation devices that do not present any kind of presentation information.

The device management unit 124 transmits a control signal to instruct the headset, notebook PC, and smart speaker to reproduce the auxiliary sound (presentation device control). When an acknowledgment to the control signal is input from the headset, the notebook PC, and the smartphone, the device management unit 124 sends the operating state information indicating that the information for the auxiliary audio is presented to the headset, the notebook PC, and the smart speaker. , and stored in the storage unit 122 .

Note that if any one or any combination of the headset, notebook PC, and smart speaker does not have the function of acquiring auxiliary audio, the device management unit 124 may select a presentation device that has the function of acquiring auxiliary audio. Acquisition of auxiliary audio for an auxiliary audio acquisition device (e.g., TV) and auxiliary audio for an auxiliary audio presentation device (e.g., headset and smart speaker) that is a presentation device that does not have a function to acquire auxiliary audio (presentation device control). When an acknowledgment to the control signal is input from the auxiliary sound acquisition device and the auxiliary sound presentation device, the device management unit 124 associates the operating state information indicating that the information for the auxiliary sound is presented with the auxiliary sound presentation device, and stores it in the storage unit 122 . memorize to

The device management unit 124 transmits device management data including updated operating state information to the input/output device 200 . The display processing unit 228 of the input/output device 200 detects an update of the operating state information from the device management data received from the control device 100, and presents the display mode of the device icons of the headset, the notebook PC, and the smartphone as an auxiliary image. Change it to indicate yes. In the example shown in FIG. 19, the display mode of the device icon of the smart phone is displayed during presentation, and together with the device icon of the TV, the device icons of the other presentation devices are painted in a darker color.

In this state, in addition to the headset, notebook PC, and smartphone, the TV located in the living room, the smart speaker that can be moved to the living room, or the operation of selecting the display area of the device icon of the smartphone indicates whether or not the auxiliary sound needs to be played. is controllable. On the other hand, the operation of selecting additional speakers not located in the living room or the display area of the device in the bedroom TV is not accepted.

FIG. 20 is a diagram showing a seventh display example of the display screen according to this embodiment. The example of FIG. 20 shows a state in which auxiliary audio is selected as the type of presentation information by pressing the presentation information selection button. In this state, the position indicated by the pointer pt of the device icon of the notebook PC is selected as the selection range on the display screen illustrated in FIG. 19, and the reproduction of the auxiliary sound is stopped. The display of the device icon of the notebook PC during presentation is canceled, and the original display mode is restored.

The display processing unit 228 updates the display screen by moving the pointer pt to the position indicated by the operation information. When the display processing unit 228 detects a click operation from the operation information, the display processing unit 228 transmits user range designation data indicating the position to the control device 100 . The device management unit 124 of the control device 100 refers to the device management data and identifies the presentation device corresponding to the device icon whose display area includes the position indicated by the user range designation data received from the input/output device 200 . A control signal for instructing the reproduction stop of the auxiliary sound is transmitted to the notebook PC as the presentation device presenting the auxiliary sound. When an acknowledgment to the control signal is input from the notebook PC, the device management unit 124 stores in the storage unit 122 the operating state information indicating no information presentation for the auxiliary sound in association with the notebook PC.

The device management unit 124 transmits device management data including updated operating state information to the input/output device 200 . The display processing unit 228 of the input/output device 200 detects an update of the operating status information from the device management data received from the control device 100, and changes the display mode of the device icon of the notebook PC for which the update is detected to no presentation of the auxiliary video. Change to what is shown.

Note that, as illustrated in FIGS. 16-18, the device management unit 124 and the display processing unit 228 select one of the presentation devices that can present the selected type of presentation information and that is available at the selected location. One or more may be selected from the beginning based on the selection range indicated based on the operation information for the position of the device icon.
The display processing unit 228 does not change the display mode of the device icon simply by moving the display position of any device icon by a drag operation. The device management unit 124 does not change the operating state of any presentation device unless there is a change in the positional relationship between the selection range and any device icon (that is, inclusion relationship, presence or absence of overlap). In the example of FIG. 21, the position of the device icon of the smart speaker is moved to the upper right along with the pointer pt compared to the example of FIG. Since the selection range indicating the position of the device icon of the smart speaker is not specified in the operation information, playback of the auxiliary audio has not started.

In this situation, when the display processing unit 228 detects a drag operation on any device icon based on the operation information input from the operation unit 236, it arranges the device icon at the position indicated by the drag operation. The display processing unit 228 transmits to the control device 100 display position information indicating the indicated position of the device icon. The device management unit 124 of the control device 100 updates the device management data by storing the display position information received from the input/output device 200 in the storage unit 122 in association with the presentation device corresponding to the device icon. In this process, the device management unit 124 does not control whether or not to present the presentation information.

FIG. 22 is a diagram showing a second example of device management data according to this embodiment. In the example of FIG. 22, the display position of the device icon of the smart speaker, the display of the video by the TV, the display of the auxiliary video by the smartphone, the playback of the audio by the additional speaker, and the playback of the auxiliary audio by the headset, This is different from the example in FIG. Display of video by TV, display of auxiliary video by smartphone, playback of audio by additional speakers, and playback of auxiliary audio by headset are respectively shown in FIGS. 16, 18, 17, and 19. is indicated by an operation on The display position of the device icon of the smart speaker is indicated by operating the display screen illustrated in FIG. 21 .

Note that the display processing unit 228 may express the input/output relationship in a set of connectable devices using the directions of arrows arranged in the set. That is, the set of the presentation device that provides the presentation information and the presentation device that provides the presentation information has the device icon of the presentation information provider as the starting point and the device icon of the presentation information provider as the end point (tip). It may be represented by an arrow (single arrow). Also, the arrows indicating the device sets to which the presentation information is input/output may be displayed in a clearer manner than the arrows indicating the device sets to which the input/output is not performed. In the example of FIG. 23 , the display processing unit 228 arranges a solid one-sided arrow that starts from the device icon of the TV and ends at the device icon of the additional speaker on the display screen. This represents outputting the sound acquired by the TV to the additional speaker. In addition, the display processing unit 228 generates a dashed arrow that starts at the device icon of the TV and ends at the device icon of the smart speaker, a dashed arrow that starts at the device icon of the TV and ends at the device icon of the headset, A dashed arrow is arranged on the display screen, starting from the device icon and ending at the device icon of the headset. This indicates that audio can be input/output from the TV to the smart speaker, from the TV to the headset, and from the smartphone to the headset, but is not actually input/output.

Therefore, the connectable device information included in the device management data may include input/output direction information indicating whether another presentation device to be connected is the source or destination of the presentation information. . In the example of FIG. 24, the symbols (I) and (O) attached to each connectable device indicate the provider (input source) and provider destination (output destination), respectively. For example, a smart phone, an additional speaker, and a headset, which are connectable devices to the TV, are all provided by the TV. A TV that is a connectable device for additional speakers provides a source for the additional speakers. The display processing unit 228 of the input/output device 200 refers to the device management data, and if a device connectable to the presentation device designated by the operation information is the presentation information provider, the presentation device is the presentation information provider. It can be specified that The device management unit 124 of the control device 100 also refers to the device management data, and if a connectable device to the presentation device designated by the user range designation data is the presentation information provider, the presentation device provides the presentation information. You can specify what comes first. In this case, the device management unit 124 may transmit a control signal instructing the connectable device that provides the presentation information to acquire the presentation information and transmit the presentation information to the presentation device. Therefore, the provision of the presentation information is instructed to the presentation device that cannot voluntarily acquire the presentation information.

Similarly, when a connectable device to the presentation device specified by the operation information is the presentation information provider, the display processing unit 228 refers to the device management data to determine that the presentation device is the presentation information provider. can judge. In this case, the display processing unit 228 can determine that connection with a connectable device is not required because the presentation device can acquire presentation information spontaneously. The device management unit 124 of the control apparatus 100 also refers to the device management data, and when a connectable device to the presentation device specified by the user range designation data is the presentation information providing destination, the presentation device spontaneously presents the presentation information. Information can be acquired, and it can be determined that connection with a connectable device is not required.

Next, an example of device selection processing in the device management system 1 according to this embodiment will be described. FIG. 25 is a sequence diagram showing a first example of device selection processing according to this embodiment. In FIG. 25, after device detection in the management area, an image is selected as presentable information, a trajectory indicated by a drag operation is indicated as a range designation figure, and a TV is selected as a presentation device for presenting the image. The case is taken as an example (see FIG. 16).

(Step S202) The router 400 detects an operating presentation device connected to itself in the management area. The device management unit 124 of the control device 100 receives detected device information indicating the detected presentation device from the router 400 .
(Step S204) The device management unit 124 stores detection information indicating the detection of the newly detected presentation device in the storage unit 122 of its own device in association with the presentation device. The device management unit 124 transmits detection information to the input/output device 200 . The storage unit 222 of the input/output device 200 updates the device management data by storing the detection information received from the control device 100 in association with the presentation device.

(Step S<b>206 ) The display processing unit 228 of the input/output device 200 reads the updated device management data from the storage unit 222 .
(Step S208) The display processing unit 228 identifies the detected presentation devices based on the device management data, arranges device icons at different positions for each presentation device, and further arranges a presentation information selection button and a location selection button. to generate the display screen.
(Step S<b>210 ) The display processing unit 228 outputs display data indicating the generated display screen to the display unit 234 . A display screen based on the display data input from the display processing unit 228 is displayed on the display unit 234 .

(Step S212) The display processing unit 228 detects pressing of the presentation information selection button on the display screen based on the operation information input from the operation unit 236, and selects video as the type of information that can be presented.
(Step S214) The display processing unit 228 refers to the device management data and identifies a presentation device capable of displaying video as the selected type. The display processing unit 228 sets the display mode of the device icon corresponding to the identified presentation device to the presentation-enabled display, and sets the display mode of the device icon corresponding to the other presentation devices to the presentation-impossible display. The display screen is transitioned as a device selection screen that allows selection of a presentation device capable of displaying video as presentation information.
(Step S<b>216 ) The display processing unit 228 outputs display data indicating the transitioned display screen to the display unit 234 .

(Step S218) The display processing unit 228 identifies the trajectory of the drag operation as a range specifying figure based on the operation information input from the operating unit 236 (range specifying operation).
(Step S220) The display processing unit 228 updates the display screen by reflecting the instructed trajectory as a range designation graphic.
(Step S<b>222 ) The display processing unit 228 outputs display data indicating the updated display screen to the display unit 234 .
(Step S224) The display processing unit 228 transmits to the control device 100 user range designation data indicating the range designation figure.
(Step S226) The device management unit 124 of the control device 100 designates the presentation device corresponding to the device icon included in the selection range of the range designation graphic indicated by the user range designation data received from the input/output device 200 as a device to be controlled. select.

(Step S228) The device management unit 124 transmits a control signal indicating video display to the selected presentation device.
(Step S230) When an acknowledgment is input as a response to the control signal, the device management section 124 stores in the storage section 122 the operation state information indicating that information is presented for the video in association with the selected presentation device. to update the device management data.
(Step S<b>232 ) The device management unit 124 reads the updated device management data from the storage unit 122 and transmits the read device management data to the input/output device 200 .
(Step S234) The display processing unit 228 of the input/output device 200 selects the display mode of the device icon of the selected presentation device from the operating state information updated in the device management data received from the control device 100. By changing the display mode to one corresponding to the operation state information (image display), the updated operation state is reflected on the display screen.
(Step S<b>236 ) The display processing unit 228 outputs display data indicating the updated display screen to the display unit 234 .

FIG. 26 is a sequence diagram showing a second example of device selection processing according to this embodiment. FIG. 26 exemplifies a case where audio is selected as presentable information, a pointer that can indicate a position by a drag operation is indicated as a range specifying figure, and an additional speaker is selected as a presentation device for presenting audio ( See Figure 17). However, it is assumed that the additional speakers can pick up the spontaneously distributed audio.

(Step S242) The display processing unit 228 detects pressing of the presentation information selection button on the display screen based on the operation information input from the operation unit 236, and selects voice as the type of information that can be presented.
(Step S244) The display processing unit 228 refers to the device management data and identifies a presentation device capable of reproducing audio as the selected type. The display processing unit 228 sets the display mode of the device icon corresponding to the identified presentation device to the presentation-enabled display, and sets the display mode of the device icon corresponding to the other presentation devices to the presentation-impossible display. The display screen is transitioned as a device selection screen that allows selection of a presentation device capable of reproducing audio as presentation information.
(Step S<b>246 ) The display processing unit 228 outputs display data indicating the transitioned display screen to the display unit 234 .

(Step S248) The display processing unit 228 determines the position of the pointer by the drag operation based on the operation information input from the operation unit 236 (range specifying operation).
(Step S250) The display processing unit 228 updates the display screen by reflecting the pointer at the indicated position as a range specifying figure.
(Step S<b>252 ) The display processing unit 228 outputs display data indicating the updated display screen to the display unit 234 .
(Step S254) The display processing unit 228 transmits to the control device 100 user range designation data indicating the position of the pointer indicated by the click operation while the pointer is being displayed.
(Step S256) The device management unit 124 of the control device 100 selects the presentation device corresponding to the device icon whose display range includes the position indicated by the user range designation data received from the input/output device 200 as a device to be controlled. .

(Step S258) The device management unit 124 transmits a control signal indicating audio reproduction to the selected presentation device.
(Step S260) When an acknowledgment is input as a response to the control signal, the device management section 124 stores in the storage section 122 the operating state information indicating that information is presented in response to voice, in association with the selected presentation device. to update the device management data.
(Step S<b>262 ) The device management unit 124 reads the updated device management data from the storage unit 122 and transmits the read device management data to the input/output device 200 .
(Step S264) The display processing unit 228 of the input/output device 200 selects the display mode of the device icon of the selected presentation device from the operating state information updated in the device management data received from the control device 100. By changing the display mode to one corresponding to the operation state information (sound reproduction), the updated operation state is reflected on the display screen.
(Step S<b>266 ) The display processing unit 228 outputs display data indicating the updated display screen to the display unit 234 .

FIG. 27 is a sequence diagram showing a third example of device selection processing according to this embodiment. FIG. 27 exemplifies a case where an auxiliary video is selected as presentable information, a rectangle designated by a drag operation is designated as a range designation figure, and a smartphone is selected as a presentation device for presenting the auxiliary video ( See Figure 18). However, it is premised that the smartphone can acquire the distributed auxiliary video spontaneously.

(Step S272) The display processing unit 228 detects pressing of the presentation information selection button on the display screen based on the operation information input from the operation unit 236, and selects auxiliary video as the type of information that can be presented.
(Step S274) The display processing unit 228 refers to the device management data and identifies a presentation device capable of displaying the auxiliary video as the selected type. The display processing unit 228 sets the display mode of the device icon corresponding to the identified presentation device to the presentation-enabled display, and sets the display mode of the device icon corresponding to the other presentation devices to the presentation-impossible display. The display screen is transitioned as a device selection screen that allows selection of a presentation device capable of displaying an auxiliary video as presentation information.
(Step S<b>276 ) The display processing unit 228 outputs display data indicating the transitioned display screen to the display unit 234 .

(Step S278) The display processing unit 228 identifies the rectangle by the drag operation as the range specifying figure based on the operation information input from the operation unit 236 (range specifying operation).
(Step S280) The display processing unit 228 updates the display screen by reflecting the indicated rectangle as the range designation figure.
(Step S<b>282 ) The display processing unit 228 outputs display data indicating the updated display screen to the display unit 234 .
(Step S284) The display processing unit 228 transmits to the control device 100 user range designation data indicating the range designation figure.
(Step S286) The device management unit 124 of the control device 100 designates the presentation device corresponding to the device icon included in the selection range of the range designation graphic indicated by the user range designation data received from the input/output device 200 as the device to be controlled. select.

(Step S288) The device management unit 124 transmits a control signal indicating display of the auxiliary video to the selected presentation device.
(Step S290) When an acknowledgment is input as a response to the control signal, the device management section 124 stores the operating state information indicating that information is presented for the auxiliary video in the storage section 122 in association with the selected presentation device. By storing, the device management data is updated.
(Step S<b>292 ) The device management unit 124 reads the updated device management data from the storage unit 122 and transmits the read device management data to the input/output device 200 .
(Step S294) The display processing unit 228 of the input/output device 200 selects the display mode of the device icon of the selected presentation device from the operating state information updated in the device management data received from the control device 100. By changing the display mode to one corresponding to the operation state information (display of the auxiliary image), the updated operation state is reflected on the display screen.
(Step S<b>296 ) The display processing unit 228 outputs display data indicating the updated display screen to the display unit 234 .

FIG. 28 is a sequence diagram showing a fourth example of device selection processing according to this embodiment. FIG. 28 shows that auxiliary audio is selected as presentable information, a pull-down menu is displayed in response to pressing of the location selection button, living room is selected as installation location information, and the head is selected as a presentation device capable of reproducing auxiliary audio in the living room. A case where a set, a notebook PC, and a smart speaker are selected is taken as an example (see FIG. 19).

(Step S302) The display processing unit 228 detects pressing of the presentation information selection button on the display screen based on the operation information input from the operation unit 236, and selects auxiliary sound as the type of information that can be presented.
(Step S304) The display processing unit 228 refers to the device management data and identifies a presentation device capable of displaying auxiliary audio as the selected type. The display processing unit 228 sets the display mode of the device icon corresponding to the identified presentation device to the presentation-enabled display, and sets the display mode of the device icon corresponding to the other presentation devices to the presentation-impossible display. The display screen is transitioned as a device selection screen that allows selection of a presentation device capable of reproducing auxiliary audio as presentation information.
(Step S<b>306 ) The display processing unit 228 outputs display data indicating the transitioned display screen to the display unit 234 .

(Step S<b>308 ) The display processing unit 228 detects (selects) pressing of the place selection button on the display screen based on the operation information input from the operation unit 236 .
(Step S310) The display processing unit 228 refers to the device management data and extracts the living room and the bedroom as the installation locations of the presentation devices capable of reproducing the auxiliary sound. The display processing unit 228 updates the display screen by arranging a pull-down menu including the extracted living room and bedroom as options as a room list adjacent to the place selection button. The display processing unit 228 outputs display data indicating the updated display screen to the display unit 234 .
(Step S312) The display processing unit 228 displays the pointer at the position indicated by the operation information, and detects the depression based on the operation information when the position indicates the display area of any of the options. selects the installation location associated with the option (room selection operation).
(Step S314) The display processing unit 228 transmits to the control device 100 installation location information indicating the selected installation location and presentation information type information indicating auxiliary audio as the type of the selected presentation information.

(Step S316) The device management unit 124 of the control device 100 refers to the device management data, and determines that the type indicated in the presentation information type information received from the input/output device 200 is capable of reproducing auxiliary audio, and the type indicated in the installation location information is Among the presentation devices available in the living room as the indicated installation location, the headset, notebook PC, and smart speaker are identified as presentation devices that do not present any type of presentation information at that time.
(Step S318) The device management unit 124 transmits a control signal for instructing the reproduction of the auxiliary audio to the headset, notebook PC, and smart speaker, which are the identified presentation devices, so that the identified presentation devices start up.
(Step S320) When an acknowledgment to the control signal is input from the headset, the notebook PC, and the smartphone, the device management unit 124 outputs the operating state information indicating that the information for the auxiliary audio is presented to the headset, the notebook PC, and the smartphone. Then, the device management data is updated by storing in the storage unit 122 in association with the smart speaker.

(Step S<b>322 ) The device management section 124 transmits the updated device management data to the input/output device 200 .
(Step S324) The display processing unit 228 of the input/output device 200 selects the display mode of the device icon of the selected presentation device from the operating state information updated in the device management data received from the control device 100. By changing the display mode to one corresponding to the operation state information (display of auxiliary sound), the updated operation state is reflected on the display screen.
(Step S<b>326 ) The display processing unit 228 outputs display data indicating the updated display screen to the display unit 234 .

FIG. 29 is a sequence diagram showing an example of device deselection processing according to this embodiment. FIG. 29 exemplifies a case where a headset, a notebook PC, and a smart speaker are selected as presentation devices for reproducing auxiliary audio, and the notebook PC stops playing the auxiliary audio (FIG. 20 reference).

(Step S<b>338 ) The display processing unit 228 of the input/output device 200 detects the drag operation as the range specifying operation from the operation information input from the operation unit 236 .
(Step S340) The display processing unit 228 updates the display screen by arranging the pointer at the position indicated by the operation information (reflecting the range designation operation).
(Step S<b>342 ) The display processing unit 228 outputs display data indicating the updated display screen to the display unit 234 .
(Step S344) When detecting a click operation from the operation information input from the operation unit 236, the display processing unit 228 specifies the position indicated by the pointer at that time, and generates user range specification data indicating the specified position. Send to the control device 100 .

(Step S346) The device management unit 124 of the control device 100 refers to the device management data, and displays the presentation device corresponding to the device icon whose display area includes the position indicated by the user range designation data received from the input/output device 200. Identify the notebook PC as. The device management unit 124 transmits a control signal indicating stop of reproduction of the auxiliary sound to the identified notebook PC.
(Step S350) When an acknowledgment is input as a response to the control signal, the device management unit 124 associates the operation state information indicating that there is no information for the auxiliary sound with the notebook PC that is the specified presentation device. By storing in the storage unit 122, the device management data is updated.
(Step S<b>352 ) The device management unit 124 reads the updated device management data from the storage unit 122 and transmits the read device management data to the input/output device 200 .
(Step S354) The display processing unit 228 of the input/output device 200 selects the display mode of the device icon of the selected presentation device from the operating state information updated in the device management data received from the control device 100. By changing the display mode to one corresponding to the operation state information (no reproduction of auxiliary sound), the updated operation state is reflected on the display screen.
(Step S<b>356 ) The display processing unit 228 outputs display data indicating the updated display screen to the display unit 234 .

As described above, in the device management system 1 according to the present embodiment, the display processing unit 228 causes the display screen to display the range designation figure in the selection range instructed according to the operation. The device management unit 124 selects the presentation device corresponding to the device icon included in the selection range as the target device whose operating state is to be controlled.
According to this configuration, it is possible to intuitively select the device whose operating state is to be controlled by the selection range defined by the operation using the range designation graphic.

<Modification>
Next, modified examples of the above embodiments will be described. In each embodiment, the input/output device 200 may be provided with a device management unit as a configuration unit that implements a function corresponding to the device management unit 124 of the control device 100, and configured as a single device management apparatus. In that case, the control device 100 may be omitted. Transmission and reception of various data between the input/output device and the control device are input/output within the device management apparatus without passing through the communication unit 226 . The data stored in the storage unit 122 of the control device 100 as described above is stored in the storage unit 222 of the input/output device 200 . The input/output device 200 configured as a single device management device constitutes the device management system 1 itself.

Even if the input/output device 200 has a device management unit, the control device 100 may remain. In this case, the input/output device 200 transmits updated device management data to the control device 100 each time the device management data is updated or at predetermined intervals (for example, several hours to several days). The device management unit 124 of the control device 100 updates the device management data stored in the storage unit 122 with the device management data received from the input/output device 200 .

The device management section of the input/output device 200 may cause the device management section 124 of the control device 100 to perform device control without performing device control on the presentation device. In this case, the device management unit of the input/output device 200, for example, sends a control signal indicating the type of presentation information and the necessity of presentation of the presentation information to the presentation device to be controlled for the necessity of presentation of the presentation information. 124 for transmission. When a confirmation response to the control signal is input, the device management unit 124 updates the device management data as described above, stores it in the storage unit 122 , and transmits it to the input/output device 200 .

In each of the above-described embodiments and modifications, the transmission target in the transmission of device management data between the device management unit 124 and the display processing unit 228 does not necessarily have to be the entire device management data. It is sufficient if the difference from the device management data is included. For example, if the display positions of device icons related to some presentation devices are updated and other information is not updated, the display position information indicating the display positions of the device icons related to the presentation devices is included as a transmission target. Just do it.

In each of the above-described embodiments, the case where a circle mark is used as a device icon, which is a display element indicating an individual presentation device, was used as an example, but the present invention is not limited to this. These display elements may be screen components (UI (User Interface) components) having different display modes such as shape, size, color (including shading), and pattern. For example, a display element of a presentation device may be a button with its typical external shape. A display element indicating a user may also be a screen component whose display mode is different from that of the user icon described in the first embodiment. For example, a button representing a person's face may be used as a display element indicating the user. Also, as the range designation graphic, a graphic having a display mode different from that described in the first embodiment may be used. For example, the shape of the range designation graphic may be a rectangle or a trapezoid.

In the second embodiment, the presentable indication and the non-presentable indication are represented by solid lines and dashed lines of the contours of the display elements, respectively. is not limited to this example. For example, if the contour in the presentable display has a higher contrast with the background and the inside of the contour, the contour of the non-presentable display may be a solid line or may be represented by other line types.
Further, in the above description, the case where the display during presentation is represented by whether or not the display element is painted is taken as an example. is not limited to this example, as long as it is clearly expressed. For example, the display mode of the display during presentation may be represented by the shade of the paint, the blinking of the outline, and the like.

Some components of each of the above embodiments may be omitted, or may be combined in other embodiments.
For example, in the second embodiment, the display processing unit 228 does not have to specify the type of presentable information based on the operation information. In that case, the device management data need not be managed including operating state information indicating the operating state for each type of presentation information. The display processing unit 228 does not have to specify the installation location of the presentation device based on the operation information. In that case, the device management data may not include installation location information for each presentation device. Also, the control device 100 does not have to be connected to the router 400 via the Internet as long as it can be connected to each device within the management area.

In the first embodiment, the display processing unit 228 identifies the type of presentable information based on the operation information, and displays the presentable display of the display element of the presentation device that can present the identified type of information. It may be expressed in a display mode (for example, color tone, fill pattern, other pattern, etc.) corresponding to the type. In that case, the device management data may include operating state information indicating the operating state for each type of presentation information. The display processing unit 228 may specify the installation location of the presentation device based on the operation information, and set the display mode of the presentation device that allows the type of information to be presented at the specified installation location to be the presentable display. In that case, the device management data is managed including installation location information for each presentation device. In the first embodiment, the control device 100 may be connected to each device within the management area via the router 400 and the Internet.
Screen parts used for selecting the installation location according to the operation information and screen parts used for the selection of the installation position can be applied as long as one of multiple options can be selected according to the operation information. is. For example, buttons, pull-down menus, slider bars, radio buttons, etc. may be used.

In each of the above-described embodiments and modifications, the control device 100 may be configured as a server device that is connected to each device in each management area via the router 400 and the Internet for each management area. good. In that case, the control device 100 manages devices independently for each management area.

As described above, the device management system 1 according to the above embodiment displays a display screen on which display elements (for example, device icons) indicating individual devices (for example, presentation devices) are arranged at different positions. (for example, the display unit 234), and at least one of the position of the display element and the range designation figure (for example, circle, locus, rectangle, pointer) indicating the selection range forming a part of the display screen is displayed according to the operation A changing display processing unit (for example, the display processing unit 228) is provided. The device management system 1 selects, as a target device, a device corresponding to a display element whose positional relationship with the range designation graphic has changed, and controls the operating state of the target device (for example, the device management unit 124). ). The display processing unit determines the display mode of the display elements of the target device to a display mode corresponding to the operating state of the target device.
According to this configuration, the operating state of the target device can be controlled by changing the positional relationship between the display element indicating the target device and the range designating figure by an operation, and the display element is displayed in a display mode corresponding to the operating state. be. The user can easily select the target device whose operating state is to be controlled, and can grasp the operating state of the device while looking at the display screen.

When the type of information is instructed in response to an operation (for example, pressing a presentation information selection button), the display processing unit refers to device management data indicating the type of information that can be presented for each device (type of information that can be presented). and identifies devices that can present the indicated type of information as presentable devices. The display processing unit may display the display elements of the presentable device in a display mode (for example, presentable display) different from the display elements of a device separate from the presentable device.
According to this configuration, by specifying the type of information that can be presented, a device that can present the specified type of information is selected. Therefore, by designating the type of information, it is possible to narrow down the devices that can be selected and to easily distinguish them from other devices.

When an installation location is instructed in response to an operation (for example, by pressing a location information selection button), the display processing unit refers to device management data indicating the installation location of each device, and selects devices that can be used in the instructed location. may be identified as available devices. The display processing unit may cause the display elements of the usable device to be displayed in a different display manner from the display elements of a device separate from the usable device.
With this configuration, by specifying an installation location, a device that can be used at the specified installation location is selected. Therefore, by designating the installation location, it is possible to narrow down the devices that can be selected and to easily distinguish them from other devices.

The display processing unit refers to device management data indicating other connectable devices for each device, and displays a figure (for example, an arrow) indicating the possibility of connection with another device for each device. It may be represented between display elements respectively corresponding to one device and said other device.
With this configuration, the user can easily understand that the devices corresponding to the display elements shown at both ends of the figure indicating the connectability can be connected to each other.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configurations are not limited to those described above, and various design changes can be made without departing from the gist of the present invention. It is possible to

The control device 100, the input/output device 200, the presentation device 300, and each component of the devices that are modifications thereof may each be configured by dedicated members or integrated circuits, but the configuration is not limited to this. The control device 100, the input/output device 200, the presentation device 300, and devices that are modifications thereof are computer systems that include processors such as CPUs (Central Processing Units) and computer-readable storage media such as ROMs and RAMs. may consist of The processor reads a program stored in advance in a computer-readable storage medium, executes the read program, cooperates with the computer-readable storage medium or other hardware resources, and performs part of the above components. Or you may play all the functions.
In the present application, "executing a program" means executing processing instructed by individual instructions written in the program. The “computer system” is a computer system built in the control device 100, the input/output device 200, the presentation device 300, and devices that are modifications thereof, and includes hardware such as an OS (Operating System) and peripheral devices. shall include clothing. The term "computer-readable recording medium" refers to portable media such as magneto-optical discs, memory chips, ROMs and CD-ROMs, and storage devices such as hard disks incorporated in computer systems. Furthermore, "computer-readable recording medium" means a medium that dynamically stores a program for a short period of time, such as a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line. It may also include a volatile memory inside a computer system that serves as a server or client in that case, which holds the program for a certain period of time. Further, the program may be for realizing part of the functions described above, or may be capable of realizing the functions described above in combination with a program already recorded in the computer system.

REFERENCE SIGNS LIST 1 device management system 100 control device 122 storage unit 124 device management unit 126 communication unit 200 input/output device 222 storage unit 226 communication unit 228 display processing unit 234 Display unit 236 Operation unit 300 Presentation device 326 Communication unit 332 Presentation information processing unit 334 Information presentation unit

Claims (10)

displaying on the display unit a display screen in which display elements indicating individual devices are arranged at different positions;
a display processing unit that changes at least one of the position of the display element and a range designation figure indicating a selection range forming a part of the display screen according to an operation;
a device management unit that selects, as a target device, a device corresponding to a display element whose positional relationship with the range designation graphic has changed, and controls the operation state of the target device, wherein the display processing unit A device management system that defines a display mode of a display element of a device to a display mode corresponding to an operation state of the target device. In the display screen, a user display element that is a display element that indicates a user is arranged at a position different from a device display element that is a display element that indicates the device,
the range designation figure is displayed in an area including the user display element;
2. The device management system according to claim 1, wherein the device management section selects a device corresponding to a device display element entering the selection range or a device display element exiting the selection range as the target device. The display processing unit displays the range designation figure in the selection range instructed according to the operation,
2. The device management system according to claim 1, wherein said device management section selects devices corresponding to display elements included in said selection range as target devices. The display processing unit
When the type of information is indicated according to the operation,
with reference to device management data that indicates the type of information that can be presented per device;
identifying devices capable of presenting the indicated type of information as presentable devices;
4. The device management system according to any one of claims 1 to 3, wherein the display elements of the presentable device are displayed in a display mode different from the display elements of devices separate from the presentable device. The display processing unit
When the installation location is indicated according to the operation,
Refer to device management data that indicates the installation location for each device,
Identify a device that can be used at the indicated location as a usable device,
5. The device management system according to any one of claims 1 to 4, wherein the display element of the usable device is displayed in a display mode different from the display element of the device separate from the usable device. The display processing unit
referencing device management data indicating other devices that can be connected to each device;
6. A figure indicating the possibility of connection with said other device is displayed for each said one device between display elements respectively corresponding to said one device and said other device. 1. The device management system according to item 1. displaying on the display unit a display screen in which display elements indicating individual devices are arranged at different positions;
a display processing unit that changes at least one of the position of the display element and a range designation figure indicating a selection range forming a part of the display screen according to an operation;
a device management unit that selects, as a target device, a device corresponding to a display element whose positional relationship with the range designation graphic has changed, and controls the operation state of the target device, wherein the display processing unit A device management apparatus that defines a display mode of a display element of a device to a display mode corresponding to the operating state of the target device. displaying on the display unit a display screen in which display elements indicating individual devices are arranged at different positions;
a display processing unit that changes at least one of the position of the display element and a range designation figure indicating a selection range forming a part of the display screen according to an operation;
The display processing unit
selecting, as a target device, a device corresponding to a display element whose positional relationship with the range specifying figure has changed;
An input/output device that defines a display mode of display elements of the target device to a display mode corresponding to an operation state of the target device. A program for causing a computer to function as the device management device according to claim 7 or the input/output device according to claim 8. a first step in which the display processing unit causes the display unit to display a display screen in which display elements indicating individual devices are arranged at different positions;
a second step in which the display processing unit changes at least one of the position of the display element and a range designation figure indicating a selection range forming a part of the display screen according to the operation;
a third step in which the device management unit selects, as a target device, a device corresponding to the display element whose positional relationship with the range designation graphic has changed, and controls the operating state of the target device;
a fourth step in which the display processing unit determines a display mode of the display elements of the target device to a display mode corresponding to the operating state of the target device;
A device management method that performs

Images