WO2016204100A1

WO2016204100A1 - Device controller, control method thereof, and device control system

Info

Publication number: WO2016204100A1
Application number: PCT/JP2016/067468
Authority: WO
Inventors: 峻介須田; 命根岸; 太郎寺社
Original assignee: キヤノンイメージングシステムズ株式会社
Priority date: 2015-06-19
Filing date: 2016-06-07
Publication date: 2016-12-22

Abstract

In the present invention, a device controller remotely operates, using an internal device and/or an external device, an information processing apparatus upon a network. The device controller stores, and transmits to the information processing apparatus, virtual composite device identification information for identification as one virtual composite device which is a combination of at least one prescribed function which the internal device or the external device comprises and is connected to the device controller. When controlling, according to a control request from the information processing apparatus, the virtual composite device which the information processing device has identified on the basis of the virtual composite device identification information, the device controller communicates data in a first data format between the virtual composite device and the internal device or the external device, and communicates data in a second data format between the virtual composite device and the device control apparatus, and controls said virtual composite device.

Description

Device control apparatus, control method therefor, and device control system

The present invention relates to a device control apparatus, a control method thereof, and a device control system for processing various requests sent from an information processing apparatus connected via a network.

In recent years, with the spread of networks, devices such as printers, storages, and scanners are used as shared devices by client PCs that are information processing apparatuses on the network. Several implementation methods have been proposed.

For example, a system has been proposed in which input / output devices such as a display terminal, a mouse, and a keyboard are directly connected to a server device for communication (see Patent Document 1). In Patent Literature 1, when a display terminal attempts to connect to a server device, the server device performs authentication. Thereafter, a session is started, and the user connects an input / output device such as a mouse or a keyboard based on information displayed on the UI of the display terminal. As a result, devices such as a mouse and a keyboard belonging to the same group as a user having a display terminal are grouped.

JP 2007-4656 A

However, in the system described in Patent Document 1, since the server device and the input / output device are connected via a network, the input / output device that is not directly connected to the server device is not considered. Further, there is a problem that session management becomes complicated as the number of input / output devices connected to the server apparatus increases. The present invention has been made for session control of devices including input / output devices (internal devices, external devices) that are not directly connected to the server device (device control device), which is a problem of the above-described conventional technology. When an information processing device controls an internal device that is a predetermined function of the device control device or an external device directly connected to the device control device and a different device (internal device, external device) on the network, Regardless of the number of sessions based on the number of devices and external devices, a device control apparatus capable of controlling one or more devices (internal device, external device) between the information processing apparatus and the device control apparatus, its control method, and device control system The purpose is to provide.

Furthermore, according to the present invention, when an internal device or an external device included in one or more device control apparatuses on a network is controlled from an information processing apparatus, a plurality of devices are combined and recognized as one device to use a session. Thus, it is possible to control connection or disconnection for each device control apparatus that uses the session, and to improve the convenience of session management.

In order to solve the above-described problem, the device control apparatus according to claim 1 is a device control apparatus that is connected to an information processing apparatus via a network and recognizes at least one predetermined function as an internal device, Identification information storage means for storing virtual composite device identification information for identifying as a virtual composite device composed of the internal device and / or an external device connected to the device controller, and the virtual composite device identification information An identification information transmitting means for transmitting to the information processing apparatus; and the internal device when controlling the virtual composite device identified by the information processing apparatus based on the virtual composite device identification information in accordance with a control request from the information processing apparatus. Or data communication between the external device and the virtual composite device. And the virtual composite device and the device control apparatus perform data communication in a first data format, and the information processing apparatus and the device control apparatus perform data communication and control in a second data format. Control means.

In order to solve the above problem, an intermediary device according to claim 15 is an intermediary device connected to an information processing device and a device control device via a network, and the at least one predetermined device in the device control device Identification information storage means for storing virtual composite device identification information for recognizing a function combining functions as a virtual composite device connected to the mediation apparatus, and identification for transmitting the virtual composite device identification information to the information processing apparatus Information transmission means; device information management means for updating and storing connection apparatus information for managing information of the own apparatus and connection device held by the information processing apparatus and the device control apparatus; and A device configuration management means for selecting a configuration of a composite device and a device that is an element thereof; When the processing device controls the virtual composite device identified based on the virtual composite device identification information in accordance with a control request from the information processing device, second data is transmitted between the information processing device and the device control device. Virtualization control means for performing data communication and controlling in a format.

According to the present invention, an internal device included in a device control apparatus, an external device directly connected to the device control apparatus, and different devices on the network (internal device, external device) are controlled from the information processing apparatus via the network. In this case, the number of sessions in data communication between the device control apparatus and the information processing apparatus can be reduced regardless of the number of internal devices and external devices of the device control apparatus.

Furthermore, by allowing the information processing apparatus to recognize a plurality of devices (internal devices and external devices) as one device, it is possible to centrally manage connection or disconnection of sessions between one or more device control apparatuses and the information processing apparatus. It becomes possible to improve the property.

1 is a block diagram schematically showing a configuration of an example of a device control system according to a first embodiment of the present invention. It is a figure which shows an example of the virtual device identification information shown in FIG. It is a figure which shows an example of the virtual composite device identification information shown in FIG. It is a figure which shows the device structure of the virtual composite device in the device control system shown in FIG. FIG. 3 is a sequence diagram for explaining an operation when the information processing apparatus performs virtualization control of an internal device provided in the device control apparatus and an external device connected to the device control apparatus in the device control system illustrated in FIG. 1. In the device control system shown in FIG. 1, the operation when disconnecting / reconnecting the session of the virtual composite device connected to the information processing apparatus in conjunction with disconnection / reconnection of the session of the external device connected to the device control apparatus It is a sequence diagram for demonstrating. It is a block diagram which shows roughly the structure of an example of the device control system which concerns on the 2nd Embodiment of this invention. It is a figure which shows the device structure of the virtual composite device in the device control system shown in FIG. FIG. 8 is a sequence diagram for explaining an operation when the information processing apparatus performs virtualization control of a virtual composite device in the device control system illustrated in FIG. 7. FIG. 8 is a sequence diagram for explaining an operation when the information processing apparatus performs virtualization control of a virtual composite device in the device control system illustrated in FIG. 7. It is a block diagram which shows roughly the structure of an example of the device control system which concerns on the 3rd Embodiment of this invention. FIG. 11 is a sequence diagram for explaining an operation when the information processing apparatus performs virtualization control of an internal device provided in the device control apparatus in the device control system illustrated in FIG. 10. FIG. 11 is a sequence diagram for explaining an operation when the information processing apparatus performs virtualization control of an internal device provided in the device control apparatus in the device control system illustrated in FIG. 10. It is a flowchart for demonstrating the device selection process performed by the device control apparatus shown in FIG. It is a block diagram which shows roughly the structure of an example of the device control system which concerns on the 4th Embodiment of this invention.

Hereinafter, an example of a device control system according to an embodiment of the present invention will be described with reference to the drawings.
[First Embodiment]

FIG. 1 is a block diagram schematically showing an example of the configuration of a device control system according to an embodiment of the present invention.

In the illustrated device control system, an information processing apparatus 100 is connected to a device control apparatus 200 via a local area network (LAN) or a wide area network (WAN). The LAN and WAN are configured by a wired line or a wireless line. The external device 300 is locally connected to the device control apparatus 200 through, for example, a communication interface (for example, USB or HDMI (registered trademark)). In the following description, the communication interface will be described as USB.
<Configuration of Information Processing Device 100>

The information processing apparatus 100 is an apparatus such as a personal computer (PC) used by a user. The hardware configuration includes a CPU, an input unit, a display unit, a memory, a communication unit (both not shown), and a storage unit 110, and these hardware units are connected to each other via an internal bus.

The CPU controls the entire information processing apparatus 100. The input unit includes, for example, a keyboard and a mouse. For example, a liquid crystal monitor is used as the display unit. The memory is a RAM, a ROM, or the like.

The communication unit is an interface for connecting to a LAN and a WAN, and performs data transmission / reception with the device control apparatus 200. As a result, the information processing apparatus 100 can transmit / receive data to / from the external device 300 locally connected to the device control apparatus 200 and the internal device 400 provided in the device control apparatus 200.

The storage unit 110 stores an operating system that is software (hereinafter referred to as “OS”: not shown), an application 101, a device driver 102, a virtualization control unit 103, and a communication control unit 104, as well as other functions. Such software is stored. These software are read into the memory and operate according to the control of the CPU.

The application 101 instructs the device driver 102 to perform a data input / output request, thereby combining an external device 300 connected to the device control apparatus 200 and an internal device 400 provided in the device control apparatus 200 (hereinafter referred to as a composite device). Software for controlling a virtual composite device).

The device driver 102 is a functional unit for controlling the external device 300, the internal device 400, and the virtual composite device 500. A data input / output request from the OS or the application 101 is converted into data in a data format corresponding to the external device 300, the internal device 400, and the virtual composite device 500 of FIG. The control command is sent to the virtualization control unit 103. Furthermore, the device driver 102 notifies the application 101 of a response to the control command. The device driver 102 is software provided by a manufacturer that is generally manufactured, and the information processing apparatus 100 controls the external device 300, the internal device 400, and the virtual composite device 500 by the device driver 102 corresponding to each device. Can do.

The virtualization control unit 103 converts a control command (that is, a data input / output request) sent from the device driver 102 into packet data (hereinafter referred to as “USB data”) conforming to the USB data format. Also, the virtualization control unit 103 converts the USB data sent from the communication control unit 104 into a data format similar to the control command and sends it to the device driver 102.

Further, the virtualization control unit 103 directly responds to the data transmission / reception request to the external device 300, the internal device 400, and the virtual composite device 500 by the external device 300, the internal device 400, and the virtual composite device 500 to the information processing apparatus 100. A function (hereinafter referred to as a “virtualization control function”) for simulating the same behavior as when connected locally (locally). With this virtualization control function, the information processing apparatus 100 has the same connection state of the external device 300, the internal device 400, and the virtual composite device 500 as when the external device 300, the internal device 400, and the virtual composite device 500 are locally connected. It is possible to transmit and receive data by recognizing that this is the state.

The communication control unit 104 performs conversion processing between the USB data sent from the virtualization control unit 103 and a network packet when communicating with the device control apparatus 200 via the LAN and WAN, and the device control apparatus 200 and the information processing apparatus 100. Data transmission / reception is controlled between them.

Also, the communication control unit 104 starts and disconnects a session between the device control apparatus 200 and the information processing apparatus 100 in response to a data transmission / reception request sent from the application 101 or the device driver 102 via the virtualization control unit 103. Take control. Sessions between the device control apparatus 200 and the information processing apparatus 100 are generated for the number of devices that the information processing apparatus 100 performs virtualization control (for the number of internal devices, external devices, and virtual composite devices).
<Configuration of Device Control Device 200>

The device control apparatus 200 is a peripheral device such as a printer or a multifunction peripheral, for example, and includes a CPU, a memory, a communication unit, and a storage unit (both not shown) as a hardware configuration, and an internal device 400. Yes. Details of the configuration of the internal device 400 will be described later.

CPU controls the entire device control apparatus 200. The input device 208 is, for example, an input device such as a keyboard, a numeric keypad, and a mouse, or a software keyboard that realizes these functions by software. For example, a liquid crystal monitor is used as the display device 210. The memory is RAM, ROM, or the like.

The communication unit is an interface for connecting to the LAN and WAN, and performs data transmission / reception with the information processing apparatus 100.

The storage unit stores an OS (not shown) that is software, a communication control unit 201, a virtualization control unit 202, a virtual device control unit 203, and a virtual composite device control unit 211, and software related to other functions. It is remembered. These software are read into the memory and operate according to the control of the CPU.

The communication control unit 201 controls transmission / reception of input / output data and the like performed with the information processing apparatus 100.

The virtualization control unit 202 controls the information processing apparatus 100 to recognize the internal device 400, the external device 300, and the virtual composite device 500 as if they are directly connected (local connection) to the information processing apparatus 100. In order to be able to do this, the virtual composite device 500 is controlled according to (in conjunction with) a control request from the virtualization control unit 103 of the information processing apparatus 100.

The virtual device control unit 203 includes virtual device identification information 204 and a data format conversion processing unit 205. Furthermore, the virtual device control unit 203 includes a switching unit such as a user interface that switches the internal device 400 included in the device control apparatus 200 to a virtual USB device.

The virtual device identification information 204 is used for causing the information processing apparatus 100 and a virtual composite device control unit 211 described later to recognize the internal device 400 in the same manner as the external device 300. Details of the virtual device identification information 204 will be described later.

The data format conversion processing unit 205 performs conversion processing between packet data conforming to the USB data format sent from the virtual composite device control unit 211 and data in a data format that can be processed by the internal device 400, and connects to the internal device 400. Data transmission / reception between the input I / F 207 and the display I / F 209 and the virtual composite device control unit 211 is enabled.

The virtual composite device control unit 211 performs control for causing the virtualization control unit 202 to recognize and store an arbitrary plurality of devices (internal devices and external devices) as one virtual composite device. The virtual composite device control unit 211 stores virtual composite device identification information 212.

The virtual composite device identification information 212 is used for causing the information processing apparatus 100 to recognize the virtual composite device 500. Details of the virtual composite device identification information 212 will be described later.
<Configuration of External Device 300>

The external device 300 includes individual identification information 301 and a communication I / F 302. The external device 300 is an input device having an HID class input device, and is externally connected to the device control device 200 via a USB interface.

The individual identification information 301 is information for identifying the external device 300, and is assigned for each model to identify the vendor ID (VID) assigned for each manufacturer that manufactured the device to identify the manufacturer. Product ID (PID) assigned, serial number assigned to each device in order to identify the individual device.

The communication I / F 302 is a USB interface or HDMI (registered trademark), and is a functional unit for communicating with the device control apparatus 200. Note that the communication I / F 302 may be other than a USB interface or HDMI (registered trademark) as long as communication with the device control apparatus 200 can be realized.

In the device control system shown in FIG. 1, only one information processing apparatus 100, device control apparatus 200, and external device 300 are shown, but the information processing apparatus 100, device control apparatus 200, and external device are shown. The number of 300 is not limited to the illustrated example.

Alternatively, the external device 300 may not be connected to the device control apparatus 200, and the device control system may be configured by only the information processing apparatus 100 and the device control apparatus 200.
<Configuration of Internal Device 400>

The internal device 400 (input / output device) includes an input device 208 and a display device 210 used as a user interface. The input device 208 and the display device 210 are connected to the internal bus 206 by an input I / F 207 and a display I / F 209, respectively, and are connected to each hardware of the device control device 200 by the internal bus 206. The internal device 400 may use a touch panel provided in the display device 210 as the input device 208.

The device control device 200 is an office device such as a printer or a multifunction device, for example. However, the device control device 200 is not limited to an office device, and may be various embedded devices, mobile phones, or mobile terminal devices. May be.
<About Virtual Device Identification Information 204>

Next, the virtual device identification information 204 will be described. Usually, since the display device 210 that is the internal device 400 does not have information corresponding to the individual identification information of the external device 300, the information processing device 100 and the virtual composite device control unit 211 may recognize the internal device 400. Can not.

Therefore, in the present embodiment, the device control apparatus 200 assigns the virtual device identification information 204 corresponding to the individual identification information to each of the input apparatus 208 and the display apparatus 210 and displays it on the information processing apparatus 100 and the virtual composite device control unit 211. The apparatus 210 is recognized as a USB device similar to the external device 300. Here, virtual device identification information 204 is obtained by artificially creating configuration information (hereinafter referred to as “USB descriptor”) that defines the data structure of a USB device. The virtual device identification information 204 is stored in advance in the virtual device control unit 203, but may be generated by the virtual device control unit 203.

FIG. 2 is a diagram showing an example of the virtual device identification information 204 shown in FIG. 1, and the virtual device identification information 204 is composed of a USB descriptor.

The pseudo USB descriptor shown in FIG. 2 includes a vendor ID, a product ID, a serial number (not shown), a class, a subclass, a protocol, and an interface.

The vendor ID is identification information assigned to each company or manufacturer, and the product ID is a code assigned by the company or manufacturer having the vendor ID so as not to be duplicated for each product or model. In this embodiment, the vendor ID and product ID assigned to the device control apparatus 200 are used.

Also, the class, subclass, protocol, and interface are information defined by the USB standard according to the type of device such as a keyboard, mouse, and display. Then, a pseudo USB descriptor that associates the vendor ID and product ID with the class, subclass, protocol, and interface is created, and the virtual device control unit 203 holds (stores) it as the virtual device identification information 204.

The input device 208 of this embodiment corresponds to a keyboard and a mouse, and the display device 210 corresponds to a display. In order to recognize the keyboard, mouse, and display as USB devices, pseudo USB descriptors (that is, virtual device identification information 204) are assigned to the keyboard, mouse, and display, respectively.
<About Virtual Composite Device Identification Information 212>

Subsequently, the virtual composite device identification information 212 will be described. Similar to the virtual device identification information 204 described above, the virtual composite device 500 does not have information corresponding to the individual identification information of the device. Therefore, the information processing apparatus 100 cannot recognize the virtual composite device 500.

In this embodiment, the virtual composite device control unit 211 assigns the virtual composite device identification information 212 corresponding to the individual identification information to the virtual composite device 500, and processes the display device 210 and the external device 300 as one virtual composite device 500. The apparatus 100 is made to recognize. Here, a virtual composite device identification information 212 is created by pseudo-creating a USB descriptor of the virtual composite device 500. The virtual composite device identification information 212 is stored in advance in the virtual composite device control unit 211, but may be generated by the virtual composite device control unit 211.

FIG. 3 is a diagram showing an example of the virtual composite device identification information 212 shown in FIG. 1, and the virtual composite device identification information 212 includes a USB descriptor.

The pseudo USB descriptor shown in the upper part of FIG. 3 is virtual composite device identification information when the virtual composite device 500 is generated by combining the external device 300 corresponding to the keyboard and the display device 210 of the internal device 400 corresponding to the display. It is.

The virtual composite device control unit 211 newly assigns a vendor ID, product ID, serial number (not shown), class, subclass, protocol, and interface, and holds (stores) it as virtual composite device identification information 212.
<Device Configuration of Virtual Composite Device 500>

FIG. 4 is a diagram showing a device configuration of a virtual composite device 500 in which a plurality of devices are virtualized as one virtual composite device in the device control system shown in FIG.

The virtual composite device control unit 211 configures device configuration information of the virtual composite device 500 based on the device configuration information of the display device 210 of the external device 300 and the internal device 400, and sets a plurality of devices as one virtual composite device 500. Constitute. As illustrated in FIG. 4, when the endpoints of the external device 300 and the display device 210 overlap, the virtual composite device 500 is transferred to another endpoint so that the endpoints do not overlap, and the correspondence relationship is changed. Remember. In the example of FIG. 4, the end point (0x81) of the external device 300 and the end point (0x81) of the display device 210 overlap. Therefore, in the virtual composite device 500, the endpoint of the display device 210 is transferred to 0x83, for example, the correspondence between the endpoint (0x81) of the display device 210 as the internal device 400 and the endpoint (0x83) of the display device 210 as the virtual composite device I remember the relationship.

When the virtual composite device control unit 211 configures the device configuration information of the virtual composite device 500, the virtual composite device control unit 211 connects the virtual composite device 500 to the information processing apparatus 100 via the virtualization control unit 202. In the information processing apparatus 100, the virtual composite device 500 is mounted using a driver (not shown) of the virtual composite device 500 incorporated in advance in the OS, and the external device 300 and the display device 210 are recognized as if they were one device. can do.
<Control Sequence of Virtual Composite Device 500>

FIG. 5 is a sequence diagram for explaining an operation when the information processing apparatus 100 performs virtualization control of the virtual composite device 500 in the device control system shown in FIG. Here, a case where the display device 210 of the internal device 400 and the external device 300 are used as one virtual composite device 500 will be described.

First, the user starts virtualization control of the display device 210 of the internal device 400 by a user operation from the device control device 200 (step S101). Specifically, the user causes the display device 210 to display an operation screen by a predetermined operation, operates the input device 208 of the internal device 400, and displays the display device 210 and the external device 300 of the internal device 400 constituting the virtual composite device 500. To start connection to the virtual device control unit 203. Here, the selectable external device 300 is based on the device driver 102 stored in the information processing apparatus 100. In addition, although the description has been given on the assumption that the internal device 400 and the external device 300 are selected in order to configure the virtual composite device 500, the internal device 400 or the external device 300 does not select the device name of the internal device 400 or the external device 300. It is also possible to select a predetermined function to be performed.

The display device 210 transmits a connection request as USB data to the virtual device control unit 203 by a predetermined operation of the user (step S102).

The virtual composite device control unit 211 receives the virtual device identification information 204 related to the display device 210 as USB data (step S103). At this time, not all USB devices constituting the virtual composite device 500 selected in step S101 are connected. Therefore, the device connection notification is not sent to the virtualization control unit 202. However, when the display device 210 is configured as a single virtual composite device 500, device connection notification is made and virtualization control is started.

In addition, the internal device 400 or the external device 300 that is not selected as a device constituting the virtual composite device 500 can coexist with the virtual composite device 500. Here, coexistence means managing the virtual composite device 500 and other devices in separate sessions, and the information processing apparatus 100 distinguishes and recognizes the virtual composite device 500 and other devices. In such a case, the virtual device control unit 203 notifies the virtualization control unit 202 of connection of coexisting devices, and virtualization control is performed between the device control apparatus 200 and the information processing apparatus 100.

Subsequently, it will be described that the virtualization control is started by locally connecting the external device 300 to the device control apparatus 200 via a USB cable.

The user connects the external device 300 to the device control apparatus 200 (step S104). As a result, the individual identification information 301 of the external device 300 is transmitted as USB data to the device control apparatus 200 via the communication I / F 302 and the virtual composite device control unit 211 is notified that the external device 300 is connected. (Step S105).

The virtual composite device control unit 211 stores the USB data received from the display device 210 and the external device 300 in the endpoint corresponding to the interface of the virtual composite device 500 based on the configuration of FIG. Accordingly, the virtual composite device control unit 211 converts the USB data received from each device into the USB data format of one virtual composite device 500 by combining the USB data of the display device 210 and the USB data of the external device 300. Then, the virtual composite device identification information 212 is generated (step S106). In addition, although it illustrated that virtualization control was started when all the devices which comprise the virtual composite device 500 were gathered, it is not limited to this. When the display device 210 is selected, in order to perform virtualization control as a normal internal device 400, a connection notification to the information processing device 100 is performed, and when the external device 300 is connected, the display device 210 and It is also possible to notify connection of the external device 300 as one virtual composite device 500.

The virtual composite device control unit 211 sends the virtual composite device identification information 212 of the virtual composite device 500 to the virtualization control unit 202 to notify the connection (step S107). Subsequently, the virtualization control unit 202 converts the USB data format into network packet data, and notifies the information processing apparatus 100 that the virtual composite device 500 is connected as network packet data via the communication control unit 201. (Step S108).

The virtualization control unit 103 of the information processing apparatus 100 notifies the device driver 102 of the virtual composite device identification information 212 related to the virtual composite device 500 received from the virtualization control unit 202 of the device control apparatus 200 (step S109). At this time, the communication control unit 104 converts the data format of the network packet received from the device control apparatus 200 into the USB data format of the virtual composite device 500.

The device driver 102 transmits a notification of the connection of the virtual composite device 500 to the application 101 (step S110). When the application 101 receives a notification of the connection of the virtual composite device 500, the application 101 instructs the virtualization control unit 103 to start virtualization control of the virtual composite device 500. The virtualization control unit 103 Start virtualization control. Then, the information processing apparatus 100 generates and activates software components necessary for virtualization control, and starts a connection with the virtual composite device 500 using these software components.

When the connection between the virtual composite device 500 and the application 101 is started, the application 101 captures the display screen of the display device in the virtual composite device 500 and acquires the display screen (step S111).

The application 101 transmits the acquired display screen of the virtual composite device 500 to the device driver 102 (step S112). The device driver 102 transmits the display screen received from the application 101 to the virtualization control unit 103 (step S113).

The virtualization control unit 103 converts the display screen into network packet data and transmits it to the virtualization control unit 202 of the device control apparatus 200 (step S114). The virtualization control unit 202 transmits the display screen as USB data of the virtual composite device 500 to the virtual composite device control unit 211 (step S115).

Based on the configuration of FIG. 4, the virtual composite device control unit 211 determines which device (internal device 400 or external device 300) the data received based on the USB data endpoint of the virtual composite device 500 is. Determine and determine a device to transmit control information. If the endpoints are duplicated, the determination is made based on the correspondence stored in the virtual composite device control unit 211.

In the sequence of FIG. 5, the USB data of the virtual composite device 500 received as the display screen information is converted into the USB data format of the display device 210 virtualized by the virtual composite device control unit 211, and the converted display screen information is displayed. Is transmitted to the virtual device control unit 203 (step S116).

The virtual device control unit 203 transmits a display screen to the display device 210 of the internal device 400. At this time, the data format conversion processing unit 205 of the virtual device control unit 203 converts the USB data sent from the virtual composite device control unit 211 into the data format of the display device 210, and displays the display device 210 via the display I / F 209. (Step S117).

Then, the display device 210 displays a screen corresponding to the screen data sent from the information processing device 100. At this time, the display device 210 displays the screen data (received display screen) sent from the information processing device 100 and waits for input to the information processing device 100 (step S118).

If the USB data sent from the virtualization control unit 202 can be passed through the display I / F 209 without data conversion, it is sent to the display device 210 via the display I / F 209 without data conversion. Further, in the case where not all the acquired screen data is transmitted in the above-described step S115 but only the difference from the previously transmitted display image is transmitted, the difference display screen is added to the already received display screen. Are combined and displayed.

The user inputs operation information for remotely operating the information processing apparatus 100 using the input device of the external device 300 (step S119).

Subsequently, the external device 300 transmits operation information as USB data to the virtual composite device control unit 211 via the communication I / F 302 of the external device 300 (step S120).

The virtual composite device control unit 211 transmits the received operation information (USB data) to the virtualization control unit 202 (step S121). At this time, the virtual composite device control unit 211 converts the operation information (USB data) received from the external device 300 into the data format of the virtual composite device 500 and transmits it to the virtualization control unit 202.

The virtualization control unit 202 converts the USB data format into network packet data, and transmits the received operation information as network packet data to the virtualization control unit 103 of the information processing apparatus 100 via the communication control unit 201 ( Step S122). The virtualization control unit 103 of the information processing apparatus 100 transmits the received operation information to the device driver 102 (step S123). The device driver 102 transmits the operation information received from the virtualization control unit 103 to the application 101 (step S124).

Subsequently, the application 101 executes an operation (processing) according to the operation information received from the device driver 102 (step S125).

At this time, the information processing apparatus 100 displays the execution result of the process (function) on the display device 210. The image data displayed on the display device 210 is transmitted to the device control device 200 in the same manner as steps S113 to S120 described above, and image data sent from the information processing device 100 to the display device 210 of the device control device 200. (Display screen) is displayed.

When the connection between the virtual composite device 500 and the application 101 is started in step S110, the processing from step S111 to step S118 (screen transfer processing) and the processing from step S119 to step S125 are performed until the connection is disconnected (terminated). Repeat the process (remote operation process) asynchronously (in parallel). At this time, the virtualization control unit 202, the virtual device control unit 203, and the virtual composite device control unit 211 perform data transmission / reception between the internal device 400 and the information processing apparatus 100 via the network.

In step S101, the display device 210 of the internal device 400 has been described as selecting a device constituting the virtual composite device 500. However, it is also possible to select a device constituting the virtual composite device 500 from the external device 300. In addition, all of the connected internal devices 400 and external devices 300 among the devices that can configure the virtual composite device 500 incorporated in the OS in advance without selecting a device that configures the virtual composite device 500 are combined into one. The virtual composite device 500 may be used.

FIG. 6 illustrates an operation when the information processing apparatus 100 disconnects and reconnects to the virtual composite device 500 after the connection processing of the virtual composite device 500 illustrated in FIG. 5 in the device control system illustrated in FIG. It is a sequence diagram for demonstrating.

When the user ends the virtualization control of the virtual composite device 500 by the information processing apparatus 100, the user disconnects one of the devices constituting the virtual composite device 500. Here, a case where the external device 300 is disconnected will be described as an example.

The user operation on the external device 300 maintains the connection of the display device 210 to the virtual composite device control unit 211, and the external device 300 is disconnected (step S201). The external device 300 requests the virtual composite device control unit 211 to disconnect as USB data (first data format) (step S202). In response to the disconnection of one of the devices constituting the virtual composite device 500, the virtual composite device control unit 211 notifies the virtualization control unit 202 of the end of control as USB data of the virtual composite device 500 (step S203).

Subsequently, the virtualization control unit 202 notifies the virtualization control unit 103 of the information processing apparatus 100 that the control of the virtual composite device 500 is ended (control end notification) as network packet data (step S204). The virtualization control unit 103 notifies the device driver 102 of the control end notification received from the virtualization control unit 202 of the device control apparatus (step S205).

The device driver 102 notifies the application 101 of the control end notification received from the virtualization control unit 103 (step S206) and cancels the control of the virtual composite device 500. As a result, communication (connection) between the application 101 and the virtual composite device 500 is disconnected (released). Note that the virtualization control unit 202 does not cause the information processing apparatus 100 to recognize the device (here, the display device 210) that has not been disconnected after being disconnected as the virtual composite device 500 as the virtual composite device 500, instead of displaying it. It is also possible to recognize the apparatus 210 as the internal device 400.

Subsequently, when the external device 300 is connected again by a user operation on the external device 300 while maintaining the connection of the display device 210 to the virtual composite device control unit 211 (step S207), the virtual composite device control unit The fact that the external device 300 is connected to 211 is notified as USB data (step S208).

The virtual composite device control unit 211 generates the virtual composite device 500 again in response to all the devices constituting the virtual composite device 500 being connected (step S209). The virtual composite device control unit 211 notifies the virtualization control unit 202 of the start of control of the virtual composite device 500 as USB data of the virtual composite device 500 (step S210).

Subsequently, the virtualization control unit 202 notifies the virtualization control unit 103 of the information processing apparatus 100 that the control of the virtual composite device 500 has been started (control start notification) as network packet data (step S211). The virtualization control unit 103 notifies the device driver 102 of the control start notification received from the virtualization control unit 202 of the device control apparatus (step S212).

The device driver 102 notifies the application 101 of the control start notification received from the virtualization control unit 103 (step S213) and starts control of the virtual composite device 500. As a result, communication (connection) between the application 101 and the virtual composite device 500 is connected (started).

In FIG. 6, the disconnection instruction from the internal device 400 or the external device 300 of the virtual composite device 500 has been illustrated so far, but it is also possible to instruct the disconnection from the information processing apparatus 100 (not shown). In response to the disconnection instruction, the device control apparatus 200 ends the control of the virtual composite device 500.

The example in which the virtual composite device 500 is generated by combining the internal device 400 (display device 210) and the external device 300 (input device) has been described, but the virtual composite device 500 may be generated by combining a plurality of internal devices. Alternatively, the virtual composite device 500 may be generated by combining a plurality of external devices. Further, a plurality of virtual composite devices may be generated in the device control apparatus 200.

In this embodiment, in order for the information processing apparatus 100 to recognize a plurality of devices virtualized by the device control apparatus 200 as one virtual composite device 500, session management is performed in units of virtual composite devices instead of session management for each device. It becomes possible to do. Considering application to a cloud environment or the like, it is possible to manage connection or disconnection as a virtual composite device instead of session management for each device from the information processing apparatus 100 side, thereby improving convenience.

As described above, in the embodiment of the present invention, the information processing apparatus 100 performs virtualization control on the virtual composite device 500 in which the internal device 400 and the external device 300 provided in the device control apparatus 200 are combined, and uses the device driver 102. Thus, the behavior can be simulated in the same manner as when the virtual composite device 500 is directly connected (locally connected). Therefore, the virtual device 500 is controlled from the information processing device 100 by reconfiguring the internal device 400 and the external device 300 provided in the device control device 200 as the virtual composite device 500 and virtually connecting to the information processing device 100 ( Use).

Then, by using a plurality of devices as one virtual composite device 500, when a device is virtual connected to the information processing apparatus 100, a session required for each virtual connection device is assigned to the device control apparatus. It can be used to become one. This makes it possible to reduce the number of sessions in data communication. Further, by causing the information processing apparatus 100 to recognize the virtual composite device 500, it becomes possible to centrally manage sessions between the information processing apparatus and the device control apparatus.
[Second Embodiment]

Subsequently, an example of a device control system according to the second embodiment of the present invention will be described. In the second embodiment, not only an internal device included in the device control apparatus or an external device locally connected to the device control apparatus, but also a device included in or locally connected to another device control apparatus 200 on the network ( The virtual composite device 500 is characterized by an internal device and an external device.

FIG. 7 is a block diagram schematically showing a configuration of an example of a device control system according to the embodiment of the present invention.

In the illustrated device control system, an information processing apparatus 100 is connected to a device control apparatus 200A and a device control apparatus 200B via a network such as a local area network (LAN) or a wide area network (WAN). The LAN and WAN are configured by a wired line or a wireless line. The external device 300A and the external device 300B are locally connected to the device control device 200A and the device control device 200B, for example, via a communication interface (for example, USB or HDMI (registered trademark)). In the following description, the communication interface will be described as USB.
<Configuration of Information Processing Device 100>

Since the CPU, input unit, display unit, and memory are the same as those of the information processing apparatus 100 of the first embodiment, description thereof is omitted.

The communication unit is a network I / F for connecting to a LAN and a WAN, and performs data transmission / reception with the device control apparatus 200A and the device control apparatus 200B. As a result, the information processing apparatus 100 can exchange data with the external device 300A locally connected to the device control apparatus 200A and the internal device 400A and the internal device 400B provided in the device control apparatus 200A. In addition, data can be transmitted / received to / from the external device 300B locally connected to the device control apparatus 200B and the internal device 400C provided in the device control apparatus 200B.

The storage unit 110 stores an operating system that is software (hereinafter referred to as “OS”: not shown), an application 101, a device driver 102, a virtualization control unit 103, a communication control unit 104, and a device information notification unit 105. In addition, software relating to other functions is stored. These software are read into the memory and operate according to the control of the CPU.

The application 101 instructs the device driver 102 to perform a data input / output request, thereby combining an external device 300 connected to the device control apparatus 200 and an internal device 400 provided in the device control apparatus 200 (hereinafter referred to as a composite device). Software for controlling a virtual composite device). The combination of the internal device 400A, the internal device 400B, the internal device 400C, the external device 300A, and the external device 300B is arbitrary, and the number of devices is not limited to this.

The device driver 102 is a functional unit for controlling the external device 300, the internal device 400, and the virtual composite device 500. Data input / output requests from the OS or application 101 are made into data in the data format corresponding to each of the external device 300A, the external device 300B, the internal device 400A, the internal device 400B, and the internal device 400C (hereinafter referred to as “control command”). After conversion, the control command is sent to the virtualization control unit 103. Further, the device driver 102 notifies the application 101 of a response to the control command. The device driver 102 is generally software provided by a manufacturer, and the information processing apparatus 100 uses an external device 300A, an external device 300B, an internal device 400A, an internal device 400B, an internal device by the device driver 102 corresponding to each device. The device 400C and the virtual composite device 500 can be controlled.

The virtualization control unit 103 provides the same function as in the first embodiment.

Furthermore, the virtualization control unit 103 is a functional unit for controlling the external device 300, the internal device 400, and the virtual composite device 500. A function for simulating the same behavior as when the external device 300, the internal device 400, and the virtual composite device 500 are directly connected (locally connected) to the information processing apparatus 100 in response to data transmission / reception requests to these devices. (Hereinafter referred to as “virtualization control function”). In the illustration of this embodiment, the external device 300A, the external device 300B, and the internal device 400C are configured as a virtual composite device 500. The virtualization control unit 103 simulates a behavior similar to that when the virtual composite device 500 is directly connected (locally connected) to the information processing apparatus 100 in response to data transmission / reception requests to the plurality of devices. A control function is provided. With this virtualization control function, the information processing apparatus 100 recognizes that the connection state of the virtual composite device 500 of the internal device 400A, the internal device 400B, the external device 300B, and the internal device 400C is the same as that in the local connection. Can send and receive data.

The communication control unit 104 provides the same function as in the first embodiment to the device control apparatus 200A.

Further, the communication control unit 104 provides functions similar to those of the first embodiment to the device control apparatus 200A.

The device information notification unit 105 manages virtual composite device configuration information required by the information processing apparatus 100, and notifies the virtual composite device configuration information to the device control apparatus 200A having the virtual composite device control unit 211. Here, the virtual composite device configuration information is information indicating what kind of device the virtual composite device 500 is configured from. The first condition of the virtual composite device configuration information is to satisfy the function request. It is also possible to set arbitrarily after the second condition. For example, the first condition that satisfies the function request is to set functions such as a video class, a storage class, and a print class. The second condition may set performance such as camera fps (frame per second), storage capacity, printing ppm (Page Per Minute), and the like.
<Configuration of Device Control Device 200A>

The device control apparatus 200A is, for example, a peripheral device such as a printer or a multifunction peripheral, and includes a CPU, a memory, a communication unit, and a storage unit (both not shown) as hardware configurations, and the internal device 400A and the internal device. 400B is provided. Details of the configuration of the internal device 400A and the internal device 400B will be described later. In the present embodiment, the device control apparatus 200A plays a role of generating a virtual composite device.

Since the CPU, the input device 208, the display device 210, and the memory are the same as those of the device control device 200 of the first embodiment, description thereof is omitted.

The communication unit is a network I / F for connecting to a LAN and a WAN, and performs data transmission / reception with the information processing apparatus 100 and the device control apparatus 200B.

The storage unit provides the same function as in the first embodiment.

The communication control unit 201 controls transmission / reception of input / output data and the like performed between the device control apparatus 200A and the information processing apparatus 100 or the device control apparatus 200B.

The virtualization control unit 202 can control the information processing apparatus 100 to recognize the internal device 400A, the internal device 400B, and the external device 300A as if they were directly connected (local connection) to the information processing apparatus 100. Therefore, control is performed according to (in conjunction with) a control request from the virtualization control unit 103 of the information processing apparatus 100.

The virtual device control unit 203 includes virtual device identification information 204 and a data format conversion processing unit 205. Furthermore, the virtual device control unit 203 has a switching function such as a user interface for switching the internal device 400A and the internal device 400B provided in the device control apparatus 200A to a virtual USB device.

The data format conversion processing unit 205 performs conversion processing between packet data conforming to the USB data format sent from the virtual composite device control unit 211 and data in a data format that can be processed by the internal device 400A and the internal device 400B. Data can be transmitted / received between the virtual device control unit 203 and the input I / F 207 and the display I / F 209 to which the device 400A and the internal device 400B are connected.

The virtual composite device control unit 211 performs control for causing the virtualization control unit 202 to recognize and store an arbitrary plurality of devices (internal devices and external devices) as one virtual composite device. The virtual composite device control unit 211 stores virtual composite device identification information 212. In the example of the present embodiment, virtual composite device identification information 212 for configuring the external device 300A, the internal device 400C, and the external device 300B as virtual composite devices is provided. Further, the virtual composite device control unit 211 provides information as if the external device 300A, the internal device 400C, and the external device 300B configured as virtual composite devices are directly connected (locally connected) to the information processing apparatus 100. In order to allow the processing apparatus 100 to recognize and control, the control is performed according to (in conjunction with) a control request from the virtualization control unit 103 of the information processing apparatus 100.

The virtual composite device identification information 212 provides the same function as in the first embodiment.

The device configuration management unit 213 stores connection information between the external device 300 and the internal device 400 and manages functions provided in each device. A function or performance constituting the virtual composite device is selected by a connection notification of necessary device information from the information processing apparatus 100, and the virtual composite device control unit 211 is requested to create a virtual composite device. Further, the functional unit transmits and receives information about the configuration of the virtual composite device to and from the device control apparatus 200B via the communication control unit 201.
<Configuration of device control apparatus 200B>

The device control apparatus 200B is a peripheral device such as a network camera, for example, and includes a CPU, a memory, a communication unit, a storage unit (both not shown), and an internal device 400C as a hardware configuration. Details of the configuration of the internal device 400C will be described later. In this embodiment, when a virtual composite device is generated under the control of the device control apparatus 200A, the information processing apparatus 100 performs data transmission / reception with the device control apparatus 200A without being conscious of the device control apparatus 200B.

CPU controls the entire device control apparatus 200B. The video input device 221 is an image input device such as a network camera, for example. The memory is a RAM, a ROM, or the like.

The communication unit is an interface for connecting to the LAN and WAN, and performs data transmission / reception with the information processing apparatus 100 and the device control apparatus 200A.

The storage unit stores an OS (not shown), which is software, a communication control unit 214, a virtualization control unit 215, and a virtual device control unit 216, as well as software related to other functions. These software are read into the memory and operate according to the control of the CPU.

The communication control unit 214 controls transmission / reception of input / output data and the like performed with the device control apparatus 200A.

In order to allow the virtualization control unit 215 to recognize and control the information processing apparatus 100 as if the internal device 400C and the external device 300B are directly connected (local connection) to the information processing apparatus 100, Control is performed in accordance with (in conjunction with) a control request from the virtual composite device control unit 211 of the device control apparatus 200A.

The virtual device control unit 216 includes virtual device identification information 217 and a data format conversion processing unit 218.

The virtual device identification information 217 is used for causing the information processing apparatus 100 and the virtual device control unit 216 to recognize the internal device 400 in the same manner as the external device 300. Details of the virtual device identification information 217 will be described later.

The data format conversion processing unit 218 performs conversion processing between packet data conforming to the USB data format sent from the virtual device control unit 216 and data in a data format that can be processed by the internal device 400C, and is connected to the internal device 400C. Data transmission / reception between the image input I / F 220 and the virtual device control unit 216 is enabled.
<Configuration of External Device 300A>

The external device 300A includes individual identification information 301 and a communication I / F 302. The external device 300A is a device including a storage class storage device 303, and is externally connected to the device control device 200A via a USB interface.

The individual identification information 301 is information for identifying the external device 300A and is the same as that in the first embodiment, and thus the description thereof is omitted. Further, the communication I / F 302 is the same as that in the first embodiment, and thus the description thereof is omitted.
<Configuration of External Device 300B>

The external device 300B includes individual identification information 304 and a communication I / F 305. The external device 300B is a device that includes a vendor-class sensor device 306, and is externally connected to the device control device 200B via a USB interface. The individual identification information 304 is the same as the individual identification information 301 of the external device 300A, and the communication I / F 305 is the same as the communication I / F 302 of the external device 300A.

In the device control system shown in FIG. 7, the information processing apparatus 100, the device control apparatus 200A, the external device 300A, the device control apparatus 200B, and the external device 300B are shown, but the number of these is limited to the illustrated example. Is not to be done.
<Configuration of Internal Device 400A and Internal Device 400B>

The internal device 400A and the internal device 400B respectively include an input device 208 and a display device 210 that are used as user interfaces. Since the input device 208 and the display device 210 are the same as those in the first embodiment, description thereof is omitted.
<Configuration of Internal Device 400C>

The internal device 400C includes a video input device 221 used as a surveillance camera. The video input device 221 is connected to the internal bus 219 by the image input I / F 220, and is connected to the hardware of the device control device 200B by the internal bus 219.

The device control apparatus 200A is an office device such as a printer or a multifunction peripheral, and the device control apparatus 200B is a video input apparatus 221 such as a network camera. However, the device control apparatus 200A is not limited thereto. It may be various embedded devices, mobile phones or mobile terminal devices.
<Virtual Device Identification Information 204 and Virtual Device Identification Information 217>

Next, the virtual device identification information 204 and the virtual device identification information 217 will be described. Normally, the internal device 400A (input device 208), internal device 400B (display device 210), and internal device 400C (video input device 221) correspond to the individual identification information 301 and individual identification information 304 of the external device 300A and the external device 300B. Does not have information to do. Therefore, the information processing apparatus 100 cannot recognize the internal device 400A, the internal device 400B, and the internal device 400C.

Therefore, in this embodiment, virtual device identification information 204 and virtual device identification information 217 corresponding to individual identification information are allocated to each of the input device 208, the display device 210, and the video input device 221, and the USB device is assigned to the information processing device 100. Recognize as Here, virtual device identification information 204 and virtual device identification information 217 are obtained by artificially creating configuration information (hereinafter referred to as “USB descriptor”) that defines the data structure of the USB device. The virtual device identification information 204 and the virtual device identification information 217 are stored in advance in the virtual device control unit 203 and the virtual device control unit 216, but are generated by the virtual device control unit 203 and the virtual device control unit 216. May be.

FIG. 2 is a diagram showing an example of the virtual device identification information 204 and the virtual device identification information 217 shown in FIG. 1, and the virtual device identification information 204 and the virtual device identification information 217 are made up of USB descriptors.

The vendor ID is identification information assigned to each company or manufacturer, and the product ID is a code assigned by the company or manufacturer having the vendor ID so as not to be duplicated for each product or model. In this embodiment, the vendor ID and product ID assigned to the device control apparatus 200A and the device control apparatus 200B are used.

The class, subclass, protocol, and interface are information defined by the USB standard according to the type of device such as a keyboard, mouse, display, camera, storage, and sensor. Then, a pseudo USB descriptor in which the vendor ID and product ID are associated with the class, subclass, protocol, and interface is created, and the virtual device control unit 203 and the virtual device control unit 216 perform virtual device identification information 204, virtual Stored (stored) as device identification information 217.

In this embodiment, the input device 208 corresponds to a keyboard and mouse, the display device 210 corresponds to a display, the video input device 221 corresponds to a camera, the storage device 303 corresponds to storage, and the sensor device 306 corresponds to a sensor. In order to recognize the keyboard, mouse, display, camera, storage, and sensor as USB devices, a pseudo USB descriptor (that is, virtual device identification information 204, virtual, virtual) is assigned to each of the keyboard, mouse, display, camera, storage, and sensor. Device identification information 217) is assigned.
<About Virtual Composite Device Identification Information 212>

Next, the virtual composite device identification information 212 will be described. Since the internal device 400C, the external device 300A, and the external device 300B each have a single individual identification information 301, individual identification information 304, or virtual device identification information 217, the information processing apparatus 100 is regarded as one virtual composite device. It cannot be recognized.

Therefore, in this embodiment, the virtual composite device control unit 211 assigns the virtual composite device identification information 212 corresponding to the individual identification information to the virtual composite device, and the video input device 221, the storage device 303, and the sensor device 306 are combined into one virtual device. The information processing apparatus 100 is caused to recognize the composite device 500. Here, virtual composite device identification information 212 is obtained by artificially creating configuration information (hereinafter referred to as “USB descriptor”) that defines the data structure of a USB device. The virtual composite device identification information 212 is stored in advance in the virtual composite device control unit 211, but may be generated by the virtual composite device control unit 211.

The pseudo USB descriptor shown in the lower part of FIG. 3 includes the storage device 303 of the external device 300A corresponding to the storage, the video input device 221 of the internal device 400C corresponding to the monitoring camera, and the sensor device 306 of the external device 300B corresponding to the sensor. Is the virtual composite device identification information 212 of the virtual composite device 500 that combines the above.

The virtual composite device control unit 211 newly assigns a vendor ID, product ID, serial number (not shown), class, subclass, protocol, and interface, and holds (stores) it as virtual composite device identification information 212.

FIG. 8 is a diagram showing an example of a device configuration in which a plurality of devices are created as one virtual composite device identification information 212 in the device control system shown in FIG.

The virtual composite device control unit 211 creates device configuration information with a plurality of devices as one virtual composite device 500 based on the device configuration information of the external device 300A, the external device 300B, and the internal device 400C. As illustrated in FIG. 4, on the virtual composite device 500, the correspondence relationship is stored by switching to another interface and endpoint so that the interface and endpoint do not overlap.

In the example of FIG. 8, the end point (0x81) of the external device 300A, the end point (0x81) of the external device 300B, and the end point (0x81) of the internal device 400C overlap. Also, the endpoint (0x82) of the external device 300B and the endpoint (0x82) of the internal device 400C overlap. Therefore, in the virtual composite device 500, the endpoint of the external device 300B is transferred to 0x85, 0x86, 0x87, and the endpoint of the internal device 400C is transferred to 0x88, 0x89. For example, the correspondence relationship between the endpoint (0x81) as the sensor device 306 of the external device 300B and the endpoint (0x85) of the sensor device 306 as the virtual composite device is stored.

When the virtual composite device control unit 211 creates the device configuration information of the virtual composite device 500, the virtual composite device control unit 211 connects to the information processing apparatus 100 via the network I / F. In the information processing apparatus 100, the virtual composite device 500 is mounted using a driver (not shown) of the virtual composite device 500 incorporated in advance in the OS, the storage device 303 of the external device 300A, the sensor device 306 of the external device 300B, and The video input device 221 of the internal device 400C can be recognized as if it were one device.
<Control Sequence of Virtual Composite Device 500>

9A and 9B are sequence diagrams for explaining an operation when the information processing apparatus 100 performs virtualization control of the virtual composite device 500 in the device control system shown in FIG.

Here, the input device 208 and the display device 210 are operated from the state in which the input device 208 of the internal device 400A and the display device 210 of the internal device 400B are virtualized. A case where the storage device 303 of the external device 300A, the video input device 221 of the internal device 400C, and the sensor device 306 of the external device 300B are controlled as one virtual composite device 500 will be described.

Specifically, the sensor device 306 is a human sensor, the video input device 221 is a surveillance camera, and the storage device 303 is an external hard disk. It can be applied to scenes where the human sensor detects a person and starts shooting with the surveillance camera, and the shot video is saved to the hard disk. Instead of controlling the session for each device, one virtual Recognizing and controlling as the composite device 500 will be exemplified.

First, virtualization control of the input device 208 of the internal device 400A is started by a user operation from the device control device 200A (step S301).

The connection request is transmitted as USB data from the input device 208 to the virtual device control unit 203 by a predetermined operation of the user (step S302).

The virtual device control unit 203 creates virtual device identification information 204, which is individual identification information of the input device 208, and transmits a connection notification to the virtualization control unit 103 of the information processing apparatus 100 via the network I / F ( Step S303).

The virtualization control unit 103 selects the device driver 102 based on the individual identification information of the input device 208, and starts virtualization control (step S304).

The application 101 creates a data input / output path from the device driver 102 by a connection notification between the input device 208 and the display device 210 (step S305) (step S306).

Next, virtualization control of the display device 210 of the internal device 400B is started by a user operation from the device control device 200A (step S307).

The connection request is transmitted as USB data from the display device 210 to the virtual device control unit 203 by a predetermined operation of the user (step S308).

The virtual device control unit 203 creates virtual device identification information 204, which is individual identification information of the display device 210, and transmits a connection notification to the virtualization control unit 103 of the information processing device 100 via the network I / F ( Step S309).

The virtualization control unit 103 selects the device driver 102 based on the individual identification information of the display device 210, and starts virtualization control (step S310).

The application 101 creates a data input / output path from the device driver 102 according to the connection notification of the display device 210 (step S311) (step S312).

Further, the application 101 generates image data (display screen) to be displayed on the display device 210 of the device control apparatus 200A (step S313), and transmits the image data (display screen) to the device driver 102 (step S314).

The device driver 102 transmits the image data (display screen) to the virtualization control unit 103 (step S315). The virtualization control unit 103 converts a control command (that is, a data input / output request) received from the device driver 102 into packet data compliant with the USB data format (hereinafter referred to as “USB data”), and performs network I / F Is transmitted to the virtual device control unit 203 of the device control apparatus 200A (step S316).

The virtual device control unit 203 performs conversion processing with data in a data format that can be processed by the display device 210, and transmits and receives data via the display I / F 209 connected to the display device 210 (step S317). The image data (display screen) generated by the application 101 is displayed on the display device 210 as a received image (step S318).

The user inputs operation information via the input device 208 (step S319).

The input device 208 transmits the input operation information to the virtual device control unit 203 via the input I / F 209 connected to the input device 208 (step S320). The virtual device control unit 203 performs conversion processing between data in a data format that can be processed by the input device 208 and packet data that conforms to the USB data format, and transmits operation information to the virtualization control unit 103 (step S321).

The virtualization control unit 103 makes a response of a control command (that is, a data input / output request) to the device driver 102 (step S322). The device driver 102 transmits operation information of the input device 208 to the application (step S323).

The application 101 executes an operation based on the operation information received from the device driver 102 (step S324).

The user connects the storage device 303 of the external device 300A to the device control device 200A (step S325). Accordingly, the individual identification information 301 of the external device 300A is transmitted as USB data to the device control apparatus 200A via the communication I / F 302, and the virtualization control unit 202 is notified that the external device 300A is connected ( Step S326).

The virtualization control unit 202 transmits a connection notification (virtual device identification information 204) of the external device 300A to the device configuration management unit 213 of the device control apparatus 200A (step S327).

The device configuration management unit 213 stores and updates the storage function and performance of the storage device 303 of the external device 300A (step S328).

The user starts virtualization control of the video input device 221 of the internal device 400C by a user operation from the device control device 200B (step S330). A connection request is transmitted as USB data from the video input device 221 to the virtual device control unit 216 (step S331).

The virtual device control unit 216 creates virtual device identification information of the video input device 221 and transmits a connection notification (virtual device identification information 217) of the internal device 400C to the device configuration management unit 213 of the device control device 200A (step). S332).

The device configuration management unit 213 stores and updates the image input function and performance provided in the video input device 221 of the external device 300B (step S333).

The user connects the external device 300B to the device control apparatus 200B (step S334). Accordingly, the individual identification information 304 of the external device 300B is transmitted as USB data to the device control apparatus 200B via the communication I / F 305, and the virtualization control unit 215 is notified that the external device 300B is connected ( Step S335).

The virtualization control unit 215 transmits a connection notification (virtual device identification information 217) of the external device 300B to the device configuration management unit 213 of the device control apparatus 200A (step S336).

The device configuration management unit 213 stores and updates the sensor function and performance of the sensor device 306 of the external device 300B (step S337).

From here, the sequence for generating the virtual composite device 500 will be described.

An operation screen is displayed on the display device 210 according to a predetermined operation of the user, and the input device 208 is operated to select which function (storage, camera, sensor) and / or performance is configured as the virtual composite device 500 (step) S340). When only performance is selected, a function associated with performance is also selected. You can also select the device name as well as the function and performance.

At this time, when the internal device 400C and / or the external device 300B is configured as the virtual composite device 500, the device configuration management unit 213 of the device control apparatus 200A may notify the device control apparatus 200B to that effect.

It is also possible to transmit a request from the device control apparatus 200B to the device configuration management unit 213 of the device control apparatus 200A so as to configure the virtual composite device 500.

Then, each device control apparatus 200 may return a response as to whether or not each device control apparatus 200 can be configured as a virtual composite device 500 in response to a notification from the device control apparatus 200A or a request from the device control apparatus 200B.

Further, when there are a plurality of device control devices (internal devices, external devices), the device control device 200A can select a device constituting the virtual composite device 500 in response to a request from the device control device 200B. The device control apparatus 200B requests the device that cannot be configured as the virtual composite device 500, and the virtualization control unit 215 of the device control apparatus 200B performs virtualization control to perform data transmission / reception with the information processing apparatus 100.

The input device 208 transmits the operation information to the virtual device control unit 203 via the input I / F 207 connected to the input device 208 (step S341). The virtual device control unit 203 performs conversion processing between data in a data format that can be processed by the input device 208 and packet data that conforms to the USB data format, and transmits operation information to the virtualization control unit 103 (step S342).

The virtualization control unit 103 responds to the control command of the device driver 102 (step S343). The device driver 102 transmits operation information of the input device 208 to the application 101 (step S344).

The application 101 executes a selection operation of the virtual composite device 500 from the operation information received from the device driver 102 (step S345). At this time, virtual composite device configuration information is generated by the application 101. Note that a device that is not selected as a device constituting the virtual composite device 500 can coexist with the virtual composite device 500 as another virtual device. Here, coexistence means managing the virtual composite device 500 and other devices in separate sessions, and the information processing apparatus 100 distinguishes and recognizes the virtual composite device 500 and other devices.

At this time, when the internal device 400C and / or the external device 300B coexist with the virtual composite device 500, the virtual device identification information 217 of the device (internal device, external device) coexisting with the device control apparatus 200B is sent to the information processing apparatus 100. Send. As a result, the information processing apparatus 100 recognizes the device control apparatus 200B.

Also, the internal device 400C and / or the external device 300B are configured as the virtual composite device 500, and the remaining devices of the device control apparatus 200B that are not configured as the virtual composite device 500 do not transmit / receive data to / from the information processing apparatus 100 (virtualized as internal devices) If not, the device control apparatus 200A transmits the virtual composite device identification information 212 to the information processing apparatus 100. Then, the information processing apparatus 100 performs data transmission / reception with the device control apparatus 200A without recognizing the device control apparatus 200B.

The application 101 requests connection of the virtual composite device 500 selected above to the device information notification unit 105 (step S346). The device information notification unit 105 notifies the connection of the virtual composite device 500 to the device configuration management unit 213 of the device control apparatus 200A via the network I / F (step S347).

The device configuration management unit 213 selects a function or performance based on the device information, transmits virtual composite device configuration information to the virtual composite device control unit 211, and requests creation of the virtual composite device 500 (step S348).

The virtual composite device control unit 211 stores the USB data received from the storage device 303, the sensor device 306, and the video input device 221 in an endpoint corresponding to the interface of the virtual composite device 500 based on the configuration of FIG. Thereby, the virtual composite device control unit 211 converts the USB data received from each device into the USB data format of one virtual composite device 500 and converts the virtual composite device identification information, which is individual identification information of the virtual composite device The virtual composite device 500 is created by generating 212 (step S349). At this time, when the device (internal device, external device) of the device control apparatus 200B is configured as the virtual composite device 500 to create the virtual composite device 500, the device configuration management unit 213 of the device control apparatus 200A displays the device control apparatus 200B. May be notified that the virtual composite device 500 has been created.

The virtual composite device control unit 211 transmits a connection notification of the virtual composite device identification information 212, which is the individual identification information of the virtual composite device 500, to the virtualization control unit 103 of the information processing apparatus 100 via the network I / F (step). S350).

The virtualization control unit 103 notifies the device driver 102 of the virtual composite device identification information 212 received from the virtualization control unit 202 of the device control apparatus 200A (step S351).

The device driver 102 transmits a connection notification of the virtual composite device 500 to the application 101 (step S352). Here, when the application 101 receives a notification of connection of the virtual composite device 500, the application 101 recognizes the virtual composite device 500 (step S353), and the virtualization control unit 103 starts virtualization control of the virtual composite device 500 (step S354). ). Then, the information processing apparatus 100 generates and activates software components necessary for virtualization control, and starts a connection with the virtual composite device 500 using these software components.

By the sequence of step S301 to step S354, the information processing apparatus 100 can recognize each device as one virtual composite device 500, instead of recognizing each device individually. In the subsequent sequence, the information processing apparatus 100 recognizes the external device 300A, the external device 300B, and the internal device 400C as one virtual composite device 500.

From here, a sequence relating to control of data transmission / reception performed between the information processing apparatus 100 and the virtual composite device 500 will be described.

External device 300B inputs sensor information of sensor device 306 (step S360). Data is transmitted to the virtualization control unit 215 via the communication I / F 305 connected to the sensor device 306 (step S361). The virtualization control unit 215 performs conversion processing between data in a data format that can be processed by the sensor device 306 and packet data that conforms to the USB data format, and transmits the sensor information to the virtual composite device control unit 211 (step S362). .

Based on the configuration of FIG. 8, the virtual composite device control unit 211 determines whether the data received based on the endpoint of the virtual composite device 500 is the control information of the sensor device 306 and determines the endpoint that transmits the control information. To do. If the endpoints are duplicated, the determination is made based on the correspondence stored in the virtual composite device control unit 211.

Subsequently, the virtual composite device control unit 211 transmits the received sensor information (USB data) to the virtualization control unit 103 of the information processing apparatus 100 via the network I / F (step S363).

The virtualization control unit 103 transmits the received sensor information to the device driver 102 (step S364). The device driver 102 transmits the sensor information received from the virtualization control unit 103 to the application 101 (step S365). The application 101 analyzes the sensor information and starts image input (shooting) from the virtual composite device 500 (video input device 221) (step S366).

The internal device 400C inputs image information from the video input device 221 (step S370). Data is transmitted to the virtual device control unit 216 via the image input I / F 220 connected to the video input device 221 (step S371). The virtual device control unit 216 performs conversion processing between data in a data format that can be processed by the video input device 221 and packet data that conforms to the USB data format, and transmits image information to the virtual composite device control unit 211 (step S372). ).

Based on the configuration of FIG. 8, the virtual composite device control unit 211 determines whether the data received based on the endpoint of the virtual composite device 500 is the control information of the video input device 221, and determines the endpoint that transmits the control information. decide. If the endpoints are duplicated, the determination is made based on the correspondence stored in the virtual composite device control unit 211.

Subsequently, the virtual composite device control unit 211 transmits the received image information (USB data) to the virtualization control unit 103 of the information processing apparatus 100 via the network I / F (step S373).

The virtualization control unit 103 transmits the received image information to the device driver 102 (step S374). The device driver 102 transmits the image information received from the virtualization control unit 103 to the application 101 (step S375). The application 101 analyzes the image information and starts an image output instruction to the virtual composite device 500 (storage device 303) (step S376).

Application 101 starts outputting image information to virtual composite device 500 (storage device 303) (step S380). The application 101 transmits image information to the device driver 102 (step S381). The device driver 102 transmits image information to the virtualization control unit 103 (step S382).

The virtualization control unit 103 converts a control command (that is, a data input / output request) of the device driver 102 into USB data conforming to the USB data format, and performs a virtual composite device of the device control apparatus 200A via the network I / F. It transmits to the control part 211 (step S383).

Based on the configuration of FIG. 8, the virtual composite device control unit 211 determines whether the data received based on the end point of the virtual composite device 500 is control information of the storage device 303, and determines the end point for transmitting the control information. To do. If the endpoints are duplicated, the determination is made based on the correspondence stored in the virtual composite device control unit 211.

Subsequently, the virtual composite device control unit 211 transmits the received image information (USB data) to the storage device 303 via the communication I / F 302 (step S384). The storage device 303 stores the image information received from the virtual composite device control unit 211 (step S385).

From here, a sequence for disconnecting the virtual composite device 500 will be described.

An operation screen is displayed on the display device 210 according to a predetermined operation by the user, the disconnection operation is performed by operating the input device 208, and a disconnection request is transmitted as USB data to the virtual device control unit 203 (step S390). .

The input device 208 transmits the operation information to the virtual device control unit 203 via the input I / F 207 connected to the input device 208 (step S391). The virtual device control unit 203 performs conversion processing between data in a data format that can be processed by the input device 208 and packet data that conforms to the USB data format, and transmits operation information to the virtualization control unit 103 (step S392).

The virtualization control unit 103 responds to the control command of the device driver 102 (step S293). The device driver 102 transmits operation information of the input device 208 to the application 101 (step S394).

The application 101 executes a disconnection operation from the operation information received from the device driver 102 (step S395).

The application 101 transmits a disconnection request for the virtual composite device 500 to the device information notification unit 105 (step S396). The device information notification unit 105 sends a disconnection notification of the virtual composite device 500 to the device configuration management unit 213 of the device control apparatus 200A via the network I / F (step S397).

The device configuration management unit 213 selects a function being connected based on the composite device information, and requests the virtual composite device control unit 211 to extinguish the virtual composite device 500 (step S398).

The virtual composite device control unit 211 deletes the virtual composite device identification information 212 that is the individual identification information of the virtual composite device 500 (step S399).

In addition, although the example which combines the internal device 400C (video input device 221), the external device 300A (storage device 303), and the external device 300B (sensor device 306) to generate the virtual composite device 500 has been described, The virtual composite device 500 may be generated in combination. Further, the virtual composite device 500 may be generated by combining a plurality of external devices. Further, a plurality of virtual composite devices may be generated in the device control apparatus 200. Further, although the sensor device 306 is illustrated as a human sensor, the video input device 221 is illustrated as a monitoring camera, and the storage device 303 is illustrated as an external hard disk, the device is not limited thereto.

In the present embodiment, the function and performance of the virtual composite device 500 are selected from the information processing apparatus 100. However, the present invention is not limited to this and is selected from devices having input functions other than the information processing apparatus 100. May be.

As described above, in the second embodiment of the present invention, the information processing apparatus 100 performs virtualization control on the virtual composite device 500 in which the external device 300A, the internal device 400C, and the external device 300B are combined, and uses the device driver 102. The behavior can be simulated in the same manner as when the virtual composite device 500 is directly connected (locally connected).

Then, by using a plurality of devices as one virtual composite device 500, when a device is virtual connected to the information processing apparatus 100, a session required for each device to be virtualized is related to the number and configuration of the devices. It can be used so that there is only one. Thereby, the number of sessions in data communication can be reduced. In addition, by causing the information processing apparatus 100 to recognize the virtual composite device 500, it becomes possible to centrally manage sessions between the information processing apparatus 100 and the device control apparatus 200A.
[Third Embodiment]

Subsequently, an example of a device control system according to the third embodiment of the present invention will be described. When a device control apparatus included in the configuration of the virtual composite device combining any predetermined internal device or external device does not exist on the network, when another device control apparatus that satisfies the function request on the network exists, There is a case where a virtual composite device is generated using an existing device control apparatus and is used in an information processing apparatus. Therefore, in the third embodiment, the configuration of the virtual composite device and the connection destination are selected from the connection status of the information processing apparatus and the device control apparatus, and the virtual composite that is a predetermined function included in the device control apparatus according to the selected content. By controlling the device, an appropriate virtual composite device according to the status of the information processing apparatus is provided.

FIG. 10 is a block diagram schematically showing a configuration of an example of a device control system according to the third embodiment of the present invention.

In the illustrated device control system, an information processing apparatus 100 is connected to a device control apparatus 200 via a network such as a local area network (LAN) or a wide area network (WAN). The LAN and WAN are configured by a wired line or a wireless line. The external device 300 is locally connected to the device control apparatus 200 via, for example, a communication interface (for example, USB or HDMI (registered trademark)). In the following description, the communication interface will be described as USB.
<Configuration of Information Processing Device 100>

The CPU controls the entire information processing apparatus 100. The input unit includes, for example, a keyboard and a mouse. For example, a liquid crystal monitor is used as the display unit. The memory is RAM, ROM, or the like.

The communication unit is a network I / F for connecting to a LAN and a WAN, and performs data transmission / reception with the device control apparatus 200. As a result, the information processing apparatus 100 can transmit / receive data to / from the external device 300 locally connected to the device control apparatus 200, the internal device 400 provided in the device control apparatus 200, and the external device 300.

The application 101 instructs the device driver 102 to perform a data input / output request, thereby combining the external device 300 connected to the device control apparatus 200 and a composite device (hereinafter, “a combination of internal devices provided in the device control apparatus 200”). Software for controlling a virtual composite device). The combination of the external device 300 and the internal device 400 is arbitrary, and the number of devices is not limited to this.

The device driver 102 is a functional unit for controlling the external device 300, the internal device 400, and the virtual composite device. A data input / output request from the OS or the application 101 is converted into data in a data format corresponding to the external device 300, the internal device 400, and the virtual composite device (hereinafter referred to as “control command”), and the control command is converted into a virtual To the control unit 103. Further, the device driver 102 notifies the application 101 of a response to the control command. The device driver 102 is generally software provided by the manufacturer, and the information processing apparatus 100 controls the external device 300, the internal device 400, and the virtual composite device 500 by the device driver 102 corresponding to each device. Can do.

Furthermore, in response to a data transmission / reception request to the external device 300, the internal device 400, and the virtual composite device 500, the virtualization control unit 103 directly sends the external device 300, the internal device 400, and the virtual composite device to the information processing apparatus 100. A function (hereinafter referred to as “virtualization control function”) for simulating the same behavior as when connected to (local connection) is provided. With this virtualization control function, the information processing apparatus 100 has the same connection state of the external device 300, the internal device 400, and the virtual composite device 500 as when the external device 300, the internal device 400, and the virtual composite device 500 are locally connected. It is possible to transmit and receive data by recognizing that this is the state.

The communication control unit 104 controls the start and disconnection of a session between the device control apparatus 200 and the information processing apparatus 100 in response to a data transmission / reception request sent from the application 101 or the device driver 102 via the virtualization control unit 103. I do. Sessions between the device control apparatus 200 and the information processing apparatus 100 are generated for the number of devices that the information processing apparatus 100 performs virtualization control (for the number of internal devices, external devices, and virtual composite devices).

When the device information notification unit 105 manages virtual composite device configuration information required by the information processing apparatus 100 and is communicable with the device control apparatus 200 having the virtual composite device control unit 211, the information on the information processing apparatus 100 is displayed. (Hereinafter referred to as “own device information”) and required device information (hereinafter referred to as “required device information”) are requested to the communication control unit 104. Here, the virtual composite device configuration is requested. The information is information indicating what kind of device the virtual composite device is configured in. The virtual composite device configuration information is based on the first condition that the function request is satisfied. For example, the first condition that satisfies the function request is the video class, storage class, print class, etc. Is to set the power. The second condition is the camera fps (frame per second), the capacity of the storage, the performance may be set such ppm print (Page Per Minute).
<Configuration of Device Control Device 200>

The device control apparatus 200 is a peripheral device such as a printer or a multifunction peripheral, for example, and includes a CPU, a memory, a communication unit, and a storage unit (both not shown) as a hardware configuration and an internal device 400. . Details of the configuration of the internal device 400 will be described later.

The communication unit is a network I / F for connecting to a LAN and a WAN, and performs data transmission / reception with the information processing apparatus 100.

The storage unit stores an OS (not shown) that is software, a communication control unit 201, a virtualization control unit 202, a virtual device control unit 203, a virtual composite device control unit 211, and a device configuration management unit 213. The software related to the function is stored. These software are read into the memory and operate according to the control of the CPU.

The virtualization control unit 202 can control the information processing apparatus 100 to recognize the internal device 400, the external device 300, and the virtual composite device 500 as if they were directly connected (local connection) to the information processing apparatus. Therefore, the internal device 400, the external device 300, and the virtual composite device 500 are controlled in accordance with (in conjunction with) a control request from the virtualization control unit 103 of the information processing apparatus 100.

The virtual device control unit 203 includes virtual device identification information 204 and a data format conversion processing unit 205. Furthermore, the virtual device control unit 203 has a switching function such as a user interface that switches the internal device 400 provided in the device control apparatus 200 to a virtual USB device.

Further, the virtual composite device control unit 211 virtualizes a plurality of devices (internal device, external device) designated by the device configuration management unit 213 described later as one virtual composite device, and designated by the device configuration management unit 213. Control is performed to cause the virtualization control unit 202 to recognize the connection to the information processing apparatus 100.

The virtual composite device identification information 212 is used for causing the information processing apparatus 100 to recognize the virtual composite device. Details of the virtual composite device identification information 212 will be described later.

The device configuration management unit 213 stores connection information of the external device 300 and the internal device 400 and manages functions provided in each device. Based on the connection notification of necessary device information from the information processing apparatus 100, the function or performance constituting the virtual composite device is selected, and the virtual composite device control unit 211 is requested to create the virtual composite device.

Further, the device configuration management unit 213 manages information of each device connected to determine the generation and connection of the virtual composite device (hereinafter referred to as “connection device information”), and notifies the information processing device 100 of device information notification. Information (own device information, necessary device information) notified from the unit 105 is also managed as connection device information. The device information of the external device 300 and the internal device 400 that the device control apparatus 200 has is also managed as connection apparatus information.

Device information also manages information on external devices and internal devices held by other device control devices connected via the network. The device configuration management unit 213 updates the connection device information every time there is a change in the connection status of the information processing device 100, the external device 300, and the internal device 400 connected to the device control device 200.

If there is an information processing apparatus 100 that is not connected to a virtual composite device, an external device 300 and an internal device 400 that match the required device information of the information processing apparatus 100 are selected, and virtual composite device generation and connection are controlled by virtual composite device control. Request to unit 211.

When the virtual composite device control unit 211 completes generation and connection of the virtual composite device, the connection status of each apparatus is reflected in the connection apparatus information for the information processing apparatus 100, the external device 300, and the internal device 400 that are connection targets. To do. Here, “not connected to the virtual composite device” means that the virtual composite device needs to be generated again due to a change in the connection state of the information processing apparatus 100, the external device 300, and the internal device 400 with the device control apparatus 200. Say. However, it is not always necessary to configure the device control system with only the virtual composite device, and the external device 300, the internal device 400, and the virtual composite device can coexist.

It should be noted that the necessary devices are selected in accordance with various condition groups (hereinafter referred to as “device selection criteria”) for specifying the target device possessed by the device configuration management unit 213. The device selection criterion is that the first condition is to satisfy the function request of the necessary device information of the information processing apparatus 100, and the second and subsequent conditions can be arbitrarily set. After the second condition, the priority condition (when multiple devices are extracted under the first condition, the determination condition for selecting with priority) and the limitation condition (which satisfies the first condition but specific information) When only the processing apparatus 100 is used, there is a limitation (restriction or prohibition condition) for excluding it from the selection target, and a device is selected in combination with the first condition. The presented condition is an example after the second condition, and the selection condition can set various conditions necessary for selecting a device. And although satisfy | filling a function request | requirement was illustrated as 1st conditions, it is not limited to this.

Further, when a plurality of information processing apparatuses 100 are connected, which information processing apparatus 100 is preferentially processed depends on the priority of device selection processing held by the device configuration management unit 213. As the priority of device selection processing, various conditions necessary for determining the priority of processing can be set. Details of the device selection process will be described later.
<Configuration of External Device 300>

Next, the virtual device identification information 204 will be described. Normally, the input device 208 and the display device 210 that are the internal device 400 do not have information corresponding to the individual identification information 301 of the external device 300, so the information processing device 100 and the virtual composite device control unit 211 are not connected to the internal device 400. Cannot be recognized.

Therefore, in this embodiment, the virtual device identification information 204 corresponding to the individual identification information is recognized as the USB device similar to the external device 300 in the input device 208 and the display device 210. Here, virtual device identification information 204 is obtained by artificially creating configuration information (hereinafter referred to as “USB descriptor”) that defines the data structure of a USB device. The virtual device identification information 204 is stored in advance in the virtual device control unit 203, but may be generated by the virtual device control unit 203.

The vendor ID is identification information assigned to each manufacturer manufactured by each company, and the product ID is a code assigned by the company or manufacturer having the vendor ID so as not to be duplicated for each product or model. In this embodiment, the vendor ID and product ID assigned to the device control apparatus 200 are used.

Subsequently, the virtual composite device identification information 212 will be described. Similar to the virtual device identification information 204 described above, the virtual composite device does not have information corresponding to the individual identification information of the device. Therefore, the information processing apparatus 100 cannot recognize the virtual composite device.

In this embodiment, the virtual composite device control unit 211 assigns virtual composite device identification information 212 corresponding to individual identification information to a virtual composite device, and the information processing apparatus 100 uses the display device 210 and the external device 300 as one virtual composite device. To recognize. Here, a virtual composite device identification information 212 is a virtual composite device USB descriptor created in a pseudo manner. The virtual composite device identification information 212 is stored in advance in the virtual composite device control unit 211, but based on information on a plurality of devices (internal devices, external devices) designated by the device configuration management unit 213, the virtual composite device identification information 212 is stored. It may be generated by the device control unit 211.

The pseudo USB descriptor shown in the upper part of FIG. 3 is virtual composite device identification information when a virtual composite device is generated by combining the external device 300 corresponding to the keyboard and the display device 210 of the internal device 400 corresponding to the display. is there.

The virtual composite device control unit 211 configures the device configuration information of the virtual composite device 500 based on the device configuration information of the display device 210 of the external device 300 and the internal device 400, and sets a plurality of devices as one virtual composite device 500. Constitute. As illustrated in FIG. 4, when the endpoints of the external device 300 and the display device 210 overlap, the virtual composite device 500 is transferred to another endpoint so that the endpoints do not overlap, and the correspondence relationship is changed. Remember. In the example of FIG. 4, the end point (0x81) of the external device 300 and the end point (0x81) of the display device 210 overlap. Therefore, in the virtual composite device 500, the end point of the display device 210 is transferred to 0x83, for example, the correspondence between the end point (0x81) of the display device 210 as the internal device 400 and the end point (0x83) of the display device 210 as the virtual composite device I remember the relationship.

When the virtual composite device control unit 211 configures the device configuration information of the virtual composite device 500, the virtual composite device control unit 211 connects the virtual composite device 500 to the information processing apparatus 100 via the virtualization control unit 202. In the information processing apparatus 100, the virtual composite device 500 is mounted using a driver (not shown) of the virtual composite device 500 incorporated in advance in the OS, and the external device 300 and the display device 210 are recognized as if they were one device. can do.
<Control sequence of device control system>

11A and 11B are sequence diagrams for explaining an operation when the information processing apparatus 100 performs virtualization control of the virtual composite device 500 in the device control system shown in FIG.

First, virtualization control of the display device 210 of the internal device 400 is started by a user operation from the device control device 200 (step S401).

Specifically, the user displays an operation screen on the display device 210 by a predetermined operation, and operates (connects) the input device 208 of the internal device 400 to start connection to the virtual device control unit 203.

The connection request is transmitted from the display device 210 to the virtual device control unit 203 by a predetermined operation of the user (step S402).

The virtual composite device control unit 211 receives a connection notification of the display device 210 (virtual device identification information 204 of the display device 210) (step S403). At this time, not all devices constituting the virtual composite device are connected. Therefore, the device connection notification is not sent to the virtualization control unit 202.

The virtual composite device control unit 211 analyzes device information from the connection notification of the display device 210 received in step S503, and extracts device information managed by the connection device information of the device configuration management unit 213 (step S404).

The virtual composite device control unit 211 sends the device information of the display device 210 to the device configuration management unit 213 to notify the connection (step S405).

The device configuration management unit 213 adds the device information of the display device 210 received as the connection notification to the connection device information (step S406).

Subsequently, virtualization control is started by connecting the external device 300 to the device control apparatus 200 via a communication I / F 302 such as a USB interface.

First, the user connects the external device 300 to the device control apparatus 200 (step S407). As a result, a connection request for the external device 300 (individual identification information 301 of the external device 300) is transmitted to the device control apparatus 200, and the virtual composite device control unit 211 is notified that the external device 300 is connected (step). S408).

The virtual composite device control unit 211 analyzes device information from the received connection notification of the external device 300, and extracts device information managed by the connection device information of the device configuration management unit 213 (step S409).

The virtual composite device control unit 211 sends device information of the external device 300 to the device configuration management unit 213 to notify the connection (step S410).

The device configuration management unit 213 adds the device information of the external device 300 received as the connection notification to the connection device information (step S411).

Subsequently, by setting the information processing apparatus 100 and the device control apparatus 200 in a communicable state, the device information notification unit 105 of the information processing apparatus 100 notifies the device configuration management unit 213 of the device control apparatus 200 of its own device information and necessary information. The device information is transmitted to notify the connection (step S412).

The device configuration management unit 213 adds the own device information and necessary device information of the information processing device 100 received as the connection notification to the connection device information (step S413).

The device configuration management unit 213 selects a device of the virtual composite device 500 after updating the connection device information. In the device selection, the external device 300 and the internal device 400 suitable for the necessary device information of the information processing apparatus 100 are searched according to the device selection criteria to determine whether or not the connection device information exists (step S414).

In this embodiment, the following sequence will be described on the assumption that the display device 210 and the external device 300 are extracted by device selection.

In FIG. 5, the device selection (step S414) is described only once. However, the device selection may be performed every time the connection device information is updated.

As a result of device selection, the device configuration management unit 213 transmits device information that is a component of the virtual composite device 500 and information about the information processing apparatus 100 that is a connection destination to the virtual composite device control unit 211 to perform virtual composite The generation and connection of the device 500 is requested (step S415).

The virtual composite device control unit 211 stores the USB data received from the display device 210 and the external device 300 in the endpoint corresponding to the interface of the virtual composite device according to the result of device selection received from the device configuration management unit 213. Thereby, the virtual composite device control unit 211 converts the USB data received from each device into the USB data of the virtual composite device 500 and generates virtual composite device identification information 212 (step S416).

The virtual composite device control unit 211 sends the virtual composite device identification information 212 of the virtual composite device 500 to the virtualization control unit 202 to notify the connection (step S417). Subsequently, the virtualization control unit 202 notifies the information processing apparatus 100 that the virtual composite device 500 is connected (step S418).

The virtualization control unit 103 of the information processing apparatus 100 notifies the device driver 102 of the virtual composite device identification information 212 related to the virtual composite device 500 received from the virtualization control unit 202 of the device control apparatus 200 (step S419).

The application 101 receives a notification of connection of the virtual composite device 500 from the device driver 102 (step S420). Here, when the application 101 receives the notification of the connection of the virtual composite device 500, the application 101 instructs the virtualization control unit 103 to start virtualization control of the virtual composite device 500, and the virtualization control unit 103 causes the virtual composite device 500 to start. Start virtualization control. Then, the information processing apparatus 100 generates and activates software components necessary for virtualization control, and starts a connection with the virtual composite device 500 using these software components.

When the information processing apparatus 100 starts virtualization control of the virtual composite device 500, the application 101 captures the display screen of the virtual composite device 500 and acquires image data (display screen) (step S421).

The application 101 transmits the acquired image data (display screen) of the virtual composite device 500 to the device driver 102 (step S422).

The device driver 102 transmits the image data (display screen) received from the application 101 to the virtualization control unit 103 (step S423).

The virtualization control unit 103 transmits image data (display screen) to the virtualization control unit 202 of the device control apparatus 200 (step S424). The virtualization control unit 202 transmits image data (display screen) to the virtual composite device control unit 211 (step S425).

Based on the configuration of FIG. 4, the virtual composite device control unit 211 determines which device (internal device 400 or external device 300) the received data is based on the USB data endpoint of the virtual composite device 500. Then, the device that transmits the control information is determined. If the endpoints are duplicated, the determination is made based on the correspondence stored in the virtual composite device control unit 211.

In the sequence of FIG. 11, the USB data of the virtual composite device 500 received as image data (display screen) is converted into USB data of the display device 210 virtualized by the virtualization control unit 202, and the converted image data (Display screen) is transmitted to the virtual device control unit 203 (step S426).

The virtual device control unit 203 transmits image data (display screen) to the display device 210 of the internal device 400. At this time, the data format conversion processing unit 205 of the virtual device control unit 203 converts the USB data sent from the virtual composite device control unit 211 into the data format of the display device 210, and displays the display device 210 via the display I / F 209. (Step S427).

The display device 210 displays a screen corresponding to the screen data (display screen) sent from the information processing device 100. At this time, the display device 210 displays the screen data (display screen) sent from the information processing device 100 and waits for input to the information processing device 100 (step S428).

If the USB data sent from the virtualization control unit 202 can be passed through the display I / F 209 without data conversion, it is sent to the display device 210 via the display I / F 209 without data conversion. Further, in the case where not all the acquired screen data is transmitted in the above-described step S223 but only the difference from the previously transmitted display image is transmitted, here, the difference display screen is added to the already received display screen. Are combined and displayed.

The user inputs operation information for remotely operating the information processing apparatus 100 using the input device of the external device 300 (step S429).

Subsequently, the external device 300 transmits operation information to the virtual composite device control unit 211 via the communication I / F 302 of the external device 300 (step S430).

The virtual composite device control unit 211 transmits the received operation information (USB data) to the virtualization control unit 202 (step S431). At this time, the virtual composite device control unit 211 converts the operation information (USB data) received from the external device 300 into USB data of the virtual composite device 500 and transmits it to the virtualization control unit 202.

The virtualization control unit 202 transmits the received operation information to the virtualization control unit 103 of the information processing apparatus 100 (step S432). The virtualization control unit 103 of the information processing apparatus 100 transmits the received operation information to the device driver 102 (step S433). The device driver 102 transmits the operation information received from the virtualization control unit 103 to the application 101 (step S434).

Subsequently, the application 101 executes an operation (processing) according to the operation information received from the device driver 102 (step S435).

At this time, the information processing apparatus 100 displays the execution result of the process (function) on the display device 210. The image data displayed on the display device 210 is transmitted to the device control device 200 in the same manner as in steps S421 to S428 described above, and image data sent from the information processing device 100 to the display device 210 of the device control device 200. (Display screen) is displayed.

When the connection between the virtual composite device and the application 101 is started in step S420, the processing from step S421 to step S428 (screen transfer processing) and the processing from step S429 to step S435 are performed until the connection is disconnected (terminated). (Remote operation processing) is repeated asynchronously (in parallel). At this time, the virtualization control unit 202, the virtual device control unit 203, and the virtual composite device control unit 211 perform data transmission / reception between the internal device 400 and the information processing apparatus 100 via the network.

Subsequently, an operation when the external device 300 connected to the device control apparatus 200 is disconnected will be described.

When the user finishes the virtualization control of the virtual composite device 500 by the information processing apparatus 100, the user disconnects one of the devices (external device 300, internal device 400) constituting the virtual composite device 500. When the external device 300 is disconnected while maintaining the connection of the display device 210 (step S436), the external device 300 requests the virtual composite device control unit 211 to disconnect (step S437). In response to the disconnection of one of the devices constituting the virtual composite device 500, the virtual composite device control unit 211 notifies the virtualization control unit 202 of the end of control of the virtual composite device 500 (step S438).

Subsequently, the virtualization control unit 202 notifies the virtualization control unit 103 of the information processing apparatus 100 that the control of the virtual composite device 500 has ended (control end notification) (step S439). The virtualization control unit 103 notifies the device driver 102 of the control end notification received from the virtualization control unit 202 of the device control apparatus (step S440).

The device driver 102 notifies the application 101 of the control end notification received from the virtualization control unit 103 (step S441) and cancels the control of the virtual composite device 500. As a result, the communication (connection) between the application 101 and the virtual composite device is disconnected (released).

When the virtual composite device control unit 211 disconnects the connection between the information processing apparatus 100 and the external device 300, the virtual composite device control unit 211 analyzes the device information from the disconnection notification of the external device 300 received in step 537, and the connection apparatus information of the device configuration management unit 213 The device information managed in step S442 is extracted (step S442).

The virtual composite device control unit 211 sends device information of the external device 300 to the device configuration management unit 213 to notify the disconnection (step S443).

The device configuration management unit 213 identifies connection device information related to the external device 300 from the device information of the external device 300 received as the disconnection notification. As the update of the identified connection device information, the connection information of the information processing device 100 is updated (updated to a state not connected to the virtual composite device 500), and the connection information of the display device 210 is updated (becomes a component of the virtual composite device 500). The device information of the external device 300 is deleted (step S444).

If the device selection of the virtual composite device 500 is performed by updating the connection device information (step S414), and a device that satisfies the device selection criteria can be extracted from the necessary device information of the information processing device 100, the virtual composite device is generated and connected again. (Steps S415 to S420) are performed, but in this sequence, the case of one device control apparatus 200 and one external device 300 is illustrated, and thus description thereof is omitted.
<About device selection processing>

FIG. 12 is a flowchart for explaining device selection processing at the time of generating the virtual composite device 500 in the device control apparatus 200 in the device control system shown in FIG.

In the illustrated example, the device selection process is started by a change in connection status (connection or disconnection) with the information processing apparatus 100, the external device 300, or the internal device 400.

The device configuration management unit 213 of the device control apparatus 200 acquires information on the information processing apparatus 100, the external device 300, and the internal device 400 that are connected or disconnected, and reflects the information on the connected apparatus in the connection apparatus information that manages the information on the connected apparatus. (Step S501).

Next, the device configuration management unit 213 searches the connection device information for the information processing device 100 that is not connected to the virtual composite device 500, and if it exists, device information required by the target information processing device 100 (required) Device information). When a plurality of information processing apparatuses 100 are extracted by the search, the information processing apparatus 100 that is a target to be configured and connected to the virtual composite device 500 according to the priority order of the device selection process determined under an arbitrary condition is specified (step S502). ).

The device configuration management unit 213 determines whether there are available external devices 300 and internal devices 400 that satisfy the first condition of the device selection criteria (satisfy the function request of the necessary device information acquired in step S502) (step S102). S503).

When the external device 300 and the internal device 400 satisfying the first condition of the device selection criterion exist (Yes in step S503), the priority order from the external device 300 and the internal device 400 according to the second condition of the device selection criterion and thereafter. High external device 300 and internal device 400 are selected as components of the virtual composite device 500 (step S504).

On the other hand, when the external device 300 and the internal device 400 that satisfy the first condition of the device selection criteria do not exist (No in step S503), the process proceeds to step S507 described later. At this time, the information processing apparatus 100 may be notified that the external device 300 and the internal device 400 that satisfy the necessary device information of the information processing apparatus 100 do not exist.

Thereafter, the device configuration management unit 213 requests the virtual composite device control unit 211 to generate a virtual composite device for the external device 300 and the internal device 400 selected in step S504, and the virtual composite device control unit 211 is requested. The external device 300 and the internal device 400 are collected as a virtual composite device (step S505).

The virtual composite device control unit 211 connects the virtual composite device 500 generated in step S505 to the information processing apparatus 100 targeted in step S502. The device configuration management unit 213 reflects the connection status of the external device 300 and the internal device 400 that are constituent elements of the connected information processing apparatus 100 and the virtual composite device 500 in the connection apparatus information (step S506).

The device configuration management unit 213 determines whether there is an information processing apparatus 100 for which device selection has not been performed (step S507).

If there is an information processing apparatus 100 for which device selection has not been performed (Yes in step S507), the process proceeds to step S502 described above. If there is no information processing apparatus 100 for which device selection has not been performed, the device selection process is terminated.

As described above, in the third embodiment of the present invention, the information processing apparatus 100 notifies the device control apparatus 200 of information (self apparatus information, necessary device information) for generating the virtual composite device 500, and performs device control. The apparatus 200 manages the information notified from the information processing apparatus 100 and the device information of the connected external device 300 and internal device 400 as connection apparatus information, and determines the configuration and connection destination of the virtual composite device 500 from the contents of the connection apparatus information. By selecting and controlling the virtual composite device, the virtual composite device 500 required by the information processing apparatus 100 can be provided to the information processing apparatus 100.

The device control apparatus 200 dynamically determines the configuration of the virtual composite device 500 and the connection destination according to the connection status, so that the configuration of the virtual composite device 500 and the connection relationship between the information processing apparatus 100 and the virtual composite device 500 are determined. It is possible to provide the information processing apparatus 100 with an appropriate virtual composite device 500 according to the situation.
[Fourth Embodiment]

Subsequently, an example of a device control system according to the fourth embodiment of the present invention will be described. In the third embodiment, the device control apparatus 200 has been described as having a function of generating the virtual composite device 500. The fourth embodiment is characterized in that the virtual composite device 500 is generated by the mediation apparatus 600 that does not directly perform device control.

FIG. 13 is a block diagram schematically showing a configuration of an example of a device control system according to the fourth embodiment of the present invention. Note that the same components as those in the device control system shown in FIG. 10 are denoted by the same reference numerals and description thereof is omitted.

In the device control system shown in FIG. 13, the information processing apparatus 100, the device control apparatus 200, and the mediation apparatus 600 are connected via a LAN or WAN, and the external device 300 is connected to the device control apparatus 200 by USB. The device control apparatus 200 includes an input device 208 and a display device 210 as an internal device 400, and the internal device 400 is connected to the internal bus 206 by an input I / F 207 and a display I / F 209.

The device information notification unit 105 of the information processing apparatus 100 notifies the mediation apparatus 600 of the own apparatus information and necessary device information.

The communication control unit 201 of the device control apparatus 200A notifies the intermediary apparatus 600 of a connection request for the external device 300 and the internal device 400 that the self apparatus has, and the virtual composite device control unit 211 of the intermediary apparatus 600 determines the device from the received connection notification. Information is analyzed, and device information managed by the connection device information of the device configuration management unit 213 is extracted. The communication control unit 201 controls transmission / reception of input / output data and the like performed with the information processing apparatus 100, but all communication with the information processing apparatus 100 is performed via the mediation apparatus 600.

The device configuration management unit 605 of the mediation device 600 manages the device information notified from the device information notification unit 105 of the information processing device 100 and the communication control unit 201 of the device control device 200 with the connection device information, and is connected to the information processing. If there is an information processing apparatus 100 that is not connected to the virtual composite device 500, the connection apparatus information is updated each time the connection status of the apparatus 100, the external device 300, and the internal device 400 changes, and the information processing apparatus 100 is required. The external device 300 and the internal device 400 that match the device information are selected, and the virtual composite device control unit 603 is requested to generate and connect the virtual composite device 500.

In this embodiment, there are two information processing apparatuses 100, the information processing apparatus 100A and the information processing apparatus 100B. The external device 300 and the internal device 400 include four devices: an external device 300A possessed by the device controller 200A, an input device 208 of the internal device 400, a display device 210 of the internal device 400, and an external device 300B possessed by the device controller 200B. . The device configuration management unit 605 manages all device information, and selects devices that meet the conditions from the external device 300 and the internal device 400 that exist according to the device selection criteria from the necessary device information of each information processing apparatus 100.

The virtual composite device control unit 603 of the mediation apparatus 600 virtualizes the devices on the plurality of networks designated by the device configuration management unit 605 into one virtual composite device 500 based on the virtual composite device identification information 604, and the device configuration management unit Control that causes the virtualization control unit 602 to recognize the connection to the information processing apparatus 100 designated from 605 is performed. Note that the virtual composite device identification information 604 is the same as the virtual composite device identification information 212 described in the third embodiment, and a description thereof will be omitted.

Also, the virtual composite device control unit 603 of the mediation apparatus 600 receives input / output data to the virtual composite device 500 from the communication control unit 104 of the information processing apparatus 100 via the communication control unit 601 and the virtualization control unit 602. Then, it is decomposed into input / output data for the external device 300 and the internal device 400 that are constituent elements of the virtual composite device 500, and connected to the device control apparatus 200 (the external device 300 and internal devices of the constituent elements of the virtual composite device 500). 400 is transmitted to the communication control unit 201 of the device control apparatus 200).

Also, the virtual composite device control unit 603 of the mediation apparatus 600 receives input / output data from the communication control unit 201 of the device control apparatus 200 to the information processing apparatus 100 via the communication control unit 601, and the virtual composite device 500. Is transmitted to the communication control unit 104 of the information processing apparatus 100 via the virtualization control unit 602 and the communication control unit 601.

The communication control unit 201 of the device control apparatus 200 receives input / output data from the communication control unit 601 of the mediation apparatus 600 to the external device 300 and the internal device 400, and the virtualization control unit 202 and the virtual device control unit 203 use the external device. 300, the data is converted into a data format that the internal device 400 can recognize, and input / output data is transmitted to the external device 300 and the internal device 400.

The communication control unit 104 of the information processing apparatus 100 receives input / output data to / from the information processing apparatus 100 from the communication control unit 601 of the mediation apparatus 600, and a data format that the application 101 can recognize by the virtualization control unit 103 and the device driver 102. And notifies the application 101 as a response result from the device.

As described above, in the fourth embodiment of the present invention, the information processing apparatus 100 notifies the mediation apparatus 600 of information for generating a virtual composite device (self apparatus information, necessary device information), and the device control apparatus 200. Notifies the mediation device 600 of device information of the connected external device 300 and internal device 400, and the mediation device 600 manages the information of each device connected on the network with the connection device information. The virtual composite device is selected, the connection destination is selected, the virtual composite device is controlled using the external device 300 and the internal device 400 of the device control apparatus 200 connected via the network, and the information processing apparatus 100 needs A virtual composite device can be provided to the information processing apparatus 100.

A device that virtualizes the external device 300 and the internal device 400 to the information processing device 100 via the mediation device 600 by separating a device having a function of generating and controlling a virtual composite device such as the mediation device 600. A control system can be provided.

Note that the processing related to the virtual composite device performed by the device control apparatus in the first embodiment and the second embodiment is performed between the device control apparatus and the information processing apparatus by the mediation apparatus 600 described in the fourth embodiment. You may do it.

As mentioned above, although this invention was demonstrated based on said each embodiment, this invention is not limited to these embodiment, Various forms of the range which does not deviate from the summary of this invention are also included in this invention. It is.

For example, as a function control method of the above embodiment, the device control apparatus may execute this control method. Further, the program having the functions of the above-described embodiments may be used as a control program, and the control program may be executed by a computer included in the device control apparatus. The control program is recorded on a computer-readable recording medium, for example.

The present invention can also be realized by executing the following processing. That is, software (program code) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various recording media, and a computer (or CPU, MPU, or the like) of the system or apparatus performs program code. Is read and executed.

In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and a computer-readable storage medium storing the program code constitutes the present invention.

Further, the OS or the like running on the computer may perform part or all of the actual processing based on the instruction of the program code, and the functions of the above-described embodiments may be realized by the processing. .

Furthermore, after the program code read from the storage medium is written in the memory of the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. In some cases, the CPU or the like provided in the board or the function expansion unit executes part or all of the actual processing, and the above-described embodiment is realized according to the processing.

In order to supply the program code, for example, a storage medium such as a floppy (registered trademark) disk, a hard disk, a magneto-optical disk, an optical disk represented by CD or DVD, a magnetic tape, a nonvolatile memory card, or a ROM is used. Can do. Alternatively, the program code may be downloaded via a network.

100, 100A, 100B Information processing apparatus 101 Application 102 Device driver:
103 Virtualization control unit 104 Communication control unit 105 Device

information notification unit

200, 200A, 200B Device control device 201, 214

Communication control unit

202, 215

Virtualization control unit

203, 216 Virtual

device control unit

204, 217 Virtual device identification information 205 218 Data format

conversion processing unit

206, 219 Internal bus 207 Input I / F
208 Input device 209 Display I / F
210 Display Device 211 Virtual Composite Device Controller 212 Virtual Composite Device Identification Information 213 Device Configuration Manager 220 Image Input I / F
221

Video input device

300, 300A, 300B External device 301, 304

Individual identification information

302, 305 Communication I / F
303 Storage device 306

Sensor device

400, 400A, 400B, 400C Internal device 600 Mediation device 601 Communication control unit 602 Virtualization control unit 603 Virtual composite device control unit 604 Virtual composite device identification information 605 Device configuration management unit

Claims

A device control apparatus connected to an information processing apparatus via a network and recognizing at least one predetermined function as an internal device,
Identification information storage means for storing virtual composite device identification information for identifying as a virtual composite device composed of the internal device and / or an external device connected to the device control apparatus;
Identification information transmitting means for transmitting the virtual composite device identification information to the information processing apparatus;
When controlling the virtual composite device identified by the information processing apparatus based on the virtual composite device identification information in accordance with a control request from the information processing apparatus, the internal device or the external device and the virtual composite device Data communication between the virtual composite device and the device control device in a first data format, and data communication between the information processing device and the device control device in a second data format. Virtualization control means to control;
A device control apparatus comprising:
The device control apparatus is further connected to another device control apparatus via a network, and the device control apparatus recognizes at least one predetermined function in the device control apparatus itself and the other device control apparatus as an internal device. ,
The identification information storage means stores virtual composite device identification information for recognizing as a virtual composite device composed of the internal device and / or the device control apparatus itself or an external device connected to the other device control apparatus. And
2. The device control apparatus according to claim 1, wherein the virtualization control unit further performs control by performing data communication between the device control apparatus itself and the other device control apparatus in a second data format. .
The identification information storage means further stores virtual device identification information for identifying at least one predetermined function included in the device control apparatus or the other device control apparatus as an internal device,
The identification information transmitting means includes the internal device and / or the external device other than the virtual composite device configured based on one or more of the internal device and / or the external device selected according to a predetermined operation. The device control apparatus according to claim 2, wherein virtual device identification information is transmitted to the information processing apparatus.
The identification information transmitting means includes the virtual composite device identification information configured based on one or more internal devices and / or external devices recognized by a device control apparatus among the internal devices and / or the external devices. The device control apparatus according to claim 2, wherein the device control apparatus transmits the information to an information processing apparatus.
When the predetermined operation is performed in the internal device or the external device, the identification information transmitting unit transmits the virtual composite device identification information configured based on the predetermined operation to the information processing apparatus. The device control apparatus according to claim 2, characterized in that:
Device information management means for updating and storing connection device information for managing information on the own device and connection device held by the information processing device, the device control device itself and / or the other device control device;
Device configuration management means for selecting the configuration of the virtual composite device and the device that is an element thereof from the connection device information;
The device control apparatus according to claim 2, further comprising:
The device information management unit acquires or determines information on the information processing apparatus connected to or disconnected from the network, the device control apparatus itself and / or the own apparatus and the connected device held by the other device control apparatus. To update the connected device information,
The device configuration management unit is included in the information processing apparatus, the device control apparatus itself, and / or the other device control apparatus according to the configuration of the virtual composite device and the device that is an element thereof from the connection apparatus information. The device control apparatus according to claim 6, wherein the device controller requests the virtualization control unit to generate a plurality of the devices as the virtual composite device.
The device configuration management means, when the internal device and / or the external device configuring the virtual composite device includes one or more of the internal device and the external device of the other device control apparatus, The device control apparatus according to claim 6, wherein the device control apparatus notifies the other device control apparatus that it is configured as a device.
7. The device configuration management means receives from the other device control apparatus that the internal device and / or the external device included in the other device control apparatus is to be configured as a virtual composite device. The device control apparatus according to 1.
When the virtualization control means transmits / receives information for the control request, based on the correspondence between the virtual device identification information and the virtual composite device identification information, which internal device or the external device transmits / receives the information 4. The device control apparatus according to claim 3, wherein the device control apparatus determines and transmits and receives the information according to the determination result.
The device control apparatus according to claim 3, wherein the predetermined operation selects a function and / or performance of the internal device and / or the external device.
3. The device control apparatus according to claim 2, wherein when a predetermined end operation is performed in the internal device or the external device, the virtualization control unit ends the control of the virtual composite device.
When a predetermined end operation is performed in the information processing apparatus, the virtualization control unit that receives an instruction based on the end operation detects an instruction from the information processing apparatus and ends the control of the virtual composite device The device control apparatus according to claim 2.
When the control of the virtual composite device is finished, the identification information transmission unit is not subject to virtualization control termination by the predetermined termination operation among the internal device and / or the external device constituting the virtual composite device. 13. The device control according to claim 12, further comprising: transmitting virtual device identification information of a selected device or virtual composite device identification information including a device that has not been a target of termination of the virtualization control to the information processing apparatus. apparatus.
An intermediary device connected to an information processing device and a device control device via a network,
Identification information storage means for storing virtual composite device identification information for recognizing a function obtained by combining at least one predetermined function in the device control apparatus as a virtual composite device connected to the mediation apparatus;
Identification information transmitting means for transmitting the virtual composite device identification information to the information processing apparatus;
Device information management means for updating and storing connection device information for managing information of the own device and connection device held by the information processing device and the device control device;
Device configuration management means for selecting the configuration of the virtual composite device and the device that is an element thereof from the connection device information;
When the virtual composite device identified by the information processing apparatus based on the virtual composite device identification information is controlled in accordance with a control request from the information processing apparatus, a second operation is performed between the information processing apparatus and the device control apparatus. Virtualization control means for data communication and control in the data format;
An intermediary device comprising:
A control method of a device control apparatus connected to an information processing apparatus via a network and recognizing at least one predetermined function as an internal device,
An identification information storage step for storing virtual composite device identification information for identifying the internal composite device as a virtual composite device configured from the internal device and / or an external device connected to the device control apparatus;
An identification information transmission step of transmitting the virtual composite device identification information to the information processing apparatus;
When controlling the virtual composite device identified by the information processing apparatus based on the virtual composite device identification information in accordance with a control request from the information processing apparatus, the internal device or the external device and the virtual composite device Data communication between the virtual composite device and the device control device in a first data format, and data communication between the information processing device and the device control device in a second data format. Virtualization control steps to control;
A device control apparatus control method comprising:
A device control apparatus connected to another device control apparatus via a network, wherein the device control apparatus recognizes at least one predetermined function in the device control apparatus itself and the other device control apparatus as an internal device. Because
The identification information storing step stores virtual composite device identification information for recognizing as a virtual composite device composed of the internal device and / or the device control apparatus itself or an external device connected to the other device control apparatus. And
The device control apparatus according to claim 16, wherein the virtualization control step further performs control by performing data communication between the device control apparatus itself and the other device control apparatus in a second data format. Control method.
The identification information storing step further stores virtual device identification information for identifying at least one predetermined function included in the device control apparatus or the other device control apparatus as an internal device,
In the identification information transmission step, the internal device and / or the external device other than the virtual composite device configured based on one or more of the internal device and / or the external device selected according to a predetermined operation 18. The device control apparatus control method according to claim 17, wherein virtual device identification information is transmitted to the information processing apparatus.
In the identification information transmission step, the virtual composite device identification information configured based on one or more internal devices and / or the external devices recognized by a device control apparatus among the internal devices and / or the external devices The device control apparatus control method according to claim 17, wherein the device control apparatus transmits the information to an information processing apparatus.
When the predetermined operation is performed in the internal device or the external device, the identification information transmission step transmits the virtual composite device identification information configured based on the predetermined operation to the information processing apparatus. The method of controlling a device control apparatus according to claim 17, wherein:
A device information management step of updating and storing connection device information for managing information of the device itself and a connection device held by the information processing device, the device control device itself and / or the other device control device;
A device configuration management step of selecting a configuration of the virtual composite device and a device as an element thereof from the connection apparatus information;
The device control apparatus control method according to claim 17, further comprising:
The device information management step acquires or determines information of the information processing apparatus connected to or disconnected from the network, the device control apparatus itself and / or the own apparatus and the connected device held by the other device control apparatus. To update the connected device information,
The device configuration management step is included in the information processing apparatus, the device control apparatus itself, and / or the other device control apparatus according to the configuration of the virtual composite device and the device that is an element thereof from the connection apparatus information. The device control apparatus control method according to claim 21, wherein the virtualization control step is requested to generate a plurality of the devices as the virtual composite device.
In the device configuration management step, when the internal device and / or the external device configuring the virtual composite device includes any one or more of an internal device and an external device of the other device control apparatus, a virtual composite The method for controlling a device control apparatus according to claim 21, wherein the device control apparatus is notified that the device is configured as a device.
The device configuration management step receives from the other device control apparatus that the internal device and / or the external device included in the other device control apparatus is to be configured as a virtual composite device. A method for controlling the device control apparatus according to claim 1.
When the virtualization control step transmits / receives information for the control request, based on the correspondence between the virtual device identification information and the virtual composite device identification information, which internal device or the external device transmits / receives the information 19. The device control apparatus control method according to claim 18, wherein the information is transmitted and received according to the determination result.
19. The device control apparatus control method according to claim 18, wherein the predetermined operation selects a function and / or performance of the internal device and / or the external device.
18. The device control apparatus control method according to claim 17, wherein when a predetermined end operation is performed in the internal device or the external device, the virtualization control step ends the control of the virtual composite device. .
When a predetermined end operation is performed in the information processing apparatus, the virtualization control step that receives an instruction based on the end operation detects the instruction from the information processing apparatus and ends the control of the virtual composite device The device control apparatus control method according to claim 17.
When the control of the virtual composite device is terminated, the identification information transmission step is not subject to virtualization control termination by the predetermined termination operation of the internal device and / or the external device constituting the virtual composite device. 28. The device control according to claim 27, wherein virtual device identification information of a selected device or virtual composite device identification information including a device that has not been a target of termination of the virtualization control is transmitted to the information processing apparatus. Control method of the device.
A control method for an intermediary device connected to an information processing device and a device control device via a network,
An identification information storage step for storing virtual composite device identification information for recognizing a function combining at least one predetermined function in the device control apparatus as a virtual composite device connected to the mediation apparatus;
An identification information transmission step of transmitting the virtual composite device identification information to the information processing apparatus;
A device information management step of updating and storing connection device information for managing information of the own device and connection device held by the information processing device and the device control device;
A device configuration management step of selecting a configuration of the virtual composite device and a device as an element thereof from the connection apparatus information;
When the virtual composite device identified by the information processing apparatus based on the virtual composite device identification information is controlled in accordance with a control request from the information processing apparatus, a second operation is performed between the information processing apparatus and the device control apparatus. A virtualization control step for performing data communication and controlling in a data format of
A control method for an intermediary device, comprising:
A device control system in which the first device control apparatus according to claim 1, the second device control apparatus that recognizes at least one predetermined function as an internal device, and the information processing apparatus are connected via a network. And
The information processing apparatus further includes device information notification means for notifying the device control apparatus of own device information and necessary device information,
The device information notifying unit notifies the device control device of the device information of the information processing device and the information of the required device when communication with the device control device becomes possible, and the device control device. A device control system that requests connection of a required device to a device.
A device control system in which an intermediary device according to claim 15, a device control device that recognizes at least one predetermined function as an internal device, and an information processing device are connected via a network,
The information processing apparatus further includes device information notifying means for notifying the intermediary apparatus of its own device information and necessary device information,
The device information notifying means notifies the mediation device of the device information of the information processing device and the information of the required device when communication with the mediation device is possible, and is necessary for the mediation device. A device control system that requests connection of a device.