JP7100468B2 - Terminal devices and programs - Google Patents

Description

The present invention relates to a terminal device and a program for associating a content with a device for reproducing the content.

Conventionally, broadcasting stations hold various types of contents such as video, audio, and text, and provide these contents to viewers through devices such as televisions and smartphones (registered trademarks).

On the other hand, research and development on IoT (Internet of Things) devices are being actively carried out, and various IoT devices capable of reproducing contents such as video, audio, and text have been developed. IoT devices refer to devices connected to the Internet, such as personal computers, smartphones (registered trademarks), televisions, digital cameras, DVD players, sensors, and the like. Content providers such as broadcasting stations can provide content in various ways by using these IoT devices.

As a method for mechanically determining whether or not an IoT device can play content in a terminal device such as a smartphone (registered trademark), the Thing Description of the IoT device (hereinafter referred to as "TD"). There is a method of referencing the data of (see, for example, Non-Patent Document 1). That is, the application software held in the terminal device (hereinafter referred to as "application") determines whether or not the IoT device can play a predetermined content by referring to the TD of the IoT device. Can be done.

TD is metadata indicating the function of the IoT device and the like. By referring to the TD of the IoT device, the application can determine, for example, that the IoT device can play video content when the IoT device is a television. Further, for example, when the IoT device is a smart speaker, the application can determine that the IoT device can play audio content. Further, the application can determine that the IoT device can reproduce the text content, for example, when the IoT device is a communication robot having a text reading function.

"Web of Things (WoT) Thing Description, W3C Editor's Draft 26 February 2018", [online], February 26, 2018, W3C, [Search February 28, 2018], Internet <https: // w3c .github.io/wot-thing-description/>

As described above, the application held in the terminal device can mechanically determine what kind of content the IoT device can play by referring to the TD of the IoT device.

However, when the content is acquired, the application held in the terminal device can determine what kind of IoT device the content can be played back by only referring to the TD of the IoT device. Can not.

With such a conventional terminal device, it is not possible to easily identify an IoT device capable of playing the content based on the acquired content, and it is difficult to play various contents on various IoT devices. Met. Therefore, as a function of the terminal device, a simple mechanism for associating various IoT devices with various contents has been desired.

Therefore, the present invention has been made to solve the above-mentioned problems, and an object thereof is to reproduce various contents such as video, audio, text, control program, coupon, etc. on various IoT devices by a simple mechanism. It is to provide a terminal device and a program which can be made.

In order to solve the above-mentioned problem, the terminal device according to claim 1 is a terminal device that acquires content from a content server and causes an IoT (Internet of Things) device to perform playback processing related to the content. The IoT device connected by communication between them is searched, and for each of the connectable IoT devices, the device type indicating the type of the IoT device and the IoT from the response accompanying the search from the predetermined server or the search. A device function that indicates what kind of content the device can play, and a TD acquisition unit that acquires TD (Thing Description) that includes attribute data that indicates the type of file of the content that the IoT device plays. , The data of the content type indicating the type of the content, the genre indicating the field to which the content of the content belongs, and the data of the playable device indicating the type of the IoT device capable of reproducing the content from the predetermined server or the content. The CD acquisition unit that acquires the included CD (Content Description), the TD that includes the device type, the device function, and the attribute data acquired by the TD acquisition unit, and the CD acquisition unit. A device determination unit that determines the IoT device that actually reproduces the content among all connectable IoT devices based on the CD including the content type, the genre, and the data of the playable device. The IoT device determined by the device determination unit is provided with a content processing unit for performing reproduction processing related to the content.

Further, the terminal device according to claim 2 includes a content acquisition unit for acquiring a plurality of contents from the content server in the terminal device according to claim 1, and all the IoT devices to which the TD acquisition unit can be connected. The CD acquisition unit generates the usable device information including the device type, the device function, and the attribute of the IoT device, and the CD acquisition unit obtains all the contents acquired by the content acquisition unit. The device type and the device function included in the usable device information generated by the TD acquisition unit are generated by the device determination unit, which generates usable content information including the content type , the genre, and the playable device. And all the contents acquired by the content acquisition unit based on the attributes and the content type , the genre, and the reproducible device included in the usable content information acquired by the CD acquisition unit. Of all the connectable IoT devices, the IoT device capable of reproducing the content is determined as a playable IoT device, and among all the contents acquired by the content acquisition unit, the user Among the reproducible IoT devices, the IoT device to be actually reproduced is determined for the selected content.

Further, the terminal device according to claim 3 is a dummy CD generation unit that generates, as a dummy CD, a CD containing specific device data indicating that only the terminal device reproduces the content in the terminal device according to claim 2. When the CD acquisition unit cannot acquire the CD from both the predetermined server and the content for the content acquired by the content acquisition unit, the dummy CD is input from the dummy CD generation unit. The usable content information including the specific device data of the content is generated, and the device determination unit uses the content selected by the user among all the contents acquired by the content acquisition unit. When the specific device data is included in the possible content information, the terminal device is determined as a device that actually reproduces the content.

Further, in the terminal device according to claim 4, in the terminal device according to claim 2, all the connectable IoT devices include a television playing a predetermined video content, and the content acquisition. The unit acquires the related content related to the predetermined video content being played by the television from the content server, and the CD acquisition unit obtains the related content acquired by the content acquisition unit. When the usable content information including the content type , the genre, and the playable device of the content is generated, and the device determination unit selects the related content by the user, the playback of the related content is performed. Among the possible IoT devices, the IoT device that actually reproduces the related content is determined, and the content processing unit is determined by the device determination unit in a state where the television is playing the predetermined video content. The IoT device is characterized in that a reproduction process related to the related content is performed.

Further, the program of claim 5 is characterized in that the computer functions as the terminal device according to any one of claims 1 to 4.

As described above, according to the present invention, various contents such as video, audio, text, control program, coupon, etc. can be reproduced by various IoT devices by a simple mechanism.

It is a schematic diagram explaining the service by the matching application held in the terminal apparatus of embodiment of this invention. It is a schematic diagram which shows the hardware configuration example of a terminal apparatus. It is a block diagram which shows the structural example of a control part. It is a flowchart which shows the processing example of a control part. It is a figure explaining the screen display example of a terminal apparatus. It is a figure which shows the example of the data structure of TD. It is a figure which shows the data structure example of a CD. It is a block diagram which shows the structural example of the TD acquisition part. It is a flowchart which shows the processing example of the TD acquisition part. It is a block diagram which shows the structural example of the CD acquisition part. It is a flowchart which shows the processing example of the content acquisition part and the CD acquisition part. It is a block diagram which shows the structural example of a device determination part and a content processing part. It is a flowchart which shows the processing example of a device determination part and a content processing part. It is a figure explaining the description example of the matching application.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings. The present invention uses a content description (Content Description, hereinafter referred to as "CD"), which is metadata indicating the type of content, and a TD, which is metadata indicating the function of an IoT device, to provide content. It is characterized by determining what kind of IoT device can be reproduced. As a result, one or more IoT devices can be associated with one content, and as a result, various contents can be reproduced by various IoT devices by a simple mechanism.

In addition, by providing software for associating IoT devices with content using CDs and TDs, various contents can be handled in a unified application description without depending on the type. It will be possible. That is, it is possible to associate one or more IoT devices with one content by a simple mechanism.

[Outline of the present invention]
First, an outline of the service by the terminal device 1 according to the embodiment of the present invention will be described. FIG. 1 is a schematic diagram illustrating a service by a matching application held in the terminal device 1.

A broadcasting station or content server (not shown) distributes content such as video, audio, and text to the terminal device 1 and IoT devices 2-1 to 2-3 via the broadcasting / communication network 6.

The terminal device 1 is a device having a function as a gateway for connecting contents and other devices, such as a mobile terminal such as a smartphone (registered trademark) or a personal computer. By operating the matching application, the terminal device 1 reproduces the content on the desired IoT devices 2-1 to 2-3 based on the CD of each content and the TD of the IoT devices 2-1 to 2-3. Control for. That is, the terminal device 1 acquires content from a broadcasting station or a content server, searches for nearby IoT devices 2-1 to 2-3 using a communication protocol of WoT (Web of Things), and desires it. Have the IoT devices 2-1 to 2-3 perform the reproduction processing related to the acquired contents.

The IoT device 2-1 is a smart speaker that reproduces audio content, the IoT device 2-2 is a television that reproduces video content, and the IoT device 2-3 is a smart mirror that reproduces text content.

The terminal device 1 and the IoT devices 2-1 to 2-3 are connected by communication in accordance with a standard such as Wi-Fi (registered trademark) or Bluetooth (registered trademark).

For example, it is assumed that the user has a portable terminal device 1 and is viewing video content on the television of the IoT device 2-2 installed in the living room. By operating the matching application, the terminal device 1 acquires the video content being viewed, the audio content and the text content related to the video content via the broadcast / communication network 6, and the IoT device 2-1 to Search for 2-3. Then, the terminal device 1 acquires the CD of each content and the TD of the IoT devices 2-1 to 2-3, and associates the IoT devices 2-1 to 2-3 with each content based on the CD and TD. .. As a result, the audio content is associated with the IoT device 2-1 and the video content is associated with the IoT device 2-2, and the text content is associated with the IoT device 2-3.

When the user moves from the living room to the kitchen, the terminal device 1 detects the movement by a predetermined means, and the audio content related to the video content being played by the television of the IoT device 2-2 is installed in the kitchen. Play back on the smart speaker of the IoT device 2-1. The IoT device 2-2 receives the audio content from, for example, the content server and reproduces the received audio content, or reproduces the already received audio content according to the instruction from the terminal device 1.

As a result, the user can continuously listen to the audio of the audio content in the kitchen in response to the video content that was being viewed on the television of the IoT device 2-2.

Further, when the user moves from the kitchen to the washroom, the terminal device 1 detects the movement by a predetermined means, and the text content related to the video content being played by the television of the IoT device 2-2 is washed. Play it back on the smart mirror of the IoT device 2-3 installed in the place. The IoT device 2-3 receives the text content from, for example, the content server and reproduces the received text content, or reproduces the text content that has already been received, according to the instruction from the terminal device 1.

As a result, the user can continuously read the text of the text content in the washroom in response to the video content that was being viewed on the television of the IoT device 2-2.

In this way, even if the user moves in space, he / she can use the IoT devices 2-1, 2-3 at the destination and use the means suitable for the output function to provide the same kind of content as the video content being viewed. You can watch the content continuously.

Although FIG. 1 shows an example in which the terminal device 1 holds and operates the matching application, when the terminal device 1 does not exist, any one of the IoT devices 2-1 to 2-3 holds the matching application. It may be operated. Further, the matching application may exist on the cloud of the broadcasting / communication network 6. In this case, the cloud (device on the cloud) operates the matching application. Further, the IoT devices 2-1 to 2-3 may acquire the content via the terminal device 1, or may acquire the content directly from the broadcasting station or the content server.

[Hardware configuration]
Next, the hardware configuration of the terminal device 1 according to the embodiment of the present invention will be described. FIG. 2 is a schematic diagram showing a hardware configuration example of the terminal device 1. The terminal device 1 includes a CPU 11, a storage unit 12 including a ROM and a RAM for storing programs, tables, and the like, a storage device (for example, a hard disk device) 13 for storing programs, tables, data, and the like, and the terminal device 1. An operation / input unit 14 that inputs and controls predetermined data according to an operation of a touch panel or the like by a user, a display output interface unit 15 that displays screen information for prompting the user to perform a data input operation, the Internet, or the like. The broadcasting / communication network 6 and the communication unit 16 for transmitting / receiving contents, programs, data, etc. via wireless communication conforming to a standard such as Wi-Fi (registered trademark) are provided, and these components are a system. They are connected to each other via the bus 17.

The storage device 13 stores an OS (operating system) program that provides the basic functions of the terminal device 1, a matching application program, a Web browser program, various tables and data used in these programs, and the like. There is.

The matching application uses the content CD and the TD of the IoT device 2 to determine what kind of IoT device 2 the content can be played on, and plays the desired content on the desired IoT device 2. It is a program to make it.

The matching application is read from the storage device 13 to the RAM of the storage unit 12 by the CPU 11 and executed when the terminal device 1 performs processing. Further, various tables and data are generated by the execution of the matching application, are written from the RAM of the storage unit 12 to the storage device 13 by the CPU 11, and are written from the storage device 13 to the storage unit 12 by the CPU 11 with the execution of the matching application. Is read into the RAM of. The same applies to the Web browser.

Here, the OS program is executed by the CPU 11, and manages the storage unit 12, the storage device 13, the operation / input unit 14, the display output interface unit 15, and the communication unit 16 as basic functions of the terminal device 1. Then, while this OS program is executed by the CPU 11, the above-mentioned matching application and Web browser are executed.

The control unit 10 is composed of a CPU 11 and a storage unit 12, and the CPU 11 collectively controls the entire terminal device 1 by reading the matching application and the Web browser stored in the storage device 13 into the storage unit 12 and executing the control unit 10. FIG. 1 shows a state in which a matching application, a Web browser, or the like is read from the storage device 13 to the storage unit 12. As described above, in the terminal device 1, the control unit 10 performs various processes according to the matching application, the browser, and the like according to the hardware configuration shown in FIG.

[Control unit 10 (Control unit 10 when the matching application operates)]
Next, the control unit 10 when the matching application operates will be described. FIG. 3 is a block diagram showing a configuration example of the control unit 10 (terminal device 1), and shows a functional configuration when the matching application is operated. FIG. 4 is a flowchart showing a processing example of the control unit 10, and FIG. 5 is a diagram illustrating a screen display example of the terminal device 1.

With reference to FIG. 3, the control unit 10 includes a TD acquisition unit 20, a TD storage unit 21, a content acquisition unit 22, a content storage unit 23, a CD acquisition unit 24, a CD storage unit 25, a device determination unit 26, and content processing. It has a part 27. Hereinafter, the control unit 10 will be described with reference to FIGS. 3 to 5.

When the user who operates the terminal device 1 selects the matching application, the terminal device 1 activates the matching application (step S401).

The TD acquisition unit 20 of the control unit 10 searches for the connectable IoT device 2 (step S402), and acquires the TD from the connectable IoT device 2 or the TD server 3 (step S403). Then, the TD acquisition unit 20 stores the acquired TD in the TD storage unit 21.

The TD acquisition unit 20 acquires the TD for each connectable IoT device 2 and stores it in the TD storage unit 21. As a result, the TD storage unit 21 stores the TDs of all the IoT devices 2 to which the terminal device 1 can be connected.

FIG. 6 is a diagram showing an example of a TD data structure. This TD is metadata showing the functions and the like of the IoT device, which describes how the thing (Thing) is configured. The TD includes data such as a device ID (id), a device type (model), a device function (actions), and an attribute (properties).

The device ID indicates an identifier of the IoT device 2. The device type indicates the type of the IoT device 2, and is, for example, a speaker, a smart speaker, a television, a smart mirror, a robot, a gateway, or the like. .. The device function indicates a function of the IoT device 2 to play back content, that is, what kind of content can be played back, and is, for example, audio play (audio play), video play (movie play), and the like. The attribute indicates the type of the content file to be played by the IoT device 2, and is, for example, an audio file, a video file, or the like.

Returning to FIGS. 3 to 5, the TD acquisition unit 20 reads the TDs of all connectable IoT devices 2 from the TD storage unit 21, extracts the device ID, device type, device function, etc. from the TD, and IoT. These information for each device 2 are generated as usable device information (step S404). Then, the TD acquisition unit 20 outputs usable device information including the device functions and the like of all connectable (usable) IoT devices 2 to the device determination unit 26.

In this way, in the processes of steps S402 to S404 by the TD acquisition unit 20, the TDs of all connectable IoT devices 2 are acquired, and the usable device information including the data of these TDs is generated. The details of the processing of the TD acquisition unit 20 will be described later.

On the other hand, with the activation of the matching application in step S401, the content acquisition unit 22 acquires the content from the content server 4 (step S405). Then, the content acquisition unit 22 stores the acquired content in the content storage unit 23 and outputs the acquired content to the CD acquisition unit 24 and the content processing unit 27.

Further, the content acquisition unit 22 displays the names of all the acquired contents on the screen. For example, as shown in steps S405 and S406 of FIG. 5, the contents A, B, and C, which are the names of the contents acquired by the content acquisition unit 22, are displayed on the screen. As a result, the user can recognize all the contents A, B, and C acquired by the terminal device 1.

The content acquisition unit 22 may read the content already stored from the content storage unit 23 and output the read content to the CD acquisition unit 24 and the content processing unit 27.

The CD acquisition unit 24 inputs content from the content acquisition unit 22, and acquires a CD of the content from the content or the CD server 5 or the like (step S406). Then, the CD acquisition unit 24 stores the acquired CD in the CD storage unit 25.

The CD acquisition unit 24 acquires the CD for each content input from the content acquisition unit 22 and stores it in the CD storage unit 25. As a result, the CD storage unit 25 stores each CD for all the contents input from the content acquisition unit 22.

FIG. 7 is a diagram showing an example of a data structure of a CD. This CD is metadata indicating the type of content and the like, which describes how the content is organized. The CD includes data such as a content ID (id), a content type (media type), a genre (genre), and a playable device (device).

The content ID indicates an identifier of the content. The content type indicates the type of content, and is, for example, audio (audio), video (movie), text (text), control program (control program), and the like. The genre indicates the field to which the content belongs, for example, sports, documentary, and the like. The playable device shows data related to the IoT device 2 capable of playing the content, and corresponds to the device type of the TD shown in FIG. For example, when the content is played only on the terminal device 1 and not played on the other IoT device 2, or when the content is played only on the terminal device 1 and the specific IoT device 2, the other IoT device 2 If it is not played back, data for specifying the device to be played back is set.

Returning to FIGS. 3 to 5, the CD acquisition unit 24 reads the CD of all the contents acquired by the content acquisition unit 22 from the CD storage unit 25, and extracts the content type, playable device, and the like from the CD. Then, the CD acquisition unit 24 generates usable content information including the content types of all the acquired contents (step S407), and outputs this to the device determination unit 26.

In this way, the content is acquired by the process of step S405 by the content acquisition unit 22, and the CDs of all the acquired contents are acquired by the process of step S406 and step S407 by the CD acquisition unit 24. Then, the content types and the like of all the acquired contents are extracted from these CDs, and usable content information is generated. The details of the processing of the CD acquisition unit 24 will be described later.

The device determination unit 26 shifts from step S404 and step S407, inputs usable device information including device functions of all usable IoT devices 2 from the TD acquisition unit 20, and also from the CD acquisition unit 24. , Enter the usable content information including the content type of all the acquired contents.

The device determination unit 26 determines the IoT device 2 that can reproduce the content based on the usable device information and the usable content information for each content (step S408). Specifically, the device determination unit 26 compares the device function and the like of the IoT device 2 included in the usable device information with the content type and the like of the content included in the usable content information, and the IoT device 2 is the relevant one. Determine if the content is playable. Then, the device determination unit 26 determines the IoT device 2 capable of reproducing the content from the IoT devices 2 having all the device IDs included in the usable device information.

For example, in the device determination unit 26, when the device function of the IoT device 2 included in the usable device information is voice reproduction and the content type of the content included in the usable content information is voice, the IoT device 2 is used. It is determined that the content can be played. Further, in the device determination unit 26, when the device function of the IoT device 2 included in the usable device information is video reproduction and the content type of the content included in the usable content information is video, the IoT device 2 is used. It is determined that the content can be played. Further, when the device function of the IoT device 2 included in the usable device information is voice reproduction and the content type of the content included in the usable content information is a control program, the device determination unit 26 determines the IoT device 2. Determines that the content cannot be played.

The device determination unit 26 determines the IoT device 2 capable of reproducing the content for each content, and then determines the content for which the reproducible IoT device 2 is determined (existing) and the reproducible IoT device 2. Identify non-existent (non-existent) content. Then, the device determination unit 26 displays on the screen whether or not each of the acquired contents can be reproduced.

For example, as shown in steps S405 and S406 of FIG. 5, among the acquired contents A, B, and C, the contents A and B in which the reproducible IoT device 2 exists are marked with white circles meaning reproducible. Is displayed, and for the content C in which the reproducible IoT device 2 does not exist, a cross mark indicating that reproducibility is not possible is displayed. As a result, the user can recognize the reproducible contents A, B and the non-reproducible contents C among all the contents A, B, C acquired by the terminal device 1.

When the user selects the content to be actually played from the playable contents, the device determination unit 26 clearly indicates the content selected by the user on the screen. For example, as shown in the upper right part of FIG. 5, when the content A is selected from the playable contents A and B by the user, a black circle indicating that the content A is selected is displayed. As a result, the user can confirm that the content A has been selected from the playable contents A and B.

The device determination unit 26 determines the IoT device 2 that actually reproduces the content selected by the user among the playable IoT devices 2 (step S409). Specifically, the device determination unit 26 determines the IoT device 2 that is actually reproduced according to a selection operation by the user or a predetermined algorithm. Then, the device determination unit 26 outputs the determined information about the IoT device 2 to the content processing unit 27 as the reproduction device information.

For example, as shown in the upper right part of FIG. 5, when the content A is selected by the user, as shown in the part of step S408 of FIG. 5, the terminal device 1 and the IoT device 2 that can reproduce the content A A white circle indicating that the IoT device α can be reproduced is displayed, and a cross mark indicating that the content A cannot be reproduced is displayed for the IoT device β which is the IoT device 2 that cannot be reproduced. The IoT devices α and β are IoT devices 2 connected to the terminal device 1. As a result, the user can recognize the reproducible terminal device 1 and the IoT device α for the content A.

Then, as shown in step S409 of FIG. 5, when the user actually selects the terminal device 1 and the IoT device α for reproduction, the black circle indicating that the terminal device 1 and the IoT device α are selected. The mark is displayed. As a result, the user can confirm that the terminal device 1 and the IoT device α that actually perform reproduction have been selected.

The content processing unit 27 inputs the content selected by the user from the content acquisition unit 22, and inputs the playback device information regarding the device that actually reproduces the content from the device determination unit 26. Then, the content processing unit 27 executes the content and transmits the control message to the IoT device 2 (IoT device 2 determined in step S409) indicated by the playback device information (step S410). The content processing unit 27 may read the content selected by the user from the content storage unit 23.

Upon receiving the control message, the IoT device 2 operates according to the control message, performs reproduction indicated by the control message, and downloads and reproduces necessary contents from the content server 4. That is, the content processing unit 27 can execute the content selected by the user on the terminal device 1 and cause the IoT device 2 determined by the device determination unit 26 to perform reproduction processing related to the content. ..

For example, as shown in step S410 of FIG. 5, the video of the content A is reproduced on the screen of the terminal device 1. In the case of the example of FIG. 5, the content A is played back on the terminal device 1, and the playback process related to the content A is performed on the IoT device α according to the control message of the content A.

In this way, the IoT device 2 to be actually reproduced is determined by the processing of step S408 and step S409 by the device determination unit 26, and the content is actually reproduced by the processing of step S410 by the content processing unit 27. It is reproduced by the IoT device 2. The details of the processing of the device determination unit 26 and the content processing unit 27 will be described later.

(TD acquisition unit 20)
Next, the processing of steps S402 to S404 by the TD acquisition unit 20 will be described in detail. FIG. 8 is a block diagram showing a configuration example of the TD acquisition unit 20, and FIG. 9 is a flowchart showing a processing example of the TD acquisition unit 20.

The TD acquisition unit 20 includes a device management unit 30, a device communication unit 31, and a TD processing unit 32. The device management unit 30 periodically outputs a search request for the connectable IoT device 2 to the device communication unit 31 in order to manage the information of the connectable IoT device 2 (step S901).

The device communication unit 31 inputs a search request from the TD acquisition unit 20, executes a search for a connectable IoT device 2 according to a predetermined communication protocol corresponding to the IoT device 2, and receives a response from the connectable IoT device 2. Receive (step S902).

Here, it is assumed that there are two types of IoT devices 2, a WoT device and a non-WoT device. The WoT device is a device conforming to the WoT standard, and includes a Web browser and a Web server. Further, the WoT device holds a TD associated with the WoT device, or holds a TDID (TD identifier, for example, the URL of the TD server 3 in which the TD is stored) that uniquely determines the TD. The device communication unit 31 communicates with the WoT device by a method compliant with the WoT standard, and connects to the WoT device by the WoT device discovery protocol.

On the other hand, non-WoT devices do not hold TDs and TDIDs. The device communication unit 31 holds, for example, the IP address of the non-WoT device in advance, and by directly designating the IP address of the non-WoT device, the device communication unit 31 connects to the non-WoT device.

When the device communication unit 31 receives a response from the connectable IoT device 2, the device communication unit 31 outputs the response to the device management unit 30 and the TD processing unit 32. The device management unit 30 inputs a response from the device communication unit 31, and manages the IoT device 2 that has returned the response as a connectable device.

The TD processing unit 32 inputs a response from the device communication unit 31, and the response includes the TD, the response includes the URL (TDID) of the TD, or the response contains both the TD and the URL. It is determined whether or not the URL has been removed (step S903).

When the TD processing unit 32 determines that the response includes TD (step S903: TD), the TD processing unit 32 extracts the TD from the response (step S904), and uses the TD of the IoT device 2 that has returned the response. Judge that it is TD.

On the other hand, when the TD processing unit 32 determines that the response does not include the TD and includes the URL of the TD (step S903: URL), the TD processing unit 32 receives the TD from the TD server 3 of the URL (step S903: URL). S905). Then, the TD processing unit 32 determines that the TD is the TD of the IoT device 2 that has returned the response.

Further, when the TD processing unit 32 determines that the response does not include both the TD and the URL (step S903: none), the process proceeds to step S906.

The TD processing unit 32 shifts from step S903 (none), step S904 or step S905, and stores the TD of the IoT device 2 in the TD storage unit 21 (step S906).

The TD processing unit 32 performs the processing of steps S903 to S906 for all connectable IoT devices 2. As a result, except for the case of step S903 (none), the TDs of all connectable IoT devices 2 are acquired and stored in the TD storage unit 21.

When the IoT device 2 is a non-WoT device, the non-WoT device does not hold the TD and the TDID, so that the non-WoT device can be a connectable IoT device 2, but the TD is not acquired.

The TD processing unit 32 reads out the TDs of all connectable IoT devices 2 from the TD storage unit 21. Then, the TD processing unit 32 generates usable device information based on the TD and outputs it to the device determination unit 26 (step S907). Specifically, the TD processing unit 32 extracts the device ID, the device type, the device function, and the like from the TD, and generates these information as usable device information for each connectable IoT device 2.

In this way, the TD acquisition unit 20 acquires the TD of the IoT device 2 that can be connected to the terminal device 1, and generates usable device information including various TD data. Since the terminal device 1 is a kind of IoT device 2, the TD processing unit 32 also stores the TD of the terminal device 1 in the TD storage unit 21 and generates usable device information including the TD data of the terminal device 1. It shall be.

(CD acquisition unit 24)
Next, the processing of step S405 by the content acquisition unit 22 and the processing of steps S406 and S407 by the CD acquisition unit 24 will be described in detail. FIG. 10 is a block diagram showing a configuration example of the CD acquisition unit 24, and FIG. 11 is a flowchart showing a processing example of the content acquisition unit 22 and the CD acquisition unit 24.

The CD acquisition unit 24 includes a content interpretation unit 40 and a dummy CD generation unit 43. The content interpretation unit 40 includes a CD interpretation unit (CD processing unit) 41 and a Web browser 42. It is assumed that the Web browser 42 is already in operation.

With the activation of the matching application, the URL of the content acquisition destination is notified to the content interpretation unit 40. For example, the TV of the IoT device 2-2 notifies the content interpretation unit 40 of the URL related to the content being played by the TV. Further, when the terminal device 1 operates to acquire the content list information from the content server 4, the URL of the content acquisition destination is notified.

When the content interpretation unit 40 acquires the URL of the content acquisition destination (step S1101), the content interpretation unit 40 outputs the URL to the content acquisition unit 22.

The content acquisition unit 22 inputs the URL of the content acquisition destination from the content interpretation unit 40, receives the content from the content server 4 of the URL (step S1102), and stores the content in the content storage unit 23. Then, the content acquisition unit 22 outputs the received content to the content interpretation unit 40. Alternatively, the content acquisition unit 22 reads the content from the content storage unit 23 and outputs the read content to the content interpretation unit 40.

Here, the content is composed of a CD and the content text, or is composed of a CDID (CD identifier, for example, the URL of the CD server 5 in which the CD is stored) and the content text that uniquely determine the CD. Some content does not have a CD or CDID. That is, the content holds a CD associated with the content, a CDID that uniquely determines the CD, or neither a CD nor a CDID.

If the content is, for example, HTML content that can be executed by the Web browser 42, the CD has a CD, and the CD corresponds to the type of WoT device or non-WoT device that the content targets (operates according to the content). The type of content that has been created is described. The content text in this case is HTML content (so-called Web application).

The CD interpretation unit 41 of the content interpretation unit 40 performs a process of acquiring a CD of the content. Specifically, the CD interpretation unit 41 inputs the content from the content acquisition unit 22, and whether the content includes a CD, the content includes a CD URL (CDID), or the content includes a CD and a CD. It is determined whether both of the URLs are included (step S1103).

When the CD interpreting unit 41 determines that the content includes a CD (step S1103: CD), the CD interpreting unit 41 extracts the CD from the content (step S1104).

On the other hand, when the CD interpreting unit 41 determines that the content does not include the CD and contains the URL of the CD (step S1103: URL), the CD interpreting unit 41 receives the CD from the CD server 5 of the URL (step S1103: URL). S1105).

Further, when the CD interpreting unit 41 determines that the content does not include both the CD and the URL (step S1103: none), the CD interpreting unit 41 acquires a dummy CD from the dummy CD generation unit 43 (step S1106).

The dummy CD generation unit 43 generates a CD containing data indicating that only the terminal device 1 reproduces the content as a dummy CD, and in response to a request from the CD interpretation unit 41, the dummy CD is sent to the CD interpretation unit 41. Output. For example, in the CD shown in FIG. 7, the data of the terminal device 1 is set as a playable device. For the dummy CD in this case, data is set only for the playable device, and null is set for the other items. Then, the CD interpretation unit 41 treats the dummy CD acquired from the dummy CD generation unit 43 as a content CD.

In addition to the terminal device 1, the dummy CD generation unit 43 may generate a CD including data indicating that only the specific IoT device 2 reproduces the content as a dummy CD. In the dummy CD in this case, the data of the terminal device 1 and the specific IoT device 2 are set as playable devices.

As a result, the content that does not include both the CD and the URL is reproduced only by the playable device set in the dummy CD.

The CD interpreting unit 41 shifts from step S1104, step S1105 or step S1106, and stores the CD of the content in the CD storage unit 25 (step S1107).

The CD interpretation unit 41 performs the processes of steps S1103 to S1107 for all the contents acquired from the content server 4. As a result, CDs of all the acquired contents are acquired and stored in the CD storage unit 25.

The CD interpretation unit 41 reads the CDs of all the acquired contents from the CD storage unit 25. Then, the CD interpretation unit 41 extracts the content type, playable device, etc. from the CD, generates usable content information including the content type, etc. of all the acquired contents, and outputs this to the device determination unit 26. (Step S1108).

In this way, the CD acquisition unit 24 acquires CDs of all the acquired contents, and generates and outputs usable content information including the content types of all the acquired contents.

(Device determination unit 26 and content processing unit 27)
Next, the processing of steps S408 and S409 by the device determination unit 26 and the processing of step S410 by the content processing unit 27 will be described in detail. FIG. 12 is a block diagram showing a configuration example of the device determination unit 26 and the content processing unit 27, and FIG. 13 is a flowchart showing a processing example of the device determination unit 26 and the content processing unit 27.

The device determination unit 26 includes a reproducible device determination unit 50, a reproduction device determination unit 51, a user selection unit 52, and a device selection unit 53. The content processing unit 27 includes a content interpretation unit 40 and a Web server unit 60, and the content interpretation unit 40 includes a CD interpretation unit 41 and a Web browser 42. The content interpretation unit 40 is the same as the content interpretation unit 40 provided in the CD acquisition unit 24 shown in FIG. In the CD acquisition unit 24 shown in FIG. 10, only the CD interpretation unit 41 of the content interpretation unit 40 functions and the Web browser 42 does not function, but in the content processing unit 27, the CD interpretation unit 41 does not function and the Web. Only the browser 42 works.

The reproducible device determination unit 50 of the device determination unit 26 inputs the usable device information from the TD acquisition unit 20, and also inputs the usable content information from the CD acquisition unit 24 (step S1301). Then, the reproducible device determination unit 50 determines the reproducible device for each of the acquired contents based on the usable device information and the usable content information (step S1302).

Specifically, the playable device determination unit 50 is included in the device functions (audio playback, video playback, etc.) of the IoT device 2 included in the usable device information and the usable content information of the content for each content. By comparing with the content type (audio, video, etc.), it is determined whether the IoT device 2 can reproduce the content. Then, the reproducible device determination unit 50 determines the reproducible device for the content from the IoT devices 2 of all the device IDs included in the usable device information, and generates the reproducible device information.

For example, the usable device information includes the device function "voice playback" of the IoT device 2-1 and the device function "video playback" of the IoT device 2-2, the device function "text playback" of the IoT device 2-3, and the IoT device 2. It is assumed that the data of the device function "video reproduction" of -4 is included and the usable content information includes the data of the content type "video" of the content A. Then, the reproducible device determination unit 50 compares these device functions included in the usable device information with the content type of the content A included in the usable content information for the content A. The playable device determination unit 50 assumes that the device function "video reproduction" of the IoT device 2-2, 2-4 corresponds to the content type "video" of the content A, and the content of the IoT device 2-2, 2-4. It is determined that A is reproducible, and reproducible device information is generated.

Further, when the reproducible device determination unit 50 includes data indicating that only the terminal device 1 reproduces the content in the usable content information (for example, the terminal device 1 is set as a reproducible device). If), the terminal device 1 determines that the content can be reproduced, and generates reproducible device information.

When the user selects the desired content to be actually reproduced, the reproducible device determination unit 50 outputs the reproducible device information of the content to the reproduction device determination unit 51. The playable device information includes information on all IoT devices 2 and the like that can play the content.

The playback device determination unit 51 inputs playable device information from the playable device determination unit 50, and determines whether the means for determining the playback device to be actually used is by the user's operation selection or by automatic selection based on the algorithm. Determination (step S1303).

When the determination means determines in step S1303 that the determination means is based on the user's operation selection (step S1303: user), the reproduction device determination unit 51 outputs a selection instruction including the reproducible device information to the user selection unit 52. Then, the playback device determination unit 51 inputs the selection result by the user from the user selection unit 52, and determines the device of the selection result as the playback device (step S1304). In this case, when the user selection unit 52 inputs a selection instruction from the playback device determination unit 51, the user selects the user by presenting a device selection screen including the device indicated by the playable device information and making an announcement by voice. The selection result including the selected device is output to the playback device determination unit 51.

On the other hand, when the determination means determines in step S1303 that the determination means is based on automatic selection based on the algorithm (step S1303: algorithm), the reproduction device determination unit 51 outputs a selection instruction including the reproducible device information to the device selection unit 53. .. Then, the playback device determination unit 51 inputs the selection result by the algorithm from the device selection unit 53, and determines the device of the selection result as the playback device (step S1305). In this case, when the device selection unit 53 inputs a selection instruction from the playback device determination unit 51, the device selection unit 53 automatically selects a desired device from the devices indicated by the playable device information by a preset algorithm, and the algorithm is used. The selection result including the selected device is output to the playback device determination unit 51.

The playback device determination unit 51 generates information about the determined device as playback device information, and outputs the playback device information to the content interpretation unit 40 of the content processing unit 27.

The Web browser 42 of the content interpretation unit 40 inputs the reproduction device information from the reproduction device determination unit 51. The Web browser 42 interprets and executes the content text of the content selected by the user (or the content read from the content storage unit 23 and selected by the user) that has already been input from the content acquisition unit 22. Then, when the control for the IoT device 2 is described in the content text as the content text is interpreted and executed, the Web browser 42 transmits a control message related to the control to the playback device included in the playback device information. (Step S1306).

When the playback device included in the playback device information is the terminal device 1, the Web browser 42 outputs a control message to the Web server unit 60, which is a WoT processing unit. The Web server unit 60 inputs a control message from the Web browser 42 and performs processing according to the control message. As a result, the content is reproduced on the terminal device 1.

When the playback device is a WoT device (WoT IoT device 2) other than the terminal device 1, the Web browser 42 transmits a control message to the Web server of the IoT device 2. The IoT device 2 receives a control message from the Web browser 42, operates according to the control message, performs reproduction indicated by the control message, and downloads and reproduces necessary contents from the content server 4. As a result, the IoT device 2 of WoT performs the reproduction processing related to the content.

When the playback device is a non-WoT device (non-WoT IoT device 2), the Web browser 42 transmits the control message to the IoT device 2 according to the information described in the content text such as the IP address. Here, when the playback device is a non-WoT device, the control message is transmitted on the premise that the means for specifying the destination is described in the content text. The IoT device 2 receives a control message from the Web browser 42, operates according to the control message, performs reproduction indicated by the control message, and downloads and reproduces necessary contents from the content server 4. As a result, the non-WoT IoT device 2 performs the reproduction processing related to the content.

If the Web browser 42 determines that the control message cannot be transmitted to the IoT device 2 due to a failure in communication with the IoT device 2, the content interpretation unit 40 may use the reproducible device of the device determination unit 26. The determination unit 50 is requested to determine the reproducible device again, or ignores this and continues the execution of the content main body.

In this way, the device determination unit 26 determines the playback device that is actually reproduced for each of the acquired contents, and the content processing unit 27 reproduces the content on the reproduction device.

For example, when the television of the IoT device 2-2 is playing a predetermined video content, the device determination unit 26 actually plays back the content (related content) related to the video content of the IoT device 2-. Playback devices for smart speakers and smart mirrors, which are 1, 2 and 3, are determined. Then, the content processing unit 27 transmits a control message to the IoT devices 2-1 and 2-3 according to the description of the content body.

That is, the content processing unit 27 uses the smart speaker of the IoT device 2-1 to play the audio content as the playback process according to the related content while the television of the IoT device 2-2 is playing the predetermined video content. Play it back and play the text content on the smart mirror of the IoT device 2-3. As a result, the voice corresponding to the image is output by the smart speaker of the IoT device 2-1 and the text corresponding to the image is displayed by the smart mirror of the IoT device 2-3.

As described above, according to the terminal device 1 of the embodiment of the present invention, the TD acquisition unit 20 searches for the connectable IoT device 2 and acquires the TDs of all the connectable IoT devices 2. Then, the TD acquisition unit 20 extracts the device functions and the like from the TD, and generates usable device information including the device functions and the like of all connectable IoT devices 2.

The CD acquisition unit 24 acquires each CD for all the contents acquired by the content acquisition unit 22, extracts the content types and the like from the CDs, and obtains usable content information including the content types and the like of all the acquired contents. Generate.

The device determination unit 26 compares the usable device information and the usable content information. Then, the device determination unit 26 determines the IoT device 2 that can reproduce the content for each of the acquired contents.

When the user selects the desired content to be actually played back, the device determination unit 26 selects or determines the desired IoT device 2 to be actually played back among the IoT devices 2 capable of playing the content. Determined according to the algorithm of.

The content processing unit 27 executes the content selected by the user, and sends a control message to the IoT device 2 determined by the device determination unit 26 according to the description of the content body.

As a result, the IoT device 2 operates according to the control message, downloads necessary contents from the content server 4, and performs playback. That is, the content processing unit 27 reproduces the content selected by the user on the terminal device 1, and also performs reproduction processing on the content selected by the user for the IoT device 2 determined by the device determination unit 26. Can be made.

Therefore, the terminal device 1 can associate one or more IoT devices 2 with one content by a simple mechanism. As a result, various contents desired by the user can be reproduced by various IoT devices 2 desired by the user by a simple mechanism.

In addition, by providing software that associates the IoT device 2 with the content as an application using a CD and TD, it is possible to handle various contents with a unified description of the application without depending on the type. Will be.

[Description example of matching application]
Next, a description example of a matching application that realizes the processing example shown in FIG. 4 under the configuration of the control unit 10 shown in FIG. 3 will be described. FIG. 14 is a diagram for explaining a description example of the matching application, and shows only the main processing among the processing examples shown in FIG.

The CD of the content A includes the data of the content type "audio", the CD of the content B includes the data of the content type "video", and the TD of the IoT device 2-1 includes the device type "audio reproduction". It is assumed that there is. That is, it is assumed that the content A is audio content, the content B is video content, and the IoT device 2-1 is a smart speaker.

According to the description of (a) of the matching application, the TD acquisition unit 20 searches for the connectable IoT device 2, determines that the connectable device is only the smart speaker of the IoT device 2-1 and determines that the IoT device is the only smart speaker. Get 2-1 TD.

According to the description of (b) of the matching application, the CD acquisition unit 24 acquires the CDs of the contents A and B. Further, the device determination unit 26 has a function of reproducing the content of the "voice" type of the IoT device 2-1 based on the device type "voice reproduction" included in the TD of the IoT device 2-1. to decide. Further, the device determination unit 26 determines that the contents A and B are audio contents and video contents, respectively, based on the content types "audio" and "video" included in the CDs of the contents A and B.

Then, the device determination unit 26 determines that the device capable of reproducing the content A is the IoT device 2-1 and determines that there is no device capable of reproducing the content B.

According to the description of (c) of the matching application, the content processing unit 27 causes the IoT device 2-1 to perform the reproduction processing for the content A selected by the user, for example.

In this way, by using the matching application, the desired content can be played back on the connectable IoT device 2. That is, according to the present invention, various contents such as video, audio, text, control program, coupon, etc. can be handled by a unified matching application description without depending on the type, and various IoT devices can be used. It is possible to play with a simple mechanism.

As the hardware configuration of the terminal device 1 according to the embodiment of the present invention, a normal computer can be used. The terminal device 1 is composed of a computer including a volatile storage medium such as a CPU and RAM, a non-volatile storage medium such as ROM, and an interface.

The functions of the TD acquisition unit 20, the TD storage unit 21, the content acquisition unit 22, the content storage unit 23, the CD acquisition unit 24, the CD storage unit 25, the device determination unit 26, and the content processing unit 27 provided in the terminal device 1 are Each of these is realized by having the CPU execute a program that describes these functions.

These programs are stored in the storage medium, read by the CPU, and executed. In addition, these programs can be stored and distributed in storage media such as magnetic disks (floppy (registered trademark) disks, hard disks, etc.), optical disks (CD-ROM, DVD, etc.), semiconductor memories, etc., and can be distributed via a network. You can also send and receive.

1 Terminal device 2 IoT device 3 TD server 4 Content server 5 CD server 6 Broadcasting / communication network 10 Control unit 11 CPU
12 Storage unit 13 Storage device 14 Operation / input unit 15 Display / output interface unit 16 Communication unit 17 System bus 20 TD acquisition unit 21 TD storage unit 22 Content acquisition unit 23 Content storage unit 24 CD acquisition unit 25 CD storage unit 26 Device determination unit 27 Content processing unit 30 Equipment management unit 31 Equipment communication unit 32 TD processing unit 40 Content interpretation unit 41 CD interpretation unit 42 Web browser 43 Dummy CD generation unit 50 Reproducible device judgment unit 51 Playback device determination unit 52 User selection unit 53 Device selection Part 60 Web server part

Claims (5)

In a terminal device that acquires content from a content server and causes an IoT (Internet of Things) device to perform playback processing related to the content.
The IoT device connected by communication with the terminal device is searched, and for each of all the connectable IoT devices, the type of the IoT device is shown from the predetermined server or the response accompanying the search. Acquires a TD (Thing Description) including device type, device function indicating what kind of content the IoT device can play, and attribute data indicating the file type of the content played by the IoT device. TD acquisition department and
From a predetermined server or the content, the content type indicating the type of the content, the genre indicating the field to which the content belongs, and the data of the reproducible device indicating the type of the IoT device capable of reproducing the content are included. A CD acquisition unit that acquires a CD (Content Description),
The device type acquired by the TD acquisition unit, the TD including the device function and the attribute data , and the content type , the genre, and the reproducible device data acquired by the CD acquisition unit. A device determination unit that determines the IoT device that actually reproduces the content among all the connectable IoT devices based on the CD including the CD.
A content processing unit that causes the IoT device determined by the device determination unit to perform playback processing related to the content.
A terminal device characterized by being equipped with. In the terminal device according to claim 1,
It is equipped with a content acquisition unit that acquires a plurality of contents from the content server.
The TD acquisition unit
For all the connectable IoT devices, the usable device information including the device type, the device function, and the attribute of the IoT device is generated.
The CD acquisition unit
For all the contents acquired by the content acquisition unit, usable content information including the content type , the genre, and the playable device of the content is generated.
The device determination unit
The device type, the device function and the attribute included in the usable device information generated by the TD acquisition unit, and the content type and the genre included in the usable content information acquired by the CD acquisition unit. And, for each of all the contents acquired by the content acquisition unit based on the playable device, the IoT device capable of reproducing the contents can be reproduced among all the connectable IoT devices. Judged as an IoT device,
A terminal device for determining an IoT device that is actually reproduced among the playable IoT devices for the content selected by the user among all the contents acquired by the content acquisition unit. In the terminal device according to claim 2,
It is provided with a dummy CD generation unit that generates a CD containing specific device data indicating that only the terminal device reproduces the content as a dummy CD.
The CD acquisition unit
When the CD cannot be acquired from both the predetermined server and the content for the content acquired by the content acquisition unit, the dummy CD is input from the dummy CD generation unit and includes the specific device data of the content. Generate the available content information and
The device determination unit
When the specific device data is included in the usable content information for the content selected by the user among all the contents acquired by the content acquisition unit, the terminal device actually reproduces the content. A terminal device characterized in that it is determined as a device to be used. In the terminal device according to claim 2,
All connectable IoT devices include a television playing a given video content.
The content acquisition unit
The related content related to the predetermined video content being played on the television is acquired from the content server, and the content is obtained from the content server.
The CD acquisition unit
For the related content acquired by the content acquisition unit, the usable content information including the content type , the genre, and the playable device of the related content is generated.
The device determination unit
When the related content is selected by the user, the IoT device that actually plays the related content is determined among the playable IoT devices for the related content.
The content processing unit
A terminal device characterized in that, in a state where the television is playing the predetermined video content, the IoT device determined by the device determination unit performs a reproduction process related to the related content. A program for operating a computer as the terminal device according to any one of claims 1 to 4.

Images