JP6178764B2 - Information processing method, communication terminal, and computer program - Google Patents

Description

  The present invention relates to an information processing method, a communication terminal, and a computer program.

  In the format of providing information by web technology, a markup language is used as a means for describing the structure of document information and providing video and other digital contents. Hypertext markup language (HTML) is the most popular web markup language and describes elements in the form of tags enclosed in angle brackets. HTML can describe the structure of web content and can include embedded script language code JavaScript that affects the behavior of web browsers and other HTML processors. In order to access each element of HTML from a code written in a script language, a web browser is provided with an API that converts and controls tree structure data called DOM.

Web technologies such as webRTC and WebSocket are used as a method for bidirectional communication between different web contents using a script language. webRTC is a mechanism that can send and receive video, audio, and data between browsers peer-to-peer, and WebSocket is a mechanism that can send and receive messages asynchronously via a server.
A technique for manipulating a DOM object of a remote communication terminal from a local communication terminal by utilizing such a technique in which communication terminals communicate bidirectionally has been proposed. For example, in Non-Patent Document 1, in order to control a DOM object of a remote communication terminal, a JavaScript code is transferred as a character string and is executed using an evaluation function (eval function) in the remote communication terminal. API specifications are open to the public.

  These technologies are framed so that creators can use them without being aware of communication methods and server-side programs.

"Remote UI API", [online], [Search July 22, 2014], Internet <URL: http://dev.webinos.org/specifications/api/remoteui.html>

  In Non-Patent Document 1, in order to operate a DOM object of a remote communication terminal, an API specification is defined independently instead of the W3C standard, and a creator can describe web content according to a procedure defined in the API. Desired. For web content creators, if the DOM object of the remote communication terminal can be operated with the same operability when the DOM object of the local communication terminal is operated, the convenience is high. However, the method disclosed in Non-Patent Document 1 merely transfers a JavaScript code directly to a remote communication terminal, and it cannot be said that an API is designed in consideration of developer convenience.

  Further, when an incorrect code is included in the transferred JavaScript program, there is a risk that an operation unintended by the user in the remote communication terminal may occur or a security risk that includes an illegal code.

  As described above, the conventional methods have problems in terms of convenience as API and security.

  The present invention has been made in view of the above, and an object of the present invention is to provide a technique for executing a method of a DOM object while ensuring security in a remote communication terminal.

  In order to solve the above problems, according to a first aspect of the present invention, a first communication terminal reads a file describing the contents of digital content, stores the file as a DOM tree, and the first communication terminal , A selector indicating an element which is a part of a DOM tree corresponding to a DOM object generated by a second communication terminal different from the first communication terminal by the DOM tree, a method name of a method to be executed by the DOM object, and Obtaining an argument to be passed to the method and transmitting it to the second communication terminal, and the second communication terminal searches for an element corresponding to the selector from a DOM tree held in the second communication terminal, A DOM object is generated from the element, stored in the second communication terminal, and the second communication terminal When the method corresponding to the method name is executed by the DOM object, the argument is passed to the method, and when a return value is obtained by processing by the method, the return value is transmitted to the first communication terminal. Features.

  According to a second aspect of the present invention, a first communication terminal reads a file describing the contents of digital content, stores the file as a DOM tree, and the first communication terminal uses the DOM tree to store the first communication terminal. Obtaining a selector indicating an element which is a part of a DOM tree corresponding to a DOM object generated by a second communication terminal different from the communication terminal of the method, a method name of a method to be executed by the DOM object, and an argument to be passed to the method, Transmitting to a second communication terminal different from the first communication terminal.

  According to a third aspect of the present invention, a second communication terminal different from the first communication terminal has a selector indicating an element that is a part of a DOM tree held in the second communication terminal, and a DOM object generated from the element The method name of the method to be executed and the argument passed to the method are received from the first communication terminal, and the second communication terminal corresponds to the selector from the DOM tree held in the second communication terminal. Search the element, generate a DOM object from the element, store the DOM object in the second communication terminal, the second communication terminal causes the DOM object to execute the method corresponding to the method name, and the argument When the method is passed to the method and a return value is obtained by processing by the method, the return value is sent to the first pass. And transmits to the terminal.

  According to a fourth aspect of the present invention, there is provided a communication terminal for storing a file describing the contents of digital content as a DOM tree, and a DOM generated by the second communication terminal different from the communication terminal by the DOM tree. A selector that indicates an element that is a part of the DOM tree corresponding to the object, a method name of a method to be executed by the DOM object, and an argument to be passed to the method, and a means for transmitting to the second communication terminal. Features.

  The fifth aspect of the present invention is a communication terminal as a second communication terminal different from the first communication terminal, and a selector indicating an element that is a part of a DOM tree held in the second communication terminal, the element Corresponding to the selector from the DOM tree held in the second communication terminal and the means for receiving from the first communication terminal the method name of the method to be executed by the DOM object generated from the method and the argument passed to the method Means for searching for an element and generating a DOM object from the element; means for storing the DOM object; means for causing the DOM object to execute a method corresponding to the method name and passing the argument to the method; If a return value is obtained by processing by the method, the return value is Characterized in that it comprises a means for transmitting to the serial first communication terminal.

  In a sixth aspect of the present invention, a first communication terminal reads a file describing the contents of digital content, stores the file as a DOM tree, and the first communication terminal uses the DOM tree to store the first communication terminal. Obtaining a selector indicating an element which is a part of a DOM tree corresponding to a DOM object generated by a second communication terminal different from the communication terminal of the method, a method name of a method to be executed by the DOM object, and an argument to be passed to the method, Transmitted to the second communication terminal, the second communication terminal searches for an element corresponding to the selector from the DOM tree held in the second communication terminal, generates a DOM object from the element, Stored in the second communication terminal, and the second communication terminal corresponds to the method corresponding to the method name. When the DOM object is executed, the argument is passed to the method, and when a return value is obtained by processing by the method, the return value is transmitted to the first communication terminal. And an information processing program for operating the second communication terminal.

  According to the present invention, when the second communication terminal is operated from the first communication terminal, the second communication terminal, that is, the remote communication terminal, is generated from the elements included in the DOM tree held in the communication terminal. Since the method of the DOM object is executed, there is no possibility that an illegal code or an incorrect code is transmitted to the communication terminal, so that security is ensured and the method of the DOM object can be executed.

It is a figure which shows the structure of the information processing system which concerns on this Embodiment. It is a figure which shows the example of the communication system of the communication which the communication terminals 1 and 2 perform. It is a figure which shows an example of the designation | designated method of a selector in case a selector is 'body'. 6 is a diagram illustrating an example of an HTML file and a script (script 15) read by the communication terminal 1 in Embodiment 1. FIG. 6 is a diagram illustrating an example of an HTML file and a script read by the communication terminal 2 in Embodiment 1. FIG. 6 is a diagram illustrating a screen of the communication terminals 1 and 2 before operating the DOM object 23 in Embodiment 1. FIG. 6 is a diagram illustrating a screen of communication terminals 1 and 2 after operating a DOM object 23 in Embodiment 1. FIG. It is a figure which shows the screen of PC and television used in Example 2, and the screen of a smart phone. It is a figure which shows the example of the HTML file and script (script 15) which are read with the smart phone (communication terminal 1) in Example 2. FIG. It is a figure which shows the example of the HTML file and script which are read by PC etc. (communication terminal 2) in Example 2. FIG.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram showing a configuration of an information processing system according to the present embodiment.

  The information processing system includes a communication terminal 1 (first communication terminal) and a communication terminal 2 (second communication terminal). The communication terminal 2 is used at a place away from the communication terminal 1, that is, a remote communication terminal.

  The communication terminal 1 reads, for example, an HTML file describing the contents of digital content, and stores the file as a DOM tree 10.

  The DOM tree 10 includes a script 15, and a processing unit 16 including the selector reception unit 11 and the communication unit 14 is formed in the communication terminal 1 when a browser (not shown) of the communication terminal 1 executes the script 15. .

  The selector reception unit 11 includes a dummy DOM object generation unit 12, and the communication unit 14 includes a selector transmission unit 141, a method name transmission unit 142, and a return value reception unit 143.

  The dummy DOM object 13 is generated by the dummy DOM object generation unit 12 and stored in the memory.

  The communication terminal 2 stores, for example, an HTML file as digital content as a DOM tree 24. When a browser (not shown) of the communication terminal 2 executes a predetermined script, a processing unit 26 including the communication unit 21 and the DOM search unit 22 is formed in the communication terminal 2. The communication unit 21 includes a selector reception unit 211, a method name reception unit 212, and a return value transmission unit 213.

  A DOM object 23 is generated from an element that is a part of the DOM tree 24 and stored in a memory.

  The script 15 includes a selector that is information indicating this element.

  The configuration of FIG. 1 is for operating the DOM object 23 from the communication terminal 1, but the communication terminals 1 and 2 are connected to the processing unit 16, so that the communication terminal 2 can also operate the DOM object of the communication terminal 1. 26 may be provided.

  FIG. 2 is a diagram illustrating an example of a communication method of communication performed by the communication terminals 1 and 2.

  The communication terminals 1 and 2 communicate with each other by P2P bidirectional communication using, for example, webRTC. Alternatively, the communication terminals 1 and 2 communicate by asynchronous communication using, for example, WebSocket. Alternatively, the communication terminals 1 and 2 communicate in asynchronous communication using, for example, WebSocket in a 1: N communication form instead of such a 1: 1 communication form.

  Hereinafter, the operation of the information processing system will be described.

  The dummy DOM object generation unit 12 of the selector reception unit 11 receives the selector from the script 15.

  When the script 15 transmits a selector to the selector receiver 11, it must be identified that the selector is to be transmitted to the remote communication terminal 2. For that purpose, various notations are conceivable, such as identifying a selector by a condition (argument), or preparing a method that explicitly indicates that the selector is to be transmitted to the remote communication terminal 2.

  FIG. 3 is a diagram illustrating an example of a method for designating a selector when the selector is “body”.

  In the examples of reference numerals 31 and 32, character strings such as rmt and remote are designated as identification symbols in order to explicitly indicate that the selector is to be transmitted to the remote communication terminal 2. As in the example of reference numeral 31, “rmt” is given to the prefix of the selector, and as in the example of reference numeral 32, the value “remote” is given as the second argument.

  In the examples of reference numerals 33 and 34, methods remote () and rmt () that explicitly indicate that the selector is to be transmitted to the remote communication terminal 2 are prepared, and the method is called by each designation method to perform remote communication. It is clearly indicated that the selector is to be transmitted to the terminal 2.

  The dummy DOM object generation unit 12 inherits the interface and method of the DOM object 23 when the selector reception unit 11 receives the selector, and when the method is called, the method is called the method name and the method name. A dummy DOM object 13 defined to return an argument delivered to the method is generated and stored in the memory of the communication terminal 1. In addition, the dummy DOM object generation unit 12 transmits the selector received by the selector reception unit 11 to the selector transmission unit 141.

  The DOM object 23 is, for example, an HTML element object having a basic element interface, and the dummy DOM object 13 inherits this element interface.

  Alternatively, the DOM object 23 has an element interface having a unique function according to the type and method of a tag that is desired to be operated from the communication terminal 1 to the remote communication terminal 2, and the dummy DOM object 13 inherits this element interface. Have.

  For example, the “DIV” tag is an HTMLDiv element object, the “VIDEO” tag is an HTMLMedia element object, and a dedicated function, method, and property are added to the basic element interface.

  Which tag object the dummy DOM object 13 inherits may be set manually with reference to the DOM tree 24, or may be automatically specified by analyzing the DOM tree 24 in advance.

  The DOM object 23 is compliant with, for example, the W3C specification. However, the DOM object 23 may include the DOM object and have an extended function (for example, a jQuery object). The DOM object 23 may be inherited.

  The inside of each method of the dummy DOM object 13 has been rewritten so that the dummy DOM object 13 cannot be operated, and is defined so that the method name and its argument are passed to the communication unit 14 when the method is called. The With this dummy DOM object 13, the creator of the digital content can call the method of the DOM object 23 in the same manner as when calling the method of the DOM object of the communication terminal 1.

  The DOM object 23 may be a wrapper object that contains the DOM object, and the dummy DOM object 13 may be an inheritance of this wrapper object.

  The selector transmission unit 141 transmits the selector received from the dummy DOM object generation unit 12 to the communication terminal 2.

  The method name transmission unit 142 receives the method name and the argument from the dummy DOM object 13 and transmits them to the communication terminal 2.

  Note that an identifier for each terminal and information on the transmission source may be transmitted.

  The selector reception unit 211 receives the selector from the selector transmission unit 141 and transmits the selector to the DOM search unit 22.

  The method name reception unit 212 receives a method name and an argument from the method name transmission unit 142.

  The DOM search unit 22 searches the DOM tree 24 for an element corresponding to the selector, generates a DOM object 23 from the element, and stores it in the memory of the communication terminal 2.

  The method name receiving unit 212 causes the DOM object 23 to execute a method corresponding to the received method name, and delivers the received argument to the method. Thus, the method performs processing based on the argument, and in some cases, the return value obtained by the processing is transmitted to the return value transmission unit 213.

  The return value transmission unit 213 transmits the return value to the communication terminal 1.

  The return value receiving unit 143 receives the return value from the return value transmitting unit 213.

  For example, when there is no element corresponding to the selector in the DOM tree 24, if the DOM object 23 cannot execute the method corresponding to the method name, an error is transmitted to the communication terminal 1.

Example 1
Next, a specific example will be described.

  FIG. 4 is a diagram showing an example of an HTML file and a script (script 15) read by the communication terminal 1, and FIG. 5 is a diagram showing an example of an HTML file and a script read by the communication terminal 2.

  In FIG. 4, a script 15 is a code described in a range surrounded by <script> </ script> tags executed as JavaScript (registered trademark).

  In FIG. js provides the functions of the processing units 16 and 26 to the communication terminal 2 from the outside. That is, the communication terminal 2 includes the processing units 16 and 26 unlike FIG.

  In this example, a selector API provided by the jQuery library is used as a method of operating the DOM object 23 using the selector. Therefore, the DOM object 23 is a wrapper object (jQuery object) that contains the DOM object. Therefore, the dummy DOM object 13 inherits this jQuery object. The jQuery library is also read as an external script.

  FIG. 6 is a diagram illustrating a screen of the communication terminals 1 and 2 before the DOM object 23 is operated.

  As shown in FIG. 6A, when the browser of the communication terminal 1 reads the HTML file of FIG. 4, the browser displays buttons labeled “String Device” and “Show Message”.

  As shown in FIG. 6B, the browser of the communication terminal 2 displays the character string “Local Device” when the HTML file of FIG. 5 is read.

  When the button in FIG. 6A is pressed, $ (“# remote”). rmt () is executed, and the dummy DOM object generation unit 12 generates a dummy DOM object 13 and stores it in the variable dummy. In this case, the selector is “#remote”, and the selector is transmitted to the communication terminal 2.

  By this selector, an element represented as <div id = “remote”> in FIG. 5 is searched, and a DOM object 23 is generated from this element.

  Since the dummy DOM object 13 is an object that inherits the method name of the jQuery object, dummy can call append () that is a method of the jQuery object. append () has been rewritten so as not to execute the original operation at the time of inheritance, and returns the method name “append” and its argument “<h2> This is displayed on the remote device </ h2>”. The method name and argument are transmitted to the communication terminal 2.

  In the communication terminal 2, the DOM object 23 executes a method corresponding to this method name, and the method performs processing according to the argument.

  FIG. 7 is a diagram showing a screen of the communication terminals 1 and 2 after operating the DOM object 23 in this way.

  As shown in FIG. 7A, the screen of the communication terminal 1 is the same as before the operation of the DOM object 23.

  As illustrated in FIG. 7B, a character string “This is displayed on the remote device” is added to the screen of the communication terminal 1 as a result of the operation of the DOM object 23.

(Example 2)
Next, another specific example will be described.

  If the technology of the present embodiment is utilized, a smartphone can be used as the communication terminal 1, a PC (personal computer) or television can be used as the communication terminal 2, and a PC (personal computer) or television screen can be operated by a tap operation on the smartphone. .

  FIG. 8 is a diagram showing a screen of a PC or television and a screen of a smartphone.

  A PC etc. is accessing a video sharing site, and the screen displays an area for displaying the video, a controller for controlling playback and stop of the video, a button for displaying the video in full screen, and a button for displaying related videos. Is done.

  On the screen of the smartphone, a controller for controlling playback and stop of the moving image, a button for displaying the moving image in full screen, and a button for displaying the related moving image are displayed.

  Controllers, buttons for full screen display, PCs for displaying related videos, and smartphone buttons are associated with each other.

  For example, when a button for full screen display is pressed on a smartphone, a button related thereto, that is, a button for full screen display on a PC or the like is pressed, and a moving image is displayed on the full screen.

  9 is a diagram showing an example of an HTML file and a script (script 15) read by a smartphone (communication terminal 1), and FIG. 10 is an example of an HTML file and a script read by a PC or the like (communication terminal 2). FIG.

  In FIG. 10, an element described as <button type = button id = “full” on click = “fullScreen ()”> indicates a button for full-screen display on a screen such as a PC, and is an identifying id As “full”. When a button for full screen display is pressed, a method called fullscreen is called. This method has the ability to display a moving image in full screen.

  In FIG. 9, an element represented as <button type = button on click = “btnClicked ()”> indicates a button for full screen display on a smartphone screen. When the button for full screen display is pressed, A method called btnClicked is called. This method operates a DOM object 23 such as a PC, calls an event for pressing a button for full screen display on a screen such as a PC, and executes full screen display.

  Specifically, when the button for full screen display on the screen of the smartphone is pressed, $ (“# full”). rmt () is executed, and the dummy DOM object generation unit 12 generates a dummy DOM object 13 and stores it in the variable dummy. In this case, the selector is “#full”, and the selector is transmitted to the communication terminal 2.

  With this selector, an element represented as <button type = button id = “full” on click = “fullScreen ()”> in FIG. 10 is searched, and a DOM object 23 is generated from this element.

  Since the dummy DOM object 13 is an object inheriting the method name of the jQuery object, dummy can call click () that is a method of the jQuery object. The method name of click () is transmitted to the PC or the like.

  In a PC or the like, the DOM object 23 executes click () that is a method corresponding to this method name, and the method performs processing. Thereby, the moving image is displayed on the full screen on a PC or the like.

  As described above, according to the information processing system according to the present embodiment, the communication terminal 2, that is, the remote communication terminal, executes the method of the DOM object generated from the element included in the DOM tree held in the communication terminal. Therefore, there is no possibility that an illegal code or an incorrect code is transmitted to the communication terminal, so that security is ensured and the method of the DOM object can be executed.

  An information processing program (computer program) for operating the communication terminals 1 and 2 can be recorded on a computer-readable recording medium such as a semiconductor memory, a magnetic disk, an optical disk, a magneto-optical disk, a magnetic tape, and the Internet. It can be widely distributed by being transmitted through a communication network such as

DESCRIPTION OF SYMBOLS 1, 2 ... Communication terminal 10, 24 ... DOM tree 11 ... Selector receiving part 12 ... Object generation part 13 ... Dummy DOM object 14, 21 ... Communication part 15 ... Script 16, 26 ... Processing part 22 ... DOM search part 23 ... DOM Object 141 ... selector transmission unit 142 ... method name transmission unit 143 ... return value reception unit 211 ... selector reception unit 212 ... method name reception unit 213 ... return value transmission unit

Claims (7)

The first communication terminal reads a file describing the contents of the digital content, stores the file as a DOM tree,
The first communication terminal is a selector indicating an element that is a part of a DOM tree corresponding to a DOM object generated by a second communication terminal different from the first communication terminal by the DOM tree. Obtain the method name of the method to be executed and the argument to be passed to the method, and send it to the second communication terminal.
The second communication terminal searches for an element corresponding to the selector from the DOM tree held in the second communication terminal, generates a DOM object from the element, stores it in the second communication terminal,
When the second communication terminal causes the DOM object to execute the method corresponding to the method name and passes the argument to the method, and if a return value is obtained by processing by the method, the return value is An information processing method characterized by transmitting to one communication terminal. The first communication terminal reads a file describing the contents of the digital content, stores the file as a DOM tree,
The first communication terminal is a selector indicating an element that is a part of a DOM tree corresponding to a DOM object generated by a second communication terminal different from the first communication terminal by the DOM tree. An information processing method comprising: obtaining a method name of a method to be executed and an argument to be transferred to the method, and transmitting the method name to a second communication terminal different from the first communication terminal. Defined so that the first communication terminal inherits the interface and method of the DOM object and returns the method name of the method and an argument passed to the method when the method is called The information processing method according to claim 1 or 2, wherein a dummy DOM object is generated, the method is called, and a method name and an argument to be transmitted to the second communication terminal are obtained from the method. The second communication terminal that is different from the first communication terminal has a selector indicating an element that is a part of the DOM tree held in the second communication terminal, and a method name of a method that is executed by the DOM object generated from the element. And an argument delivered to the method from the first communication terminal,
The second communication terminal searches for an element corresponding to the selector from the DOM tree held in the second communication terminal, generates a DOM object from the element, stores it in the second communication terminal,
When the second communication terminal causes the DOM object to execute the method corresponding to the method name and passes the argument to the method, and if a return value is obtained by processing by the method, the return value is An information processing method characterized by transmitting to one communication terminal. A communication terminal,
Means for storing a file describing the contents of digital content as a DOM tree;
By the DOM tree, a selector indicating an element that is a part of a DOM tree corresponding to a DOM object generated by a second communication terminal different from the communication terminal, a method name of a method to be executed by the DOM object, and the method are delivered to the method. Means for obtaining an argument and transmitting it to the second communication terminal. A communication terminal as a second communication terminal different from the first communication terminal,
From the first communication terminal, a selector indicating an element that is a part of the DOM tree held in the second communication terminal, a method name of a method to be executed by the DOM object generated from the element, and an argument to be passed to the method are sent from the first communication terminal. Means for receiving;
Means for searching an element corresponding to the selector from the DOM tree held in the second communication terminal, and generating a DOM object from the element;
Means for storing the DOM object;
Means for causing the DOM object to execute a method corresponding to the method name and passing the argument to the method;
A communication terminal comprising: means for transmitting the return value to the first communication terminal when a return value is obtained by processing by the method. The first communication terminal reads a file describing the contents of the digital content, stores the file as a DOM tree,
The first communication terminal is a selector indicating an element that is a part of a DOM tree corresponding to a DOM object generated by a second communication terminal different from the first communication terminal by the DOM tree. Obtain the method name of the method to be executed and the argument to be passed to the method, and send it to the second communication terminal.
The second communication terminal searches for an element corresponding to the selector from the DOM tree held in the second communication terminal, generates a DOM object from the element, stores it in the second communication terminal,
When the second communication terminal causes the DOM object to execute the method corresponding to the method name and passes the argument to the method, and if a return value is obtained by processing by the method, the return value is An information processing program for operating the first communication terminal and the second communication terminal to transmit to one communication terminal.

Images