JP2018205897A - Apparatus, server, program and method for automating device management - Google Patents

Abstract

To mitigate a burden of a user in creation of an automation program to perform processing related to device management automatically.SOLUTION: An apparatus 100 arranged in a position where communication between a device 30 and a terminal 40 that displays a screen to manage the device and receives an operation from a user can be mediated, embeds a user interface for giving a meaning to the operation in communication for management screen display from the device to the terminal (106), collects information of the operation and information for giving the meaning to the operation performed through the user interface along with communication for informing the device from the terminal of the operation from the user received through the management screen (107, 108). Then, a server 200 collects the information from the multiple apparatuses 100 (202) and creates an automation program by combining information of multiple operations according to the information for giving the meaning collected together with information of the operation (203, 204).SELECTED DRAWING: Figure 5

Description

  The present invention relates to a technique for generating an automation program for automatically performing processing related to device management.

  Currently, attacks that illegally operate IoT (Internet of Things) devices are increasing, which is a problem. For example, as of October 2016, there is a research report that there are 1.3 million IoT devices infected with viruses (http://www.tokyo-np.co.jp/article/national/ list / 201612 / CK2016120102000255.html). When such an IoT device infected with a virus is used as a stepping stone for an attack, for example, in the United States, a service such as Twitter (registered trademark) or Spotify (registered trademark) has failed for a long time. (Http://www.itmedia.co.jp/news/articles/1610/22/news024.html), a situation that affects a wide range is caused.

  The cause of the IoT device becoming a virus attack target is considered to be inappropriate management of the IoT device. Inappropriate management means that the initial password has not been changed, the firmware has not been updated, or the like. Such setting changes can be performed by the device administrator (user) according to the manual attached to the device or the official website, but there are many devices that are not actually operated, It is the attack target of the attacker.

  In many cases, the management of the IoT device displays a web screen on a terminal used by the user, and causes the user to check a check box, press a button, or input a value in an input field (an IoT device). Has a function of a web server).

  On the other hand, as a technique for substituting an operation performed by a user on a web screen with an automation program, a tool called iMacros (http://www.koikikukan.com/archives/2014/05) for the user himself to create an automation program. /15-023333.php) is known. A technique for automatically generating a program that automatically executes a communication sequence that causes the same screen transition as a communication sequence performed with a web server in response to a user operation (Patent Document 1). Has been proposed.

JP 2009-87032 A

  The reason for preventing the proper management of IoT devices is that the number of IoT devices that one user must manage is much larger than before, and people who are unfamiliar with the operation of information equipment It is considered that the IoT device is owned and management is required.

  Therefore, in order to properly manage the IoT device and reduce the risk of being attacked, the user can manage the device by executing an automation program instead of performing an operation on the web screen. At the same time, it is desirable that the automated program can be automatically generated with as little burden on the user as possible. Even if there is a tool such as iMacros, it is difficult for a person unfamiliar with the operation of information equipment to create an automation program by himself.

  An issue in automatically generating the operation automation program on the management screen of the IoT device is how to learn the operation method. The contents to be learned include the operation procedure of the management screen and the meaning of the input field. Examples of input fields include INPUT tags, SELECT tags, TEXTAREA tags, and the like. LINK tags etc. can also be included in the input field.

  Since the operation procedure of the management screen and the display contents of the input field and the like are different for each product, the IoT device has to perform the work of learning the operation method and generating the automation program for the number of products. Moreover, IoT devices are expected to increase in the future as a wide variety of products have been distributed so far.

  On the other hand, in the technique of Patent Document 1, the operation of automatically detecting a communication sequence candidate to be automatically executed is only automated, and various communication sequences are actually created as an automation program. Since the automation program is not generated unless the user himself / herself determines and specifies what the parameter means and what value should be set, the burden on the user who manages the IoT device is sufficient. It cannot be reduced.

  The present invention has been made in view of the above circumstances, and an object thereof is to make it possible to generate an automation program for automatically performing processing related to device management while reducing the burden on the user. .

  Note that the present invention can be applied not only to an IoT device but also to any device having a web server function (to the web server itself). Therefore, not only the application to the management of the IoT device but also the application to the RPA (Robotic Process Automation) field in which the robot automatically performs the web screen operation is possible.

  An apparatus according to one aspect according to the principle of the present invention is provided at a position capable of mediating communication between a device and a terminal that displays a screen for managing the device and receives an operation from a user. This contributes to the generation of an automation program for automatically performing the processing related to. The apparatus includes means for embedding a user interface for giving meaning to an operation in communication for displaying a management screen from the device to the terminal, and an operation from a user received via the management screen. And means for collecting and recording information on the operation and information on the meaning of the operation performed via the user interface in accordance with the communication for transmitting the information to the device. The automation program is generated by combining a plurality of recorded operation information according to the meaning information recorded together with the operation information.

  The server according to one aspect according to the principle of the present invention is provided at a position where each can display a screen for managing the device and a terminal that receives an operation from the user to mediate communication. Connected to multiple devices. This apparatus embeds a user interface for giving meaning to an operation in communication for displaying a management screen from the device to the terminal, and receives an operation from the user received via the management screen from the terminal. Accompanying the communication for transmitting to the device, the operation information and the meaning information of the operation performed via the user interface are collected, and the server receives the operation information from the apparatus. And means for receiving the meaning information and a plurality of the received operation information according to the meaning information received together with the operation information, thereby automatically performing processing related to device management. Means for generating an automated program.

  A program according to an aspect according to the principle of the present invention is provided in a computer provided at a position capable of mediating communication between a device and a terminal that displays a screen for managing the device and receives an operation from a user. , At least a part of which is installed and contributes to generation of an automation program for automatically performing processing related to device management. The program includes a program code for embedding a user interface for giving meaning to an operation in communication for displaying a management screen from the device to the terminal, and an operation from a user received through the management screen. And a program code for collecting and recording the operation information and the meaning information of the operation performed via the user interface in accordance with the communication to be transmitted from the terminal to the device. The automation program is generated by combining a plurality of recorded operation information according to the meaning information recorded together with the operation information.

  A program according to another aspect according to the principle of the present invention is provided at a position where each can display communication for a device and a terminal that receives an operation from the user by displaying a screen for managing the device. At least a part is installed in a server computer connected to a plurality of devices. This apparatus embeds a user interface for giving meaning to an operation in communication for displaying a management screen from the device to the terminal, and receives an operation from the user received via the management screen from the terminal. Accompanying the communication for transmitting to the device, the operation information and the meaning information of the operation performed via the user interface are collected, and the program for realizing the server is obtained from the apparatus. Processing related to device management by combining the program code for receiving the operation information and the meaning information and the received plurality of the operation information according to the meaning information received together with the operation information. Program code for generating an automated program for automatically performing

  According to one aspect of the present invention, there is provided a method in an apparatus provided at a position capable of mediating communication between a device and a terminal that displays a screen for managing the device and receives an operation from a user. In the communication for displaying the management screen from the device to the terminal, a user interface for giving meaning of the operation is embedded, and in the device, the operation received from the user via the management screen is transmitted from the terminal. Accompanying the communication for transmitting to the device, information on the operation and information on the meaning of the operation performed via the user interface are collected, and the collected information is transmitted to a server connected to the plurality of devices. Transmitting and combining the plurality of received operation information in the server according to the meaning information received together with the operation information. By Rukoto to generate an automated program for processing according to the device management automatically.

  In each of the above-described aspects, the operation information and the meaning information are collected using communication for transmitting the operation from the user received via the management screen from the terminal to the device. Alternatively, the communication may be performed using a communication (communication from the terminal to the device) different from the communication, or by a function equivalent to the device existing in the terminal (regardless of communication). You may go. The collection in the case of the latter two may be performed at completely different timing from the communication from the terminal to the device.

  Moreover, the apparatus in each aspect mentioned above, ie, the apparatus provided in the position which can mediate communication between a device and a terminal may be an apparatus connected to the network between a device and a terminal, It may be an apparatus provided integrally with the terminal inside the terminal, or may be an apparatus provided integrally with the device inside the device.

  A device provided inside the terminal and integrated with the terminal embeds a user interface in communication (for example, HTML data) input / output to / from screen display software (for example, a web browser) inside the terminal, Can be realized by software that inputs / outputs to / from (a program that operates separately from the browser or a plug-in that operates by being loaded into the browser). Similarly, an apparatus provided integrally with a device inside the device is embedded by a software that embeds a user interface in communication input / output to / from the web server function inside the device and inputs / outputs to / from a communication path outside the device. Can be realized.

  Each of the above-described modes can be in a different category, combined with another mode, or partially extracted into another mode. For example, a program for executing the above method, a method executed by the above device, a method executed by the above server, a system including the above device and the above server, the above device or the above server An apparatus that implements some functions or a device or terminal that communicates with the above apparatus, a method or program executed there, a storage medium that stores each of the above-described programs, and the like are each in accordance with the principle of the present invention. Can be.

  According to the present invention, for example, while a user performs an operation on a device management screen, information on the meaning of the operation can be naturally input in accordance with a user interface embedded therein, whereby device management can be performed. It is possible to automatically generate an automation program for automatically performing the processing according to the above. Therefore, for example, even when a large number of automation programs must be generated as in the case of an IoT device, it is possible to generate an automation program while reducing the burden on the user.

The figure which shows the meaning of the term for demonstrating the advantage of one Embodiment of this invention. The figure which shows the comparative example for demonstrating the advantage of one Embodiment of this invention. The figure for demonstrating an example of the advantage of one Embodiment of this invention. The figure which shows an example of the whole structure containing the system which concerns on one Embodiment of this invention. The figure which shows an example of the internal structure of each apparatus which concerns on one Embodiment of this invention. The figure which shows another example of the whole structure containing the system which concerns on one Embodiment of this invention. The figure which shows an example of the outline | summary of operation | movement of the system (henceforth this system) which concerns on one Embodiment of this invention. The figure which shows an example of the screen interface (graphic user interface: GUI) for making a user select the meaning of an input field in this system. The figure which shows an example of the information memorize | stored in the input field standard meaning DB (database) 102,205 in this system. The figure which shows an example of the operation | movement (GUI embedding part 106) for creating the input field meaning selection GUI (FIG. 8) in this system. The figure which shows an example of the flow (including the input field meaning receiving part 107) of the information collection by the input field meaning selection GUI in this system. The figure which shows an example of the information regarding the input field memorize | stored in collection data DB202 in this system. The figure which shows an example of the screen interface (GUI) for recording the operation procedure by a user in this system. The figure which shows an example of the information memorize | stored in automation object operation DB103,206 in this system. The figure which shows an example of the operation | movement (GUI embedding part 106) for creating the operation procedure recording GUI (FIG. 13) in this system. The figure which shows an example (in the case of a browser operation method) of operation | movement for recording the operation procedure in this system. The figure which shows an example of the operation | movement for recording an operation procedure in this system (in the case of an HTTP request system). The figure which shows an example of the flow (including the operation procedure recording part) of the information collection by the operation procedure recording GUI in this system. The figure which shows an example (in the case of a browser operation system) of the information of the meaningful operation record set memorize | stored in collection data DB202 in this system. The figure which shows an example (in the case of an HTTP request system) of the information of the meaningful operation record set memorize | stored in collection data DB202 in this system. The figure which shows an example of the operation | movement (input field meaning determination part 203) for determining the meaning of an input field in this system. The figure which shows an example of the information memorize | stored in input field meaning DB104,207 in this system. The figure which shows an example of the operation | movement (automated program creation part 204) for creating the automation program in this system. The figure which shows an example (in the case of a browser operation system) of the information memorize | stored in automation program DB105,208 in this system. The figure which shows an example (in the case of an HTTP request system) of the information memorize | stored in automation program DB105,208 in this system. The figure which shows an example of the information memorize | stored in device information DB111 and setting information DB112 in this system. The figure which shows an example of the operation | movement (automatic operation execution part 110) for performing an automation program in this system. The figure which shows an example (in the case of a browser operation system) of the automation program run in this system. The figure which shows an example (in the case of a browser operation method) of the flow of automatic operation by execution of the automation program in this system. The figure which shows an example (in the case of an HTTP request system) of the automation program run in this system. The figure which shows an example (in the case of an HTTP request system) of the flow of the automatic operation by execution of the automation program in this system. The figure which shows an example of the screen interface (GUI) for performing an automatic operation by a user's instruction | indication in this system. FIG. 33 is a diagram showing an example of an operation (GUI embedding unit 106) for creating an automatic operation GUI (FIG. 32) in the present system. The figure which shows an example of the method of executing an automation program using automatic operation GUI in this system. The figure which shows an example of the method of performing an automation program regularly in this system. The figure for demonstrating the application example of this system. The figure which shows an example of the information memorize | stored in each DB with which this system is provided in the example of FIG. The figure which shows an example of operation | movement in the case of changing the initial password of many devices collectively in the example of FIG. The figure which shows an example of the flow of the device initial password batch change in the example of FIG. 36, and the subsequent login authentication. The figure which shows the example which updates the content of input field standard meaning DB in this system, and the content of automation object operation DB. The figure for demonstrating the variation of the method of embedding GUI in this system.

  Hereinafter, embodiments of the present invention will be described, but the following description shows specific examples of the present invention and does not limit the scope of the present invention.

  In the aspect according to the principle of the present invention described above, the meaning may be performed by associating one operation name with a plurality of operations belonging to one process, and the user interface performs the operation. An operation name candidate may be displayed on the management screen for receiving and the user may select the operation name.

  As a result, when a user performs a series of operations, if an operation name representing a process performed by these operations is associated, an automated program corresponding to the series of operations is automatically generated and operated thereafter. It becomes possible to give a name. In addition, since the operation name can be associated with the candidates displayed on the user interface, even an unfamiliar user can easily perform the operation name association.

  In the aspect in accordance with the principle of the present invention described above, the apparatus executes the automation program instead of receiving a plurality of operations from the user via the management screen, thereby allowing the plurality of operations to be performed on the device. There may be further provided means for creating a communication for transmission.

  That is, an apparatus that collects operation information and semantic information (learns an operation method) according to the principle of the present invention contributes to generation of an automation program, and executes the automation program generated thereby. The means may be provided in a device management apparatus different from the apparatus, but may be provided in the apparatus itself. Advantages of providing the device itself will be described later.

  In the aspect according to the above-described principle of the present invention, the meaning may be performed by defining the meaning of the field when the operation is to input a value in the field, and the user interface is A configuration may be included in which candidates for field meaning are displayed on the management screen for receiving the operation, and the user selects the meaning of the field.

  As a result, when a user performs an operation of inputting a value into a field, if the user also specifies the meaning of the field (which determines what value should be input), then the user automatically In addition, it is possible to generate an automation program including input of values in the field. In addition, since the instruction of the meaning of the field only needs to be selected from the candidates displayed by the user interface, even an inexperienced user can easily perform the instruction.

  In the aspect in accordance with the principle of the present invention described above, when the apparatus including the means for executing the generated automation program inputs the value to the field, the device is associated with the meaning of the field, Means for storing a value that may be different from each other and communication for transmitting an operation including an input value to a field to the device by executing the automation program is stored in association with the meaning of the field for the device. And means for producing using the value.

  Here, the means for storing the value for each device in association with the meaning of the field may be provided in the apparatus in advance as a database, or stores information that can be acquired from a server on the cloud side. Alternatively, information on values obtained by collecting operation information and meaning information from the user may be stored. Alternatively, when an automated program including input of a value to a field is executed, it may be stored by indicating the meaning of the field and allowing the user to input a value. In the case of the former three, it becomes possible to execute the automation program completely automatically. In the latter case, the automation program can be executed simply by entering a value on the spot for an unregistered device. Device management can be performed.

  In the aspect according to the principle of the present invention described above, the meaning may be performed so that a plurality of operations over a plurality of management screens are associated as a plurality of operations belonging to one of the processes. The user interface may display the management screen for receiving the operation so that the user can specify the start and end of the series of screens.

  That is, a plurality of operations belonging to one process (corresponding to one operation name) may be within one management screen, but may be over a plurality of management screens. In particular, in the case of a plurality of management screens, it is convenient to have a user interface that can specify from which management screen the process starts and ends at which management screen.

  In the embodiment according to the principle of the present invention described above, there are a plurality of different types of devices as the device, and the management screen of one type of device and the management screen of another type of device may have different configurations. In some cases, the user interface may be created so that the same meaning is given to an operation on a certain type of device and an operation on another type of device.

  As a result, even when the operation contents of the management screen, the display contents of the input fields, etc. are different for each product, it is possible to generate an automation program by giving meanings common to those products. If you want to automatically change passwords, update firmware, etc.) for all products, specify the same processing with the same meaning (instructions for the same operation name) without being aware of product differences. Thus, the automation program can be executed for all devices.

  In the aspect according to the principle of the present invention described above, when the operation is to input a value into a field, the meaning is performed by defining the meaning of the field, and the user interface is Even if the field in the operation and the field in another operation are shown in different expressions, they may be created so that the same meaning can be given.

  Thus, for example, when different names are displayed depending on the management screen even though the fields originally have the same meaning (even for the same device, different names depending on the management screen in the same meaning field) The same meaning can be given, and the same value can be automatically entered in the field with the same meaning without being aware of the name difference due to the management screen. Is possible. In addition, when a means for storing a value to be input to the field is provided in association with the meaning of the field, the storage can be aggregated, so that the storage means can be easily managed.

  In the aspect according to the principle of the present invention described above, when there are a plurality of terminals used by different users as the terminal, meaning information recorded by one user and recorded by another user for the same operation The semantic information given may be different. In that case, the generation of the automation program may be performed by determining information on a plurality of operations to be combined according to the amount of recording of different meaning information for the same operation.

  For example, if most users assume that the meaning is correct for each operation in the majority of cases (a few users rarely make the meaning), each operation is recorded frequently. What is necessary is just to generate an automation program using the meaning given. Whether or not the frequency is high may be determined, for example, based on whether or not the absolute recording count of the meaning is greater than or equal to a predetermined value, or the ratio of the meaning when the record of the predetermined number of operations is accumulated. It may be determined by whether or not is greater than or equal to a predetermined value. Or, it may be possible to adopt a reasonable meaning by giving a high weight to the meaning given by a user who is experienced in device management, or by removing the meaning that is understood as an error by analyzing the meaning. .

  In the aspect according to the principle of the present invention described above, the operation is performed by the device toward a server that generates the automation program based on the operation information and the meaning information recorded in each of a plurality of the devices. And means for transmitting the meaning information may be further provided. The plurality of devices are used by different users, and the generation of the automation program includes information on a plurality of operations to be combined according to the number of records of different meanings for the same operation. You may make it carry out by determining.

  By providing the server on the cloud side when viewed from a plurality of the devices, it becomes possible to collect operation information and meaning information from more users (form collective intelligence). Then, even if there are a considerable number of users who are unfamiliar with the operation, it is possible to generate an automated program by giving reasonable meaning as a whole in the server on the cloud side.

  In the aspect in accordance with the principle of the present invention described above, the generation of the automation program includes recording by a user who made meaning corresponding to the process including operations unnecessary as a plurality of operations belonging to one process, Corresponding to the recording by the user who gave meanings corresponding to the processing without including the operations required as a plurality of operations belonging to the processing, and processing different from the processing for a plurality of operations belonging to the processing Excluding at least one of the recording by the user who made the meaning and the recording by the user who made the meaning different from the meaning of the operation for one or more operations belonging to the one process May be performed.

  Such analysis can be performed automatically by collecting operation information and meaning information from a large number of users, so that each user performs operations on the device management screen in the initial stage. However, if information on the meaning of the operation is input according to the user interface, the automatically generated automation program can be used regardless of the creation of the automatic program, and the burden is reduced. .

  In the aspect according to the principle of the present invention described above, the user interface displays information for supporting the meaning of the operation on the management screen for receiving the operation, and the meaning of the operation is indicated to the user. May be included, and the apparatus may further include means for storing information to be referred to in order to create information for the support.

  Here, the means for storing the information to be referred to for the display that supports the meaning of the operation may be built in the apparatus in advance, or stores information that can be acquired from the server on the cloud side. There may be. The display supporting the meaning of the operation may be, for example, the above-described operation name candidates or field meaning candidates.

  This makes it possible to focus on the information necessary for generating an automation program, and to display information to assist and collect information only within that range, so it is collected compared to an increase in the amount of automation programs to be generated. The amount of information to be reduced can be reduced.

  Furthermore, the apparatus may include means for updating the information to be referred to based on meaning information obtained from a plurality of users.

  This update may also be performed based on operation information and meaning information collected by the device, or may be performed based on operation information and meaning information collected more broadly by the server on the cloud side. Good.

  By referring to the updated information, for example, the above-mentioned operation name candidates and field meaning candidates that are narrowed down to those that seem to be highly relevant are displayed, or an automated program has already been generated. For example, candidates may not be displayed, and the burden on the user can be further reduced.

  In addition, the support information may be created so that semantic information obtained from a plurality of users for the same operation is the same.

  For example, when an operation that is the original purpose is performed by tracing a plurality of management screens having a hierarchical structure, whether the operation in the screen transition that follows the hierarchical structure belongs to one process or the operation that is the original purpose is Whether the operation in the hierarchy to be performed should be extracted and belong to one process, or an operation that is always performed to perform a certain process but belongs to another process is treated as belonging to the certain process, If it is not determined systematically which policy should be adopted, the meaning of each user may vary.

  For such meanings, most users have the same meaning by displaying for support in accordance with a predetermined policy. It is possible to automatically generate an automation program under the assumption that meanings for operations are correctly performed.

  In the aspect in accordance with the principle of the present invention described above, the device transmits a communication for transmitting an operation from a user received via the management screen from the terminal to the device, to the device, and It is also possible to further comprise means for creating a communication for transmitting a plurality of operations belonging to the process to the device by executing the automation program and transmitting the communication to the device.

  As described above, when the device itself that collects the operation information and the meaning information includes a means for executing the automation program, device management (and learning of the operation method associated therewith) by the operation from the user, Since device management by execution can coexist, for example, various processes related to device management are performed by operations from the user, and automation is performed sequentially from the process in which sufficient information for generating the automation program is collected It is possible to perform operations such as replacing with a program (some processing is performed by a user operation, and another processing is performed by executing an automated program).

  The apparatus further includes means for embedding a user interface for designating the processing and starting execution of the automation program in communication for management screen display from the device to the terminal, When a process is designated by the user via the above, execution of the automated program for automatically performing the process may be started.

  As a result, the process designated by the user can be performed by executing the automation program, and the other processes can be performed by an operation from the user. In addition, the same processing can be performed by executing an automated program at one time and by a user operation at another time.

  In the embodiment according to the principle of the present invention described above, an apparatus comprising means for executing the generated automation program further comprises means for setting a condition under which execution of the automation program should be automatically started, and the condition is satisfied. Then, the execution of the automation program may be started.

  As a result, for example, processing that should be performed periodically, processing that should be performed when the specified date and time arrives, processing that should be performed when the device is in a predetermined state, and the like are performed by executing the automation program without any user intervention It becomes possible.

  In the aspect according to the principle of the present invention described above, there is provided a server for generating the automation program based on the operation information and the meaning information recorded in each of the plurality of devices, and means for executing the automation program The apparatus provided (the apparatus or another apparatus) may further include means for receiving the automation program.

  The apparatus or another apparatus includes means for creating a communication for transmitting a plurality of operations belonging to the process to the device by executing the automation program, and transmitting the communication to the device The server may further include means for distributing the generated automation program to the apparatus.

  By using the above server as a server on the cloud side, it is possible to collect information from more users and generate an automated program with high accuracy.

  In the aspect in accordance with the principle of the present invention described above, the operation information includes the contents of communication (for example, an http request) for transmitting the operation from the user received from the terminal to the device via the management screen. It is good.

  The content of the operation conveyed by the http request is the content of the corresponding http response (communication for displaying the management screen from the device to the terminal in which a user interface for meaning of the operation is embedded). Therefore, the operation information may be the content of an http response and the content of an http request.

  In the aspect according to the principle of the present invention described above, the operation information may include the contents of the screen operation performed by the user on the management screen.

  As a result, content that is not included in the http communication message during the screen operation by the user (for example, when the value of the input field is converted on the web screen by JavaScript (registered trademark) or the like, the content of the http communication message) From this, it is possible to generate an automation program including a value input on the web screen).

  In the aspect in accordance with the principle of the present invention described above, the functions of the device may be integrated and provided in each of the terminals, or the device may be provided separately from the terminals and provided to a plurality of the terminals. You may make it connect.

  Hereinafter, as an example, a device management automation system according to an embodiment of the present invention will be described with reference to the drawings.

  First, the advantages of this system will be described with reference to FIGS.

  As shown in FIG. 1 (a-1), all the operations by the user are automatically recorded, and what is unclear what each operation means (for example, information collected by Patent Document 1) This is called meaningless operation recording. This operation corresponds to, for example, a browser operation (clicking <a ... /> tag or input to <input ... /> tag) or HTTP communication. In this state, as shown in FIG. 1A-2, a series of operations that are automatically extracted by analyzing meaningless operation records are referred to as meaningless operation record sets. Since it is unclear what this set means and what each operation means, the meaningless operation record set cannot be used as an automation program as it is.

  On the other hand, as shown in FIG. 1 (b), a user's operation is recorded in a state that is given meaning by a human, and a set of a series of operations that are grouped by the meaning is called a meaningful operation record set. Call. The information collected in this embodiment is this meaningful operation record set. The meaning of the set (operation name) and the meaning of each operation (including the meaning of the input field) have been obtained, but if there are extra operations by humans or incorrect meanings, use them as they are as an automation program. I can't.

  Therefore, an automated program (a program that automatically executes an operation equivalent to a user's operation) as shown in FIG. 1C is meaningless by humans with respect to the meaningless operation record set in FIG. It is created either by attaching or automatically removing a meaning error from the meaningful operation record set of FIG. 1B.

  FIG. 2 shows an example of a system in which a human gives meaning to the meaningless operation record set shown in FIG. In this example, first, in order to create a meaningless operation record DB, information on the operation (1) performed manually on the device management screen is collected. Next, a set of regularly performed operations is automatically extracted from the meaningless operation record DB (2) to create a meaningless operation record set DB. Finally, manually (using a GUI, for example), a meaningless operation record set related to the operation to be automated is selected, and an automation program is created (3) by giving meanings and corrections.

  The last manual task is to select a meaningless operation record set to focus on as an object of automation from a large number of meaningless operation record sets (in the example in the figure, related to login authentication and password change), and make a meaning. Therefore, the workload becomes heavier as more settings can be made on the device. Login authentication and password change are targets that need to be automated to automatically change to a secure password.

  On the management screen of the IoT device, various processes can be performed by a user operation. For example, if the device has a camera, login authentication, password change, firmware update, factory reset initialization, IP address change, wireless setting, time setting, mail setting, video image setting, video display setting Video night vision setting, video privacy setting, video volume setting, motion detection range, motion detection schedule, storage destination setting, storage schedule, SD card storage setting, QR code (registered trademark) creation, and the like.

  Therefore, in particular, in the case of an IoT device, the size of the meaningless operation record DB and the meaningless operation record set DB increases, and a human selects a set to be automated from the meaningless operation record set DB and creates an automation program. Becomes heavier. Furthermore, since a wide variety of products are distributed in IoT devices, even if the processing has the same meaning, it depends on the manufacturer of the product, the category to which the product belongs, and the same category depending on the product model. This problem grows even more because the same product can be a collection of different operations through different management screens depending on the type of terminal that manages the product.

  FIG. 3 shows an example of a system (this system) that automatically removes a meaning error from the meaningful operation record set of FIG. 1B and creates an automation program. In this example, when an operation (1) performed manually on the device management screen is recorded, a meaningful operation record set is collected by simultaneously giving meanings of operations by humans.

  At this time, the system embeds a GUI for supporting meaning in the device management screen so that the meaning is easily given by a human. Among these, the operation procedure record GUI supports the meaning of the set (for example, determination of the operation name indicating which process each operation belongs to), and the input field meaning selection GUI indicates the meaning of the operation (for example, its operation). Determine the meaning of the input field the operation contains). By providing these GUIs, humans can naturally give meaning information to the system while performing operations using the device management screen, so a large number of humans (users using the device management screen) ) Makes it possible to collect a meaningful operation record set.

  As shown in FIG. 3, the system then automatically creates (2) an automated program from the meaningful operation record set DB. The meaningful operation record set DB includes a set (3) including an extra operation for the process of interest, a set (not shown) not including an operation necessary for the process of interest, and a series of processes belonging to the process of interest. Mistakes in meaning associated with different processes (4), misrepresentations in an operation or input fields included in the operations (5), etc. may be included. Assuming that the frequency is reduced, and excluding them based on the frequency, automatic generation (2) of the automation program becomes possible.

  As described above, in this system, it is possible to reduce the burden of creating an automated program by humans by automating the generation of an automated program after collecting information. Further, in this system, information is collected only for operations belonging to the process to be automated, so the size of data to be collected can be reduced.

  Therefore, this system can reduce the influence due to the increase in the number of products to be automated and the number of operations, and is also suitable for creating an automation program for management screen operations of IoT devices. Even for improper management of IoT devices, which is currently a problem, this system collects information related to device management screen operations from a large number of users and generates programs that automate operations for various products. Can be solved. In this system, it is more effective if the generated automation program is shared by many users and appropriate management operations of the IoT device are widely automated.

  FIG. 4 shows an example of the overall configuration of this system. As shown in FIG. 4A, the components of this system in this example are a device 30, a client 40, a client side device 10, a cloud side device (server) 20, and a network 50 connecting them.

  Here, the device 30 is, for example, a television, a camera, a printer, or the like, and can be an IoT device. The client 40 is a terminal such as a personal computer or a smartphone, and a user (for example, a person who owns the device) operates the management screen of the device 30 using the browser of the client 40. Even if the devices are different, if they are devices of the same product, the management screen and the operation using the same when viewed from the same terminal are the same.

  The client-side device 10 relays HTTP communication between the client 40 and the device 30, collects information related to device management screen operations performed by the user on the client 40, and performs operations on the device 30 via the management screen. Or do it automatically. The cloud side device (server) 20 generates an automation program for automatically performing an operation via the management screen based on the information regarding the operation of the device management screen collected by the client side device 10, and uses the generated automation program as a client Delivered to the side device 10.

  FIG. 4B shows a specific configuration example of FIG. In this example, the client side device 100 is installed for each local area network (LAN) 502 and relays HTTP communication between the device 30 and the client 40 in the LAN. The cloud side device (server) 200 collects information related to the operation of the device management screen from the plurality of client side devices 100 via the Internet 501 and distributes the generated automation program. The client 40 and the device 30 may be connected to the LAN 502 by wire or may be connected wirelessly via the access point 503. A plurality of devices 30 may be managed from one client 40, and one device 30 may be managed from a plurality of clients 40.

  FIG. 5 shows an example of functional configurations of the client side device 100 and the cloud side device (server) 200. Each functional unit may be configured by software, hardware, or a combination thereof.

  The client-side device 100 stores a data reception unit 101 that receives data from the data transmission unit 209 of the server 200 and a copy of part or all of the information in the DBs 205 to 208 of the server 200 received by the data reception unit 101. The data from the DBs 102 to 105, the GUI embedding unit 106, the input field meaning receiving unit 107, the operation procedure recording unit 108, the input field meaning receiving unit 107, and the operation procedure recording unit 108. And a data transmission unit 109 for transmission to the network.

  The GUI embedding unit 106 embeds the input field meaning selection GUI and the operation procedure recording GUI in the HTTP response transmitted from the device 30 to the client 40, and transmits it to the client 40.

  The input field meaning receiving unit 107 receives data from the input field meaning selection GUI and passes it to the data transmitting unit 109 as input field information. The operation procedure recording unit 108 receives data from the operation procedure recording GUI, records the HTTP communication between the client and the device and the browser operation, and passes them to the data transmission unit 109 as a meaningful operation record set.

  The client-side device 100 may include an automatic operation execution unit 110 in addition to the above-described function units or separately from the device having the function units. The automatic operation execution unit 110 automatically performs an operation via the device management screen using an automation program.

  In order to execute this automation program, the GUI embedding unit 106 may embed an automatic operation GUI in the HTTP response transmitted from the device 30 to the client 40 and transmit it to the client 40.

  The client side device 100 inputs the device information DB 111 for storing the MAC address, product name, etc. of each device 30 and each field of the device management screen as a DB for storing information used by the automatic operation execution unit 110. A setting information DB 112 for storing values may be provided.

  The cloud side device (server) 200 stores the data received from the data transmission unit 109 of the client side device 100 in the collected data DB 202 and the collected input field information and the meaningful operation record set. Collected data DB 202, the input field meaning determination unit 203 that determines the meaning of the input field from the input field information stored in the collected data DB 202, and saves it in the input field meaning DB 207, and the meaningful operation stored in the collected data DB 202 An automation program creation unit 204 that generates an automation program from a record set and stores it in the automation program DB 208, and a data transmission unit that transmits a copy of part or all of the information in the DBs 205 to 208 to the data reception unit 101 of the client side device 100 209 and A.

  The input field standard meaning DB 205 of the server 200 stores the input field standard meaning, and the client-side apparatus 100 displays an input for displaying a copy of this information stored in the input field standard meaning DB 102 on the input field meaning selection GUI. Use as a candidate for field meaning.

  The automation target operation DB 206 of the server 200 stores the operation name of the automation target, and the client side device 100 has an operation name for displaying a copy of this information stored in the automation target operation DB 103 on the operation procedure recording GUI. Used as a candidate or as an operation name to be displayed on the operation target of the automatic operation GUI.

  The input field meaning DB 207 of the server 200 stores the input field meaning determination result, and the client side apparatus 100 executes an automation program using a copy of this information stored in the input field meaning DB 104.

  The automation program DB 208 of the server 200 stores the generated automation program, and the client side apparatus 100 executes the automation program using a copy of this information stored in the automation program DB 105.

  FIG. 6 shows another example of the overall configuration of the present system. As shown in FIG. 6A, the components of this system in this example are a device 30, a client 40, an automation device 60 that doubles as a client side device and a cloud side device (server), and a network 50 that connects them. It is.

  FIG. 6B shows a specific configuration example of FIG. In this example, the automation device 600 is installed in the LAN 502. The automation apparatus 600 includes, for example, each of the DBs 102 to 105, the GUI embedding unit 106, the input field meaning receiving unit 107, the operation procedure recording unit 108, the collected data DB 202, the input field meaning determining unit 203, and an automation program creation. The unit 204 may be included.

  The automation device 600 may include an automatic operation execution unit 110 in addition to the above function units or separately from the devices having the above function units. For this purpose, the device information DB 111 and the setting information DB 112 are provided. You may prepare.

  FIG. 6B shows an example in which one automation apparatus 600 collects information related to operations under the control of all devices 30 and all clients 40. However, a plurality of automation apparatuses 60 are provided and each automation is performed. The apparatus 60 may collect information regarding operations regarding the devices 30 and clients 40 under its control, and may collect information regarding operations regarding the devices 30 and clients 40 not under its control from other apparatuses. . Then, an automation program generated by a certain automation device may be distributed to another automation device.

  As another configuration example, in order to reduce communication between the client side device 100 and the cloud side device (server) 200, or to distribute the processing performed by the server 200, an edge is included in the LAN 502 in FIG. A device may be installed. The edge device in this case has a function of acting as a proxy for some functions of the server 200 with respect to the client side device 100 and relaying communication between the client side device 100 and the server 200.

  As another configuration example, the automation device 60 may be installed in the cloud (on the other side of the Internet 501 as viewed from the device 30) instead of being installed in the LAN as shown in FIG. In this case, communication from the cloud to the device 30 is enabled.

  As another configuration example, the function of the client side device 10 or the function of the automation device 60 may be distributed as software, and the software may be installed and used in each client 40. This software may be a program (FIG. 41 (b)) that operates separately from the browser, or may be a plug-in (FIG. 41 (c)) that is loaded into the browser and operates.

  In the present embodiment, an example (FIG. 41A) in which the client side device 10 or the automation device 60 is connected to the network between the device 30 and the client 40 will be mainly described. However, this example can be realized by embedding a GUI in an HTTP response (more specifically, HTML data therein) transmitted from the Web application of the device 30 to the browser of the client 40. Therefore, it is clear that the example of installing software in the client 40 (FIGS. 41B and 41C) can be similarly realized by embedding the GUI in the HTML data.

  Although this system is effective when used by a large number of users, it may be used by a small number or a single user.

  FIG. 7 shows an outline of the operation of this system. The system first collects operation methods (information related to operations) on the device management screen from a large number of users (700). There are two types of contents to be collected: input field information and meaningful operation record sets.

  The system then analyzes the collected information and performs input field semantic determination and automation program generation (800). The input field meaning is information used to determine which value is input to which input field when the automation program is executed.

  Thereafter, the system automatically operates the device management screen using the generated automation program (900).

  A method for collecting information related to input fields on the device management screen will be described with reference to FIGS. FIG. 8A shows an example of a device management screen.

  FIG. 8B is an example in which a link to the input field meaning selection GUI is embedded in the device management screen (FIG. 8A). In this example, when the meaning of the input field is unknown (unknown or uncertain for this system), a “?” Icon is displayed, and when the user clicks this icon, the input field meaning selection GUI (FIG. 8 ( c)) is displayed.

  The input field meaning candidates displayed on the input field meaning selection GUI in FIG. 8C are preferentially displayed using the tag ID or label name of the target input field. Since the user can easily select the input field meaning using this GUI, the system can collect input field information from many users.

  FIG. 9 shows an example of information stored in the input field standard meaning DBs 102 and 205. The input field standard meaning is used as a candidate input field meaning to be displayed on the input field meaning selection GUI.

  The input field standard meaning is stored in association with highly relevant operations (related operations), input field tag IDs, and label names (related keywords). Such information is registered in advance by the manufacturer or administrator of this system, but it is also possible to dynamically register and update related operations and related keywords as shown in FIG.

  FIG. 10 shows the GUI embedding unit 106 for extracting the input field in the HTTP response and creating the input field meaning selection GUI as required (for example, when the meaning of the extracted input field is unknown). Shows the details of the operations performed by.

  First, an HTTP response from the device 30 to the client 40 is analyzed, and an input field included in the HTTP response is extracted (710). Then, for all the extracted input fields (712), the input field meaning DB 104 is referred to and it is confirmed whether the meaning of the input field has been determined (714). If not yet determined (716), the input field meaning A selection GUI is created (722).

  At that time, in order to display the input field meaning candidates on the input field meaning selection GUI, the input field standard meaning DB 102 is referred to, and the highly likely input field standard meaning is acquired (718). As input field standard meanings having high possibility, for example, all input field standard meanings stored in the DB 102 may be acquired, or input field standard meanings in which an operation being performed matches a related operation are acquired. Alternatively, the input field standard meaning in which the input field tag ID or the label name matches the related keyword may be acquired. In the example of FIG. 8C, the “user name” of the label name is used to acquire “login ID” and “person name” that match the related keywords in FIG.

  Next, the acquired input field standard meanings are rearranged in the descending order of possibility (720). For example, the possibility of the input field standard meaning that the operation being performed matches the related operation and the input field tag ID or the like matches the related keyword is high. You may judge that the possibility of the input field standard meaning selected frequently among the input field information collected so far is high.

  Finally, an input field meaning selection GUI is created, and the acquired input field standard meaning is displayed as a candidate for the input field meaning (722), so that the user can select the meaning of the input field from the standard meaning. .

  FIG. 11 shows an example of an operation sequence of the user 45, the client 40, the client side device 10, the device 30, and the cloud side device (server) 20 until the input field information is collected using the input field meaning selection GUI. ing. When the client-side device 10 is provided with the function of the cloud-side device 20, the automation device 60 operates as the client-side device 10 in the figure.

  The user 45 activates the browser of the client 40 and accesses the authentication screen of the device 30. Then, an HTTP request is transmitted from the client 40 to the client side device 10 in order to obtain an authentication screen. The client side device 10 transmits the HTTP request as it is to the device. The device 30 transmits an HTTP response (FIG. 8A) that is a message for displaying the authentication screen to the client side device 10.

  When the client side apparatus 10 receives the HTTP response (FIG. 8A) from the device 30, the client side apparatus 10 analyzes the HTTP response and selects the input field meaning by the input field meaning selection GUI creation processing (the details of the processing are shown in FIG. 10). A GUI (FIG. 8C) is created. Then, an HTTP response (FIG. 8B) in which the created links to FIG. 8C and FIG. 8C are embedded is created and transmitted to the client 40.

  The user 45 clicks the link from FIG. 8B to FIG. 8C displayed on the browser of the client 40, and selects an appropriate input field meaning. The selected input field meaning is transmitted to the client side device 10 as input field information. The client side device 10 transmits the input field information to the cloud side device (server) 20, and the cloud side device (server) 20 registers the received input field information in the collection data DB 202.

  FIG. 12 shows an example of input field information registered in the collected data DB 202 in this way. The device ID and client IP are examples of information for identifying a plurality of existing devices (client-side device 10 or automation device 60) and client 40. A user name may be included to identify the user 45. The input field standard meaning selected by the user is stored in association with input field identification information (for example, product name, screen name, input field tag ID, etc.). Even if the device 30 is the same product, if the management screen differs depending on the type of the client 40, the type of the client may be recorded at the same time.

  With reference to FIGS. 13 to 20, a method for collecting information (an operation record set with meaning) related to the operation procedure of the device management screen will be described. FIG. 13A is an example of a device management screen.

  FIG. 13B is an example in which the operation procedure recording GUI is embedded in the device management screen (FIG. 13A). FIG. 13C is an example when the select box of the target operation of the operation procedure recording GUI is clicked. In this example, the display content of the operation procedure recording GUI changes between before operation recording (FIG. 13D) and during recording (FIG. 13E).

  In the select box for the target operation, an operation name of a process for which an automation program has not yet been generated is displayed (FIG. 13C). Here, the user selects the operation name of the process to which the operation to be performed belongs as the target operation, and clicks the start button (FIG. 13D). Next, the user performs a series of operations (which may be completed within one management screen or may span a plurality of management screens) belonging to the processing of the selected operation name. Meanwhile, the operation procedure recording GUI is displayed during operation recording, and the user clicks an end button (FIG. 13E) at the end of a series of operations to finish recording.

  FIG. 14 shows an example of information stored in the automation target operation DBs 103 and 206. The operation name is used as the operation name displayed in the select box of the target operation of the operation procedure recording GUI or the automatic operation GUI.

  The operation name is stored in association with the classification, importance, and related keywords. Classification is used to distinguish between operations common to all products and operations common to some products. The importance level is used to express the importance of generating the automation program, and the importance level is increased as the operation is necessary for device management. Such information is registered in advance by the manufacturer or administrator of this system, but as shown in FIG. 40, related keywords can be dynamically registered and updated.

  FIG. 15 shows details of an operation performed by the GUI embedding unit 106 in order to create an operation procedure recording GUI embedded in the HTTP response.

  First, the operation name displayed on the operation procedure record GUI is acquired from the automation target operation DB 103 (730). At this time, only the operation name “classification = common” may be acquired, or the operation name “classification = unique” may be acquired as necessary. Then, referring to the automation program DB 105, the operation names for which the automation program has been created are excluded from the acquired operation names (732).

  Next, the acquired operation names are rearranged in descending order of priority (734). For example, based on the importance level and related keywords stored in the automation target operation DB, it may be determined that the priority of the operation name including the related keyword in the embedded HTTP response is high. It may be determined that the priority of the operation name with high importance is high, or the frequency of the operation name selected by the user is measured for each screen, and the priority of the operation name with high selection frequency is determined to be high Good. Also, operation names with low priority may be excluded so as not to be displayed.

  Finally, an operation procedure recording GUI is created, and the acquired operation name (operation name not excluded) is displayed in the select box of the target operation (736), whereby the user selects the operation name and starts and ends recording. Can be specified.

  FIG. 16 and FIG. 17 show the details of the operation recording operation procedure in the browser operation method and the HTTP request method, respectively. In the case of the browser operation method, an automation program is generated from HTTP communication and browser operation, and in the case of the HTTP request method, the automation program is generated only from HTTP communication.

  In FIG. 16, first, it is determined whether the HTTP request received from the client is a browser operation notification such as a button click or input to an input field or an HTTP request to the device (740). If it is a notification of browser operation, the operation content is recorded (742). If it is an HTTP request to the device, the content is recorded (744) and transmitted to the device (746). Then, after receiving and recording (748) the HTTP response from the device, an operation procedure recording GUI is created as shown in FIG. 15 (750), embedded in the HTTP response, and transmitted to the client (752). The operation procedure recording GUI at this time displays the content being recorded.

  The content of the operation in FIG. 17 is the same as that in the case where the conditional branch in FIG.

  FIG. 18 illustrates an example of an operation sequence of the user 45, the client 40, the client side device 10, the device 30, and the cloud side device (server) 20 until the meaningful operation record set is collected using the operation procedure recording GUI. Show. When the client-side device 10 is provided with the function of the cloud-side device 20, the automation device 60 operates as the client-side device 10 in the figure.

  The user 45 activates the browser of the client 40 and accesses the authentication screen of the device 30. Then, an HTTP request is transmitted from the client 40 to the client side device 10 in order to obtain an authentication screen. The client side device 10 transmits the HTTP request to the device 30 as it is. The device 30 transmits an HTTP response (FIG. 13A), which is a message for displaying the authentication screen, to the client side device 10.

  When the client side apparatus 10 receives the HTTP response (FIG. 13A) from the device 30, the client side apparatus 10 embeds the operation procedure record GUI created by the operation procedure record GUI creation processing (the details of the processing is FIG. 15) (FIG. 13). 13 (b)) is created and transmitted to the client 40.

  The user 45 selects the operation name to be operated in the operation procedure record GUI (FIG. 13C) displayed on the browser of the client 40, and clicks the start button. Simultaneously with the click of the start button, the display content of the operation procedure recording GUI changes from before recording (FIG. 13D) to during recording (FIG. 13E).

  The user 45 performs an operation belonging to the processing of the selected operation name on the device management screen displayed on the browser of the client 40. During recording, all browser operation information such as button clicks and input fields and HTTP communication information between the client 40 and the device 30 are all sent to the client side device 10 and recorded (details are shown in FIG. 16 or FIG. 17).

  The communication from the client 40 to the client side device 10 for recording the operation procedure may be performed through HTTP communication from the client 40 to the device 30 via the client side device 10, or from the client 40 to the device 30. The communication may be performed as a communication different from the HTTP communication.

  At this time, it is effective to make it difficult for the difference in the operation associated with the operation name to occur due to the judgment of each user. For example, it is highly likely that the determination of whether or not to include the operation until reaching the IP address change screen “top screen → menu list → IP address change” in the operation name “IP address change” varies depending on the user. For this reason, when the start button is clicked, it is preferable to forcibly return to the top screen so that the user can operate from the top screen.

  When the user finishes the last operation belonging to the processing of the selected operation name and clicks the end button of the operation procedure recording GUI, the display content of the operation procedure recording GUI is changed to the content before recording (FIG. 13D). change. Also, the client side device 10 transmits the contents recorded so far to the cloud side device (server) 20 as a meaningful operation record set. The cloud side device (server) 20 stores the received meaningful operation record set in the collected data DB 202.

  19 and 20 show an example of information on the meaningful operation record set registered in the collection data DB 202 in this way (FIG. 19 is for the browser operation method, and FIG. 20 is the HTTP request method). .

  In FIG. 19, the device ID and client IP are examples of information for identifying a plurality of existing devices (client-side device 10 or automation device 60) and the client 40. A user name may be included to identify the user 45. Information related to the operation performed by the user (operation type, URL, operation content, HTTP body part, operation date, etc.) is stored in association with the product name and operation name. Even if the device 30 is the same product, if the management screen differs depending on the type of the client 40, the type of the client may be recorded at the same time.

  20 is different from FIG. 19 in that “input parameters” are stored instead of “operation contents”. Other than that, the browser operation method and the HTTP request method store the same contents.

  With reference to FIGS. 21 to 25, a method for analyzing information about collected operations will be described.

  FIG. 21 shows an example of an operation in which the input field meaning determination unit 203 determines the input field meaning from the input field information (FIG. 12) collected from the user.

  First, input field information is acquired for each input field from the collected data DB 202 (810). Each input field is identified by a product name, a screen name, and an input field tag ID.

  Next, the frequency and ratio of the input field standard meaning in the acquired input field information are calculated (812).

  If the input field has a standard meaning that has the highest ratio and the frequency and ratio are equal to or greater than the threshold, it is determined that the input field has a meaning (814). By doing so, a meaning error as shown in FIG. If there is no input field standard meaning that exceeds the threshold, the determination is not possible.

  FIG. 22 shows an example of information on the determined input field meaning, and this information is stored in the input field meaning DBs 104 and 207.

  The input field meaning judgment result (input field standard meaning, judgment result) is information (product name, screen name, input field tag ID) for identifying the input field, together with information (selection frequency and selection ratio) that is the basis of the judgment. ) And saved. If there is no information in the input field meaning DB 104 or if the determination result cannot be determined, it is determined that the meaning of the input field is unknown.

  FIG. 23 shows an example of an operation in which the automated program creation unit 204 creates an automated program from the collected collection of meaningful operations (FIG. 19 or 20).

First, a meaningful operation record set is acquired for each product name and operation name from the collected data DB 202 (820). A meaningful operation record set refers to a series of operations recorded from when the user clicks the start button and clicks the end button using the operation procedure record GUI. It can be acquired by searching from the DB 202.
(A) Product name and operation name are the same (B) Device ID is the same (C) Client IP address is the same (D) Satisfies (A) to (C) above, and “Operation type” is “Record start” Records included during “recording end” Then, operations included at high frequency are extracted from all acquired meaningful operation record sets (822). By doing so, an extra operation of the user as shown in FIG. 3 (3) and a meaning error as shown in FIG. 3 (4) are excluded.

  Finally, the extracted operation is converted into an automation program format and registered in the automation program DB 208 (824).

  24 and 25 show examples of information stored in the automation programs DB 105 and 208 in the case of the browser operation method and the case of the HTTP request method, respectively.

  The information of the automation program stored in FIG. 24 is obtained by associating the order of operations to be performed and the operation contents (URL, input field tag ID, input value acquisition method, operation) for each product name and operation name. is there.

  The information of the automation program stored in FIG. 25 associates the order of HTTP requests to be transmitted and the message contents (URL, input field tag ID, input value acquisition method, operation) for each product name and operation name. Is.

  In the browser operation method, the content registered in “operation” is the operation content performed on the browser, whereas in the HTTP request method, the operation content is related to HTTP request creation.

  As shown in FIG. 40 as an example, by using information registered in the input field meaning DBs 104 and 207 and the automation programs DB 105 and 208, the input field standard meaning DBs 102 and 205 and the automation target operation DBs 103 and 206 are stored. The stored related operations and related keywords can be updated.

  Specifically, the correspondence (1) is extracted from the input field meaning DB and the automation program DB. Correspondence (1) is obtained by associating both DBs with “input field tag ID = account_id”.

  Then, “login authentication” is added to the related operation of the “input field standard meaning = login ID” record in the input field standard meaning DB, and “account_id” is added to the related keyword. Further, “account_id” is added to the related keyword of the record of “operation name = login authentication” in the automation target operation DB.

  If input field label names and screen names are registered in the input field meaning DB and the automation program DB, such information can be added to the related keywords of the input field standard meaning DB and the automation target operation DB.

  A method of automatically operating a device by executing an automation program will be described with reference to FIGS.

  FIG. 26 shows an example in which information necessary for execution is prepared in advance. When executing the automation program, for example, device information held by the user and value information set in the device are required.

  FIG. 26A shows an example of information stored in the device information DB 111. Information (for example, a MAC address or an IP address) for identifying a device held by the user is stored in association with the product name of the device.

  FIG. 26B shows an example of information stored in the setting information DB 112. Information (for example, MAC address) identifying the device held by the user and the input field standard meaning are stored in association with the set value. The setting information DB 112 can store values that can be different for each device. For example, when the value is different for each product, such as an initial password, when the value is different for each user, such as an alarm destination, It can be used in any case such as when the value is different each time, such as a time password.

  These DBs may be created manually or automatically, or a separately prepared DB may be used. In addition, these DBs can be dynamically changed using values input by the user or values automatically generated by the system.

  FIG. 27 shows an example of an operation in which an automatic program is acquired and executed by the automatic operation execution unit 110 in order to automatically operate a certain device.

  First, the product name of the device to be executed is acquired (910). The product name is obtained by searching the device information DB 111 with the MAC address.

  Then, an automation program for the operation to be executed is acquired (912). The automation program searches and acquires the automation program DB 105 by product name and operation name.

  Next, the meaning (input field standard meaning) of the input field included in the acquired automation program is acquired (914). The input field standard meaning is obtained by searching the input field meaning DB 104 with the product name, screen name, and input field tag ID. As for the meaning of the input field, the information stored in the input field meaning DB is specified by the user each time when executing the automation program without collecting the information and determining the meaning, or from outside the system. The system may be configured to obtain.

  Then, the value set in the input field included in the automation program is acquired (916). The value to be set is acquired by searching the setting information DB 112 with the MAC address and the input field standard meaning. When a newly generated value is set (for example, a new password when changing a password), for example, a value is randomly generated, the value is set in an input field, and the setting information DB 112 is written. It should be noted that the setting value of the input field is not provided with the setting information DB 112 so that the user inputs it every time the automation program is executed, or the information stored in the setting information DB is obtained from outside the system. In addition, this system may be configured.

  Finally, using the acquired automation program and input field setting value, an automatic operation for the device is executed (918).

  FIG. 28 shows an example of information of the browser operation method automation program (before execution). In the setting value column, the value acquired in the automated program execution process (FIG. 27) is set. FIG. 29 shows a flow in which the device is automatically operated by the automation program of FIG.

  The login screen is displayed by the automation program (1). The automation programs (2) to (4) perform login authentication on the displayed login screen. Then, the automation program (5) clicks the link on the screen after login and opens the password change screen. Finally, the automation programs (6) to (8) complete the password change.

  In the example of FIG. 29, the “password input field of the login screen” and the “old password input field of the password change screen” originally have the same meaning although the tag ID and the label name are different. By assigning the same meaning via the meaning selection GUI, if the setting information DB stores one value corresponding to the one meaning, the automated program can be executed.

  FIG. 30 shows an example of the information of the HTTP request method automation program (before execution). Although the value acquired in the automation program execution process (FIG. 27) is set in the setting value column, the setting value of hidden_id is acquired and set from the immediately preceding HTTP response when the automation program is executed. FIG. 31 shows a flow in which the device is automatically operated by the automation program of FIG.

  The automation program (1) creates an HTTP request for displaying a login screen. The created HTTP request is transmitted to the device, and an HTTP response (1) 'is received from the device. Next, the automation program (2) creates an HTTP request for login authentication. At this time, the set value of hidden_id is acquired from the HTTP response (1) 'and set. The created HTTP request is transmitted to the device, and an HTTP response (2) 'is received from the device.

  Then, the automation program (3) creates an HTTP request for displaying the password change screen, transmits it to the device, and receives an HTTP response (3) 'from the device. Finally, the automation program (4) creates an HTTP request for changing the password, transmits it to the device, receives the HTTP response (4) 'from the device, and the password change is completed.

  To execute the automation program, there are a method (for example, FIGS. 32 to 34) executed from the automatic operation GUI and a method (for example, FIG. 35) that is automatically executed when a predetermined condition is satisfied.

  FIG. 32A shows an example of a device management screen. FIG. 32B shows an example in which the automatic operation GUI is embedded in the device management screen, and the operation name for which the automation program has been created is displayed in the select box for the target operation. When the user selects the target operation and clicks the execute button, the automated program for the selected operation can be executed.

  FIG. 33 shows an example of an operation for creating an automatic operation GUI to be embedded in an HTTP response by the GUI embedding unit 106 of the client side device 10 or the automation device 60.

  First, an operation name displayed on the automatic operation GUI is acquired (930). At this time, only the operation name “classification = common” may be acquired from the automation target operation DB 103, or the operation name “classification = unique” may be acquired as necessary.

  Then, referring to the automation program DB 105, the operation names for which the automation program has not been created are excluded from the acquired operation names (932).

  Next, the acquired operation names are rearranged in descending order of priority (934). For example, the priority level of the operation name including the related keyword in the embedded HTTP response may be determined to be high, or the priority level of the operation name having high importance may be determined to be high. Then, the frequency of the operation name selected by the user may be measured for each screen, and it may be determined that the priority of the operation name having a high selection frequency is high. Also, operation names with low priority may be excluded so as not to be displayed.

  Finally, an automatic operation GUI is created and the acquired operation name is displayed in the selection box for the target operation (936), so that the user can select the operation name and execute the automation program.

  FIG. 34 shows an example of an operation sequence of the user 45, the client 40, the apparatus 10 or 60, and the device 30 until the automation program is executed using the automatic operation GUI.

  The user 45 activates the browser of the client 40 and accesses the authentication screen of the device 30. Then, an HTTP request is transmitted from the client 40 to the apparatus 10/60 in order to obtain an authentication screen. The apparatus 10/60 transmits the HTTP request to the device 30 as it is.

  The device 30 transmits an HTTP response including the authentication screen data (FIG. 32A) to the device 10/60. When the apparatus 10/60 receives the HTTP response (FIG. 32A) from the device 30, the apparatus 10/60 embeds the automatic operation GUI created by the automatic operation GUI creation processing (FIG. 33 for details of the processing) (FIG. 32 (FIG. 32)). b)) is created and sent to the client 40.

  When the user 45 selects an operation name to be operated from the automatic operation GUI displayed on the browser of the client 40 and clicks the execution button, the apparatus 10/60 causes the automatic program execution process of the operation to be performed (details of the process are illustrated in FIG. 27), the device is automatically operated.

  FIG. 35 shows a method for periodically executing the automation program, and FIG. 35A shows an example of an operation sequence of the client 40 and the apparatus 10/60 when setting the periodic execution. ) Shows an example of an operation sequence of the device 10/60 and the device 30 when periodically executed.

  At the time of setting the periodic execution, the client 40 sets the apparatus 10/60 for the periodic execution. At this time, in order to identify the device to be executed and the operation, device identification information (for example, IP address or MAC address) The operation name and the execution date are set in association with each other. For periodic execution, the apparatus 10/60 performs automatic operation of the specified device by an automated program execution process (details of the process shown in FIG. 27) when the set execution date and time is reached.

  FIG. 36 is a diagram for explaining an example in which the initial passwords of the devices held by the user are collectively changed using the present embodiment.

  When the user has a plurality of devices, it is a heavy burden to change the password on the device management screen one by one and store the changed password. If this embodiment is used, the initial passwords of the owned devices are changed at once, and it becomes unnecessary to store the changed password. After the password change, login authentication to the management screen of each device can also be performed using the automation program of this embodiment.

  FIG. 37 shows an example of information registered in each DB of this embodiment for batch change of the initial password of the device.

  In the device information DB, information on devices owned by the user is registered. In the automation program DB, an automation program for login authentication and password change related to the product of the owned device is registered. In the input field meaning DB, the meaning of the input field related to the automation program is registered.

  The initial password DB is a DB that is newly added in order to check whether the owned device is unchanged from the initial password. In this DB, an initial account and a password relating to the product of the owned device are registered.

  In the setting information DB, the new password for the device management screen changed by the device initial password batch change process (details of the process shown in FIG. 38) is registered. The new password is used when executing the login authentication automation program.

  FIG. 38 shows details of processing for collectively changing the initial password of the management screen for the owned devices.

  First, information on all devices owned is acquired. Next, with reference to the initial password DB, it is confirmed whether login authentication can be performed with the initial password on the device management screen for each device. If the login authentication is successful, a new password is randomly generated, registered in the setting information DB, and the password is changed.

  FIG. 39 shows an example of an operation sequence of the user 45, the client 40, the apparatus 10/60, and the device 30 until the initial password on the device management screen is changed at once and then the login to the device management screen.

  The user 45 requests the device 10/60 to execute the device initial password batch change. The device 10/60 changes the initial password of each device 30 to a new password by a device initial password batch change process (details of the process are shown in FIG. 38).

  After the device initial password batch change, the user 45 may perform login authentication to the device management screen by using the automatic operation GUI (details of the process of executing the automation program using the automatic operation GUI are shown in FIG. 27).

  Of the above-described system, it is created based on information collected via the input field meaning GUI (by the GUI embedding unit 106, the input field meaning receiving unit 107, the collected data DB 202, and the input field meaning determining unit 203). The input field meaning DBs 104 and 207 can be used for the following purposes in addition to being used by the automatic operation execution unit 110 of the present system.

  For example, by using the input field meaning DBs 104 and 207 and the setting information DB 112 as necessary, the same content can be automatically input to a plurality of Web screens (each having an input field), The same contents can be automatically set on the management screen.

  The apparatus in this case is provided at a position where the communication between the device and the terminal that receives the operation from the user by displaying a screen for managing the device can be mediated, for example, from the device to the terminal. In the communication for displaying the management screen, when the operation is to input a value to the field, a user interface for giving the meaning of the field (displaying the meaning of the field on the management screen Means for embedding (which may include a configuration for the user to select), and means for collecting and recording information on the meaning of the field given via the user interface. Then, this device or another device stores a value to be input to the field in association with the meaning of the field, and communication for transmitting an operation performed via the management screen from the terminal to the device. Means for inputting the value stored in the field included in.

  Although the embodiments of the present invention have been described above, it is a matter of course that those skilled in the art can variously modify, improve, and apply the above-described embodiments, and such embodiments are also included in the scope of the present invention. .

Claims (28)

An automated program for automatically performing processing related to device management, provided at a position where communication between a device and a terminal that displays a screen for managing the device and receives an operation from a user can be mediated A device that contributes to generation,
Means for embedding a user interface for giving meaning to an operation in communication for displaying a management screen from the device to the terminal;
Collecting information about the operation and meaning of the operation performed via the user interface in accordance with communication for transmitting the operation from the user received via the management screen from the terminal to the device And recording means,
The automatic program is generated by combining a plurality of recorded operation information according to the meaning information recorded together with the operation information. The meaning is performed by associating one operation name with a plurality of operations belonging to one process.
The apparatus according to claim 1, wherein the user interface includes a configuration in which operation name candidates are displayed on the management screen for receiving the operation, and the user selects the operation name. .   In addition to receiving a plurality of operations from the user via the management screen, the system further comprises means for creating a communication for transmitting the plurality of operations to the device by executing the automation program. The apparatus according to claim 1 or 2. The meaning is performed by defining the meaning of the field when the operation is to input a value in the field.
The user interface includes a configuration in which candidates for field meaning are displayed on the management screen for receiving the operation and the user is allowed to select the meaning of the field. The apparatus of any one of these. Means for storing a value that can be different for each device in association with the meaning of the field when the operation is to input a value in the field;
Instead of receiving an operation including an input value to the field from the user via the management screen, communication for transmitting the operation to the device by executing the automation program is performed for the device. The apparatus according to any one of claims 1 to 4, further comprising: means for generating using the value stored in association with the meaning of the field. The meaning can be performed so that operations over a series of the plurality of management screens are associated as a plurality of operations belonging to one process.
6. The user interface according to claim 1, wherein the user interface displays the management screen for receiving the operation so that the user can specify the start and end of the series of screens. The apparatus according to claim 1. As the device, there are a plurality of different types of devices, and the management screen of one type of device and the management screen of another type of device can have different configurations,
7. The user interface according to claim 1, wherein the user interface is created so that the same meaning is given to an operation on a certain type of device and an operation on another type of device. The device according to item. When the operation is to input a value in a field, the meaning is performed by defining the meaning of the field,
The user interface is created so that a field in one operation and a field in another operation can be given the same meaning even if they are shown in different expressions. 8. The apparatus according to any one of items 7. There are a plurality of terminals used by different users as the terminal, meaning information recorded by one user for the same operation may differ from meaning information recorded by another user And
The generation of the automation program is performed by determining information of a plurality of operations to be combined according to the amount of recording of different meaning information for the same operation. The apparatus of any one of Claims. Means for transmitting the operation information and the meaning information to a server that generates the automation program based on the operation information and the meaning information recorded in each of the plurality of devices; Prepared,
The plurality of devices are used for different users,
The generation of the automation program is performed by determining information on a plurality of operations to be combined according to the amount of recording of different meaning information for the same operation. The apparatus of any one of Claims.   The generation of the automation program is necessary for recording by a user who made a meaning corresponding to the process including unnecessary operations as a plurality of operations belonging to one process, and as a plurality of operations belonging to one process. Recording by a user who made a meaning corresponding to the process without including an operation, and recording by a user who made a meaning corresponding to a process different from the process for a plurality of operations belonging to one process The recording is performed by excluding at least one of recordings made by a user who has given a meaning different from the meaning of the operation to one or more operations belonging to one process. The apparatus of any one of 1-10. The user interface includes a configuration for displaying information for supporting the meaning of the operation on the management screen for receiving the operation and prompting the user to give the meaning of the operation.
The apparatus according to any one of claims 1 to 11, further comprising means for storing information to be referred to in order to create the information for support.   The apparatus according to claim 12, further comprising means for updating the information to be referred to based on semantic information obtained from a plurality of users.   The apparatus according to claim 12 or 13, wherein the support information is created so that meaning information obtained from a plurality of users for the same operation is the same. Means for transmitting, to the device, communication for transmitting the operation from the user received via the management screen from the terminal to the device;
The communication system further comprises means for executing a communication for transmitting a plurality of operations belonging to the process to the device by executing the automation program and transmitting the communication to the device. The apparatus of any one of 1-14. Means for embedding a user interface for designating the processing and starting execution of the automation program in communication for management screen display from the device to the terminal;
The process according to any one of claims 1 to 15, wherein when a process is designated by a user via the management screen, execution of the automation program for automatically performing the process is started. apparatus. Means for setting conditions under which execution of the automated program should be automatically started;
The apparatus according to claim 1, wherein when the condition is satisfied, the execution of the automation program is started.   2. The apparatus according to claim 1, further comprising means for receiving the automation program from a server that generates the automation program based on the operation information and the meaning information recorded in each of the plurality of devices. The apparatus of any one of -17.   19. The operation information according to claim 1, wherein the operation information includes a content of communication for transmitting an operation from a user received via the management screen from the terminal to the device. The device described.   The apparatus according to claim 1, wherein the operation information includes a content of a screen operation performed by a user on the management screen.   21. The apparatus according to claim 1, wherein the apparatus is incorporated in the terminal.   The device according to claim 1, wherein the device is separate from the terminal and is connected to a plurality of the terminals. Each of which is a server connected to a plurality of devices provided at positions capable of mediating communication between a device and a terminal that displays a screen for managing the device and receives an operation from a user,
In the communication for displaying the management screen from the device to the terminal, a user interface for giving the meaning of the operation is embedded, and the operation from the user received through the management screen is transmitted from the terminal to the device. Means for receiving the operation information and the meaning information from the device that collects the operation information and the meaning information of the operation performed via the user interface along with communication for ,
Means for generating an automation program for automatically performing a process related to device management by combining a plurality of received operation information according to the meaning information received together with the operation information. A server characterized by   By executing the automation program, a communication for transmitting a plurality of operations belonging to the process to the device is created, and the generated automation program is distributed to the apparatus that transmits the communication to the device. 24. The server according to claim 23, further comprising means for: At least a part is installed in a computer provided at a position capable of mediating communication between a device and a terminal that displays a screen for managing the device and receives an operation from a user. A program that contributes to the generation of an automated program for automatically performing processing,
A program code for embedding a user interface for giving meaning to an operation in communication for management screen display from the device to the terminal;
Collecting information about the operation and meaning of the operation performed via the user interface in accordance with communication for transmitting the operation from the user received via the management screen from the terminal to the device Program code to be recorded,
The automation program is generated by combining a plurality of recorded operation information according to the meaning information recorded together with the operation information. At least one server computer connected to a plurality of devices provided at positions that can mediate communication between the device and a terminal that displays a screen for managing the device and receives an operation from the user. Is a program to be installed,
In the communication for displaying the management screen from the device to the terminal, a user interface for giving the meaning of the operation is embedded, and the operation from the user received through the management screen is transmitted from the terminal to the device. A program code for receiving the operation information and the meaning information from the device that collects the operation information and the meaning information of the operation performed through the user interface along with communication for When,
And a program code for generating an automation program for automatically performing processing related to device management by combining a plurality of received operation information according to the semantic information received together with the operation information. A program characterized by that. In a device provided at a position capable of mediating communication between a device and a terminal that displays a screen for managing the device and receives an operation from a user, for displaying a management screen from the device to the terminal Embed a user interface to make the meaning of the operation in
In the apparatus, accompanying the communication for transmitting the operation from the user received through the management screen to the device from the terminal, the operation information and the meaning of the operation performed through the user interface Information is collected, and the collected information is transmitted to a server connected to a plurality of the devices,
Generating an automation program for automatically performing processing related to device management by combining the plurality of received operation information according to the semantic information received together with the operation information in the server A method characterized by. A program for executing the method according to claim 27.