JP2018537734A - Factory automation system and remote server - Google Patents

Abstract

  The factory automation system 1 includes programmable logic controllers 3, 5, and 7, a microphone 15, and a remote server 10 connected via a network 9. The microphone 15 is connected to the programmable logic controller 3, and the device 13 is connected to the programmable logic controllers 3, 5, 7 and controlled by the programmable logic controllers 3, 5, 7. The programmable logic controller 3 encodes the spoken language of the user 19 recorded by the microphone 15 into an electronic voice expression. The electronic voice representation is transmitted to the remote server 10. The remote server 10 determines a device command based on the electronic voice expression and transmits the device command to the programmable logic controller 3. The programmable logic controller 3 controls the device 13 based on the received device command.

Description

  The present invention relates to a factory automation system and a programmable logic controller.

  Programmable logic controllers are commonly used in factory automation systems for controlling industrial processes such as manufacturing processes by controlling motors, sensors, energy supplies, and the like. The programmable logic controller is often housed in an environment that is difficult for a user to access due to, for example, a small space between electronic racks. Furthermore, programmable logic controllers are optimized for lowering costs and for real-time device control, which is the main purpose. Thus, programmable logic controllers typically do not include powerful processors and memories as found in today's general purpose computers. Because the computational resources of the programmable logic controller and the free space around the programmable logic controller are limited, the controller typically does not include a comfortable user interface including, for example, a large screen, keyboard, or computer mouse. Accordingly, there is a need to extend the possible ways in which a user can interact with a programmable logic controller.

  U.S. Pat. No. 7,249,023 discloses an industrial human machine interface that allows control of navigation of a display menu with voice commands analyzed by a speech processor of the human machine interface. However, such speech recognition requires intensive calculations and large amounts of vocabulary data and linguistic information that are constantly changing. This requires a powerful processor and memory that cannot be used with conventional programmable logic controllers for the reasons described above.

  Accordingly, it is an object of the present invention to provide an alternative user interface for a programmable logic controller.

  The present invention provides a factory automation system and programmable logic controller having a user interface that accepts voice commands from a user.

  According to an exemplary embodiment, a factory automation system includes a plurality of programmable logic controllers, a microphone, and a remote server connected to the plurality of programmable logic controllers via a network, wherein the microphone is a plurality of programmable logic controllers. One or more devices are connected to the first programmable logic controller and controlled by the first programmable logic controller, the first programmable logic controller being a microphone Is configured to encode the user's spoken language recorded by the computer into an electronic voice representation, the first programmable logic controller comprising: The child voice representation is configured to be transmitted over the network to a remote server, the remote server determining and determining at least one device command based on the electronic voice representation transmitted by the first programmable logic controller At least one device command transmitted to a first programmable logic controller, wherein the first programmable logic controller is one or more based on at least one device command transmitted by a remote server. Further configured to control the device.

  Such a system is a first programmable that allows a user to speak words that are recognized by the system and translated into device commands suitable for controlling a device connected to a first programmable logic controller. Provides a user interface to the logic controller. For this purpose, the first programmable logic controller is configured to encode the user's spoken language into an electronic voice representation. The hardware necessary to perform this function includes a microphone and a suitable encoder, neither of which is particularly expensive. The main task of determining device commands based on electronic speech representation, which is a computationally intensive speech recognition task, is executed on the remote server. For this purpose, voice recognition software is running on the remote server, and the remote server has the computing power and memory necessary to perform voice recognition. Since many programmable logic controllers can be connected to the remote server, these programmable logic controllers can share the voice recognition function provided by the remote server. It is therefore possible to provide a programmable logic controller that provides speech recognition at the user interface without substantially increasing the hardware requirements of the programmable logic controller. In addition, frequent updates of speech recognition software running on the remote server are immediately available to all programmable logic controllers connected to the remote server.

  According to an exemplary embodiment, the remote server includes a first conversion module that provides a first mapping from an electronic voice representation to a plurality of predefined first device commands, and a plurality of pre- A second conversion module that provides a second mapping to a defined second device command, wherein the plurality of predefined second device commands can be edited by a user, In order to determine a second device command that has the best correlation with the electronic speech representation and to determine a confidence value that represents the quality of the correlation between the electronic speech representation and the determined second device command, The second mapping is configured to apply to the electronic speech representation, and the remote server determines the determined first value if the determined confidence value satisfies the predefined requirement. To determine the first device command that has the best correlation with the electronic speech representation if the determined confidence value does not meet the predefined requirements And applying the first mapping to the electronic speech representation and transmitting the determined first device command to the first programmable logic controller.

  The second conversion module can be configured by the user, and the user can specifically define device commands that the user wants the system to recognize. Compared to the total number of different types of programmable logic controllers and controlled devices, a particular user operates a relatively small number of different programmable logic controls and devices. Thus, the number of device commands that a particular user wants to recognize is much less than the total number of device commands required by all possible users operating many different types of programmable logic controllers and devices. Since the remote server first tries to recognize the command spoken by the user using the second conversion module specific to the user, the recognition quality is high. This is because the search space in the speech recognition process can be relatively small because the number of device commands to be recognized is relatively small.

  The first conversion module is not user-specific but is shared by all possible users of the system, in which case all possible devices commands for all devices and programmable logic controllers to be used with the system Definitions can be included.

  The speech recognition process converts spoken words into device commands using the first conversion module or the second conversion module. For each conversion, the process also generates a confidence value that represents the quality of the correlation between the spoken language and the determined device command. This confidence value can also be understood as an estimated probability that the word spoken by the user will be correctly recognized as the determined device command. Typically, the confidence value is high if the user speaks loudly and clearly in a quiet environment, and the confidence value is low if the user speaks quietly or tweetes in a noisy environment.

  According to an exemplary aspect, the remote server first attempts to convert the spoken language into a recognized device command using a user-specific second conversion module, and the device command defined in the second conversion module is sufficient. If the remote server cannot identify with a high confidence value, the remote server attempts to convert the same spoken word into a device command using the first conversion module. If the device command defined in the first conversion module can be identified with a sufficiently high confidence value, the device command is transmitted by the remote server to the first programmable logic controller. If a device command having a sufficiently high confidence value cannot be determined, the remote server sends electronic data to the first programmable logic controller indicating that the spoken word conversion to the device command failed.

  According to an exemplary embodiment, one or more devices are connected to and controlled by a second programmable logic controller of the plurality of programmable logic controllers, and the at least one device command is: At least one command for controlling one or more devices connected to the second programmable logic controller.

  As a result, the user has a user interface that provides speech recognition because the user does not have a microphone or encoder for encoding spoken words into an electronic speech expression using the speech recognition interface of the first programmable logic controller. It is possible to control devices connected to other programmable logic controllers that may not have.

  According to certain embodiments, a display is connected to the first programmable logic controller, and the first programmable logic controller is configured to display a visual representation of at least one device command. The visual representation can be, for example, a menu in which multiple commands are listed in different lines or frames, and selected or recognized commands are highlighted. This provides visual feedback to the user so that the user knows which commands are available and whether the user's spoken language has been correctly recognized.

  According to an exemplary embodiment, the at least one device command includes a command for setting an operating parameter of the device to a user determined value. The operation parameters can include, for example, network module settings such as baud rate and network address, motor driver settings such as maximum frequency, and the like.

  The at least one device command may then include a command that triggers storage of the operating parameter setting on the remote server. The settings stored on the remote server can then be reused later to initialize a new programmable logic controller or device connected to the system.

  According to another embodiment, a programmable logic controller is provided, the programmable logic controller connected to a microphone, a remote server and one or more devices to be controlled by the programmable logic controller and detected by the microphone. Is encoded into an electronic voice representation, the electronic voice representation is transmitted to the remote server, and at least one device command is received from the remote server in response to the transmission of the electronic voice representation, and the received at least one device command is It is configured to control one or more devices based on it.

  These and other advantageous features of the present disclosure will become more apparent upon reading the following detailed description of exemplary embodiments in connection with the accompanying drawings. Note that not every possible embodiment necessarily represents each, all, or any of the advantages specified herein.

Schematic showing the factory automation system Flowchart illustrating a process performed by the programmable logic controller of the factory automation system shown in FIG. Flowchart showing the process executed by the remote server of the factory automation system shown in FIG. Schematic diagram of the menu displayed on the display of the factory automation system shown in FIG. Table showing the configuration data of the factory automation system shown in Figure 1 Table showing the configuration data of the factory automation system shown in Figure 1 The figure which shows the menu displayed on the display of the factory automation system shown in FIG. 1 when a user performs operation. The figure which shows the menu displayed on the display of the factory automation system shown in FIG. 1 when a user performs operation. The figure which shows the menu displayed on the display of the factory automation system shown in FIG. 1 when a user performs operation. The figure which shows the menu displayed on the display of the factory automation system shown in FIG. 1 when a user performs operation. The figure which shows the menu displayed on the display of the factory automation system shown in FIG. 1 when a user performs operation.

  In the exemplary embodiments described below, components that are similar in function and structure are designated by the same reference numerals as much as possible. Thus, to understand the characteristics of the individual components of a particular embodiment, reference should be made to other embodiments and summaries of the present disclosure.

  FIG. 1 is a schematic diagram illustrating a factory automation system 1 according to one embodiment. The factory automation system 1 includes a plurality of programmable logic controllers 3, 5, and 7 connected to a network 9 such as the Internet. The remote server 10 is also connected to the network 9 so that the programmable logic controller 3 can communicate with the remote server 10 and the other programmable logic controllers 5 and 7 via the network 9.

  The programmable logic controller 3 includes a plurality of network modules 11 such as a PROFIBUS module, and a plurality of devices 13 can be connected to each network module 11. The device 13 may include a machine, a robot, a motor, a sensor, and the like, and is controlled by a programmable logic controller to which they are connected. For this purpose, the programmable logic controller executes a program containing device commands. When the programmable logic controller 3 executes a device command, the programmable logic controller 3 transmits an electronic signal to each device 13 so that each device 13 performs a function associated with the device command.

  The device 13 is also connected to the programmable logic controller 5 and the programmable logic controller 7, and the network modules of the programmable logic controllers 5 and 7 are not shown in FIG.

  The factory automation system 1 further includes a microphone 15 and a display 17 connected to the programmable logic controller 3 to provide the user 19 with a user interface of the programmable logic controller 3. This user interface can in particular perform speech recognition and convert the spoken words of the user 19 into device commands that are executed by the programmable logic controller 3. To perform speech recognition, the programmable logic controller 3 cooperates with a remote server 10 that includes a processor and memory that are not available in the programmable logic controller 3 but are necessary to perform speech recognition tasks.

  The speech recognition task includes a process executed by the programmable logic controller 3 and a process executed by the remote server 10. FIG. 2 shows a flowchart showing a process executed by the programmable logic controller 3, and FIG. 3 shows a flowchart showing a process executed by the remote server 10.

  The microphone 15 is connected to the programmable logic controller 3 via a cable connected to the programmable logic controller 3 via a separate input. However, the microphone can also be connected to one of the network modules 11 of the programmable logic controller 3. Similarly, the display 17 is also connected to the programmable logic controller 3.

  The programmable logic controller 3 further includes an encoder 21 configured to encode the spoken words of the user 19 detected by the microphone 15 into an electronic voice representation (step 31 and step 33 of the flowchart shown in FIG. 2). . Programmable logic controller 3 then transmits the electronic voice representation over network 9 to remote server 10 (step 35 of the flowchart shown in FIG. 2).

  The remote server 10 receives the electronic voice representation transmitted by the programmable logic controller 3 (step 51 in the flowchart of FIG. 3). After receiving the electronic voice representation, the remote server 10 uses the electronic voice representation to determine one or more device commands corresponding to the user's spoken language encoded by the programmable logic controller 3 into the electronic voice representation. To analyze.

  The remote server 10 includes a first conversion module 23 and a second conversion module 25, each conversion module 23, 25 providing a mapping from an electronic voice representation to a plurality of predefined device commands. When one of the conversion modules 23, 25 is used to analyze the received electronic speech representation, each of the device commands may correspond to the user's spoken language encoded as the analyzed electronic speech representation Candidates can be determined. Here, a confidence value is determined for each device command candidate, and the confidence value represents the quality of correlation between the electronic voice representation and each device command candidate. Normally, the remote server 10 selects the device command candidate having the highest reliability value as the device command determined based on the electronic voice expression.

  As far as mentioned above, it is sufficient to have one conversion module to convert spoken language into a number of predefined device commands.

  In the illustrated example, the remote server 10 includes two conversion modules 23 and 25, and the conversion module 25 is specific to a specific user 19, but the conversion module 23 is not specific to the user 19. The remote server 10 can include a plurality of conversion modules 25, each of which is specific to one of a plurality of users registered to interact with the factory automation system. When one of these users interacts with the factory automation system, the remote server 10 uses a conversion module 25 that is specific to a particular user and a conversion module 23 that is not specific to any registered user. .

  The conversion module 25 unique to the user 19 can be edited by the user 19. The user can in particular edit a predefined command set that is recognized when the conversion module 25 is used. For example, the user can edit a predefined command by editing a text file stored on the remote server 10, where each line of the text file includes a written representation of the device command, and the device 13 or command May also include the identifier of the group of devices 13 that should be executed. The user 19 can further register a unique command used only by the user 19. For example, the user can use optical character recognition to generate a written expression from the device's operating manual. Such editing can be performed using a computer such as a laptop computer 27 connected to the remote server 10 via the network 9.

  After receiving the electronic voice representation, the remote server 10 determines at least one device command using the user specific conversion module 25 and also determines a confidence value associated with the conversion from the electronic voice representation to the at least one device command. (Step 53 in FIG. 3). The remote server 10 then determines whether the trust value meets the predefined requirement. If the confidence value is a numerical value, this determination can be made by comparing the confidence value with a predefined threshold (step 55 of FIG. 3). If the trust value satisfies the predefined requirement, the remote server 10 transmits the determined at least one device command to the programmable logic controller 3 (see step 57 in FIG. 3).

  If the confidence value does not meet the predefined requirement, the remote server 10 determines at least one device command corresponding to the electronic speech representation analyzed using the conversion module 23 (step 59 in FIG. 3). If the confidence value indicating the quality of the correlation between the electronic speech representation and the at least one device command determined using the conversion module 23 satisfies a predefined requirement (step 61 in FIG. 3), the conversion module At least one device command determined using 23 is transmitted to the programmable controller 3 (step 63 in FIG. 3).

  After receiving at least one device command corresponding to the recorded spoken language of the user 19 (step 37 in FIG. 2), the programmable logic controller 3 is connected to the received at least one device command with a corresponding electronic signal. This is executed by transmitting to one or more devices 13 (step 39 in FIG. 2).

  If the trust value does not satisfy the predefined requirement, the remote server 10 generates error information indicating that the conversion of the electronic voice representation into the device command could not be performed with sufficient accuracy, and the error information is displayed. It transmits to the programmable logic controller 3 (step 65 of FIG. 3). The programmable logic controller 3 can then inform the user of the error so that the user repeats the command, i.e. speaks again.

  The programmable logic controller 3 can provide a user interface to the user 19 using the display 17. For example, the display 17 can display a menu 31 (see FIG. 4), which shows a visual representation of a set of device commands that can be executed by the programmable logic controller 3 in a given situation. In the example shown in FIG. 4, these commands are “control / command” 32, “programming” 33, “status” 34, “configuration” 35, “other” 36, and “add” 37.

  The voice recognition system can be used to control the device 13 to perform an immediate action, such as instructing a motor to move an object, or instructing a sensor to provide a measurement. Furthermore, the speech recognition system can be used for other purposes. For example, if the user 19 utters the word “configuration” 35, the programmable logic controller 3 enters a mode in which the spoken language is recognized as a device command for configuring the programmable logic controllers 3, 5, 7 and the device 13. Can do. The device commands will then include commands for setting or changing the operating parameters of the programmable logic controllers 3, 5, 7 and device 13. For example, the operation parameters can include settings of the network module 11 of the programmable logic controller 3, and these settings include a network address, a baud rate, and the like.

  The device command is, for example, a new programmable logic controller connected to the factory automation system 1 or a programmable logic controller that sends operating parameters to the remote server 10 so that the operating parameters can be reused later to initialize a new device. A command for instructing 3 can be further included.

  Further, the device command is a command for instructing the programmable logic controller 3 to acquire a file including setting information for the device 13 already connected to the programmable logic controller 3 or the newly connected device from the remote server 10. Can further be included. The programmable logic controller 3 can apply the settings defined in such a file to the device 13 after reception from the remote server 10.

  The device command may further include a command that instructs the programmable logic controller 3 to control an actuation device such as a motor to move the object a distance. This movement can cause a collision with another object if performed at an inappropriate time, and voice commands given by the user can be misinterpreted by the factory automation system, This command is potentially harmful. The programmable logic controller can be configured to output a predefined signal to the user before executing a potentially harmful command. This signal may be, for example, a sound from a buzzer or an artificial voice generated by a speaker. The programmable logic controller then waits until the user issues a further confirmation command and executes the command only after receiving a confirmation command from the user.

  FIG. 5 shows an example of a voice command table that can be used with the programmable logic controller of the factory automation system 1. The table has five columns of “number”, “PLC number”, “auto / manual”, “voice command”, and “link”. The column “number” indicates a row number or a record number of the table. The column “PLC number” indicates the identifier of the programmable logic controller with which the table row is associated. For example, the identifier “1” represents the programmable logic controller 3, the identifier “2” represents the programmable logic controller 5, and the identifier “3” represents the programmable logic controller 7. Generally, the programmable logic controller executes a series of steps according to a sequence program stored in a storage area of the programmable logic controller. However, if one of the devices connected to the programmable logic controller has a failure, the user can execute the sequence step by step to see which steps in the sequence program are executed correctly or not. There is a need to. In the column “Auto / Manual”, the category “Auto” means that the voice command can automatically execute all the steps of the sequence. On the other hand, the category “manual” means that a voice command can only perform one step in the sequence.

  As shown in the column “Voice Command”, different programmable logic controllers have the same meaning and represent the same device command, but may be manipulated by voice commands having different voice representations. For example, the spoken words “process 1” (line number 1), “process” (line number 51), and “automatic process” (line number 101) are different sounds, but indicate the same device command. Thus, the column “link” shows a list of table row or record numbers that contain the same device command with different possible phonetic representations. The column “link” can also be used to identify voice commands that have the same voice representation but are associated with different device commands. For example, “positioning” (line number 1 and line number 102) and “positioning” (line number 52) have the same audio representation but are associated with different device commands. The above table helps to prevent device malfunction due to unintended voice input.

  FIG. 6 shows another view of the table shown in FIG. 5 when a new programmable logic controller having “PLC number” 4 is connected to the factory automation system 1. The user enters “process” as a written voice representation of the command to be used with the new programmable logic controller in column “voice command” in row 151 and the user enters the number “1” in column “link” in the same row. input. The number “51”, the number “101”, and the number “151” already connected to the number “1” are automatically added by the system and displayed in the cell of the column “link” in the row number 151, and “number 1 ”,“ Number 51 ”, and“ number 101 ”, the number“ 151 ”is added to each line related to the same device command. Such a procedure prevents the occurrence of mismatch of link attributes.

  FIG. 7 shows an example of “PLC number 1” in local control during operation. First, the user 19 says “control and command” that presents the “voice menu” 31 (FIG. 4) on the display 17 to the microphone 15 connected to the programmable logic controller 3. Then, the display 17 shows a menu 40. User 19 then selects “local”, “automatic”, and “process 1” by saying the corresponding words. When “local” is selected, “PLC number” is fixed to “1”.

  FIG. 8 shows an example of the menu 41 on the display 17 of “PLC number 1” in the global control. Similar to the example shown in connection with FIG. 7 above, selecting “Global” and “Process 1” allows the user 19 to select another remote programmable logic controller linked to the voice command “Process 1” in the table. (For example, “PLC number 2”, “PLC number 3”, “PLC number 4”) can be designated.

  FIG. 9 shows an example of the menu 42 on the display 17 when the user selects the “status” 34 entry from the “voice menu” 31 (FIG. 4) on the display 17. On the display 17, the programmable logic controller having the numbers 1 to 8 and the numbers 10 to 20 is operated by the voice command “Process 1”, and the programmable logic controller having the numbers 7 and 9 is operated by another voice command. And a menu indicating that the programmable logic controller having the number 8 is stopped due to an abnormality.

  FIG. 10 shows an example in which the user selects an entry “configuration” 35 from the “voice menu” 31 (FIG. 4) on the display 17. In this case, the display 17 shows a menu 43 indicating that the user 19 has selected local control and instructed the download to the programmable logic controller having the configuration file number 1 having the file name “*******”. Is displayed.

  Similarly, FIG. 11 shows an example in which the user selects an entry “configuration” 35 from the “voice menu” 31 (FIG. 4) on the display 17. In that case, the display 19 indicates that the user 19 has selected “global” control and instructed the download to the programmable logic controller having the configuration file numbers 1 to 4 having the file name “****”. A meaning menu 44 is displayed.

  Although the present disclosure has been described in connection with several exemplary embodiments of the present disclosure, it will be apparent to those skilled in the art that many alternatives, modifications, and variations will be apparent. Accordingly, the exemplary embodiments of the present disclosure described herein are intended to be illustrative and not limiting in any way. Various changes may be made without departing from the spirit and scope of the disclosure as defined in the claims.

US Pat. No. 7,249,023

The present invention relates to a factory automation system and a remote server .

The present invention provides a factory automation system and a remote server having a user interface that accepts voice commands from a user.

Claims (8)

Multiple programmable logic controllers;
A microphone,
A remote server connected to the plurality of programmable logic controllers via a network;
Including
The microphone is connected to a first programmable logic controller of the plurality of programmable logic controllers;
One or more devices are connected to the first programmable logic controller and controlled by the first programmable logic controller;
The first programmable logic controller is configured to encode a user's spoken language recorded by the microphone into an electronic voice representation;
The first programmable logic controller is configured to transmit the electronic audio representation to the remote server via the network;
The remote server determines at least one device command based on the electronic audio representation transmitted by the first programmable logic controller, and the determined at least one device command to the first programmable logic controller. Configured to send,
The factory automation system, wherein the first programmable logic controller is further configured to control the one or more devices based on the at least one device command transmitted by the remote server . The remote server includes a first conversion module that provides a first mapping from an electronic speech representation to a plurality of predefined first device commands; and a plurality of predefined second devices from the electronic speech representation A second conversion module that provides a second mapping to commands;
The plurality of predefined second device commands can be edited by the user;
The remote server determines the second device command having the best correlation with the electronic voice representation and the quality of the correlation between the electronic voice representation and the determined second device command Is configured to apply the second mapping to the electronic speech representation to determine a confidence value representing
The remote server sends the determined second device command to the first programmable logic controller if the determined confidence value meets a predefined requirement, and the determined confidence value is If the pre-article definition requirement is not met, the first mapping is applied to the electronic speech representation to determine the first device command having the best correlation with the electronic speech representation; The factory automation system of claim 1, wherein the factory automation system is configured to send the first device command to the first programmable logic controller.   The remote server represents the quality of the correlation between the electronic voice representation and the determined first device command if the determined confidence value satisfies the predefined requirement. Electronic data indicating that the device command could not be determined if a trust value is determined, the determined first device command is transmitted, and the trust value does not satisfy the predefined requirement The factory automation system of claim 2, wherein the factory automation system is configured to transmit to the first programmable logic controller. One or more devices are connected to a second programmable logic controller of the plurality of programmable logic controllers and controlled by the second programmable logic controller;
4. The device according to claim 1, wherein the at least one device command includes at least one command for controlling the one or more devices connected to the second programmable logic controller. 5. The factory automation system according to one item.   The display device of claim 1, further comprising a display connected to the first programmable logic controller, wherein the first programmable logic controller is configured to display a visual representation of the at least one device command. The factory automation system according to any one of items 1 to 4.   The at least one device command includes at least one command for setting an operation parameter of the at least one device to a user determined value, and the setting of the operation parameter of the at least one device to the remote server. The factory automation system according to any one of claims 1 to 5, further comprising a command for storing.   The at least one device includes a motor, the at least one device command includes a command to instruct the motor to move an object, and the first programmable logic controller outputs a predefined signal to the user. 6. A factory automation system according to any one of claims 1 to 5, wherein the factory automation system is configured to receive confirmation from the user prior to controlling the motor to perform the movement. A programmable logic controller,
The programmable logic controller is
Connected to a microphone, a remote server and one or more devices to be controlled by the programmable logic controller;
Encode the user's spoken language detected by the microphone into an electronic voice representation,
Sending the electronic voice representation to the remote server;
Receiving at least one device command from the remote server in response to the transmission of the electronic audio representation;
A programmable logic controller configured to control the one or more devices based on the received at least one device command.

Images