KR20230135199A - Apparatus and method for remotely controlling heavy construction equipment using high speed mobile communication technology - Google Patents

Abstract

It relates to a remote control device and method for heavy construction equipment to remotely control the PLC (Programmable Logic Controller) of heavy construction equipment using high-speed mobile communication technology, and is a high-speed communication connection for the processor to communicate with the control device of heavy construction equipment to be remotely controlled. Allocating an identification number to a communication modem, the processor matching a high-speed communication modem to which the identification number is assigned and a control device of the heavy construction equipment, the processor matching a PLC corresponding to the control device and the high-speed communication modem. Controlling the network unit to communicate through communication, Controlling the output unit to display information on a high-speed communication modem connected to the PLC by the processor, Remote control commands for the processor to remotely control the control device of the heavy construction equipment Generating, the processor controlling the network unit to transmit the remote control command to a PLC connected to the high-speed communication modem, the processor receiving execution result information for the remote control command through the network unit. It may include a step, and a step of the processor controlling the output unit to display the execution result information.

Description

Device and method for remote control of heavy construction equipment using high-speed mobile communication technology {APPARATUS AND METHOD FOR REMOTELY CONTROLLING HEAVY CONSTRUCTION EQUIPMENT USING HIGH SPEED MOBILE COMMUNICATION TECHNOLOGY}

The present invention relates to a remote control device for heavy construction equipment, and more specifically, to a remote control device and method for heavy construction equipment for remotely controlling a PLC (Programmable Logic Controller) of heavy construction equipment using high-speed mobile communication technology.

In general, heavy construction equipment is used for work that is very dangerous due to the nature of equipment operation or requires heavy momentum that is difficult to handle with human power, so it includes devices with various special functions.

These devices with various functions can be controlled through control devices such as control levers or control buttons disposed for each device.

However, existing heavy construction equipment has a physically expanded structure in which corresponding control devices are added whenever devices performing special functions are added, so it takes a lot of time and money to manage each control device. There was a problem.

In addition, due to the nature of heavy construction equipment, if the control device does not operate correctly or operates abnormally, a situation is bound to occur in which the safety of workers as well as workers related to heavy construction equipment is seriously threatened.

In addition, since heavy construction equipment is operated in a very dangerous and harsh environment, it is very difficult to manage it using general communication means such as WAN or LAN.

Therefore, in the future, there is a need for the development of a remote control device for heavy construction equipment that can remotely control and monitor heavy construction equipment equipped with PLC at ultra-high speed without communication limitations even in remote or extensive areas where communication is not possible through general communication means.

Republic of Korea Patent No. 10-1678027 (November 15, 2016)

The technical problem to be achieved by an embodiment of the present invention is to transmit a remote control command through a high-speed communication modem connected to a PLC corresponding to a control device of heavy construction equipment and receive execution result information for the remote control command, which is a general communication means. We aim to provide a remote control device and method for heavy construction equipment that can remotely control and monitor heavy construction equipment equipped with a PLC at ultra-high speed without communication limitations even in remote or extensive areas where communication is not possible.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below. You will be able to.

In order to solve the above technical problems, the heavy construction equipment remote control method according to an embodiment of the present invention is a construction heavy construction equipment remote control method of a heavy construction equipment remote control device including a processor that controls the network unit and the output unit, A step of assigning an identification number to a high-speed communication modem for communication connection with a control device of heavy construction equipment that the processor wishes to remotely control, and matching the high-speed communication modem to which the identification number is assigned with the control device of the heavy construction equipment by the processor. , controlling the network unit so that the processor communicates with a PLC (Programmable Logic Controller) corresponding to the control device through the high-speed communication modem, and the processor outputs the information to display information about the high-speed communication modem that is communicated with the PLC. Controlling the unit, the processor generating a remote control command for remotely controlling the control device of the heavy construction equipment, the processor controlling the network unit to transmit the remote control command to a PLC connected to the high-speed communication modem. It may include the step of receiving, by the processor, execution result information for the remote control command through the network unit, and the step of controlling the output unit so that the processor displays the execution result information.

In an alternative embodiment of the method for remotely controlling heavy construction equipment, the high-speed communication modem may be a high-speed communication modem capable of connecting to a mobile communication network including 5G or LTE.

In an alternative embodiment of the heavy construction equipment remote control method, the step of matching the high-speed communication modem and the control device of the heavy construction equipment includes selecting a control device to be remotely controlled through the high-speed communication modem among the control devices of the heavy construction equipment. The matching information can be stored by matching the selected control device with a high-speed communication modem for transmitting a remote control command to the selected control device.

In an alternative embodiment of the heavy construction equipment remote control method, the step of matching the high-speed communication modem and the control device of the heavy construction equipment may involve one-to-one matching of the high-speed communication modem and the control device of the heavy construction equipment.

In an alternative embodiment of the heavy construction equipment remote control method, the step of matching the high-speed communication modem and the control device of the heavy construction equipment may match a plurality of control devices to the single high-speed communication modem.

In an alternative embodiment of the heavy construction equipment remote control method, the step of selecting a control device to be remotely controlled through the high-speed communication modem includes calculating the importance of the control devices of the heavy construction equipment, and based on the calculated importance, the You can select the control device you want to remotely control via a high-speed communication modem.

In an alternative embodiment of the remote control method for heavy construction equipment, the step of communicating and connecting the PLC and the high-speed communication modem includes, if there is one control device matched with the high-speed communication modem, communication is connected to the PLC of the corresponding control device, If there are multiple control devices matched to the high-speed communication modem, communication can be connected to multiple PLCs of the corresponding control devices.

In an alternative embodiment of the method for remotely controlling heavy construction equipment, the step of displaying information on a high-speed communication modem connected to the PLC includes checking whether the PLC corresponding to the control device of the heavy construction equipment is connected to the high-speed communication modem, When the PLC corresponding to the control device of heavy construction equipment and the high-speed communication modem are connected, the identification information of the connected high-speed communication modem and the identification information of the control device are extracted, and the PLC of the high-speed communication modem and the control device including the extracted identification information The communication connection status between the devices can be displayed on the output unit.

In an alternative embodiment of the heavy construction equipment remote control method, the step of displaying information on a high-speed communication modem communication connected to the PLC includes, when displaying the communication connection status on the output unit, current location information of the heavy construction equipment to be remotely controlled. Acquires the communication network information, weather information, and past communication status history information of the location based on the current location information of the heavy construction equipment, and communicates based on the acquired communication network information, weather information, and past communication status history information. The connection state can be predicted, and prediction result information corresponding to the predicted communication connection state can be displayed on the output unit.

In an alternative embodiment of the remote control method for heavy construction equipment, the step of predicting the communication connection state includes inputting communication network information, weather information, and past communication state history information corresponding to the current location information into a pre-trained neural network model. The communication connection status at the current location of the heavy construction equipment can be predicted.

In an alternative embodiment of the remote control method for heavy construction equipment, the step of generating the remote control command includes, when a user input requesting remote control of the control device of the heavy construction equipment is received, a remote control corresponding to the user input or a preset setting value is received. Control commands can be created.

In an alternative embodiment of the heavy construction equipment remote control method, the step of generating the remote control command includes checking whether a set time for remotely controlling the control device of the heavy construction equipment has arrived, and remotely controlling the control device of the heavy construction equipment. When the set time for control arrives, a remote control command corresponding to the preset setting value can be generated.

In an alternative embodiment of the heavy construction equipment remote control method, the step of receiving execution result information for the remote control command may include receiving execution result information for the remote control command from an operation status information provider of the heavy construction equipment. there is.

In an alternative embodiment of the remote control method for heavy construction equipment, the execution result information for the remote control command includes a video signal of a control device operated by the remote control command and a control device operated by the remote control command. It may include a sensing signal that senses motion.

In an alternative embodiment of the remote control method for heavy construction equipment, the step of receiving execution result information for the remote control command includes transmitting the remote control command to the PLC of the control device, and executing the remote control command during a preset time. If execution result information for is not received, the execution result information request may be transmitted to the operation status information provider of the heavy construction equipment.

Meanwhile, a remote control device for heavy construction equipment according to an embodiment of the present invention is a remote control device for heavy construction equipment for providing a remote control method for heavy construction equipment, and includes a processor including one or more cores, and a memory, and the processor , assigning an identification number to a high-speed communication modem for communication connection with the control device of the heavy construction equipment to be remotely controlled, matching the high-speed communication modem assigned the identification number with the control device of the heavy construction equipment, and corresponding to the control device. Controls the network unit to communicate with a PLC (Programmable Logic Controller) through the high-speed communication modem, displays information on the high-speed communication modem connected to the PLC, and remotely controls the control device of the heavy construction equipment. Generate, transmit the remote control command to the PLC connected to the high-speed communication modem, receive execution result information for the remote control command, and display the execution result information.

The effects of the remote control device and method for heavy construction equipment according to the present invention will be described as follows.

The present invention transmits remote control commands through a high-speed communication modem connected to a PLC corresponding to the control device of heavy construction equipment and receives information on the execution results of the remote control commands, thereby enabling remote or extensive areas where communication is not possible through general communication means. You can remotely control and monitor heavy construction equipment equipped with PLC at ultra-high speed without communication limitations.

In addition, the present invention monitors heavy construction equipment equipped with a PLC in remote areas or extensive areas where Wi-Fi communication networks cannot be used, and provides control based on preliminary analysis of malfunctions, localization (confirmation of work location information), and remote control requests. To support the performance of control tasks, etc.

Further scope of applicability of the present invention will become apparent from the detailed description that follows. However, since various changes and modifications within the spirit and scope of the present invention may be clearly understood by those skilled in the art, the detailed description and specific embodiments such as preferred embodiments of the present invention should be understood as being given only as examples.

Figure 1 is a diagram for explaining a remote control device for heavy construction equipment according to the present invention.
Figure 2 is a schematic diagram showing a network function of a neural network model, according to an embodiment of the present invention.
Figure 3 is a flowchart for explaining the remote control method for heavy construction equipment according to the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

The suffixes “module” and “part” for the components used in the following description are simply given in consideration of the ease of writing this specification, and the “module” and “part” may be used interchangeably.

Furthermore, embodiments of the present invention will be described in detail below with reference to the accompanying drawings and the contents described in the accompanying drawings, but the present invention is not limited or limited by the embodiments.

The terms used in this specification are general terms that are currently widely used as much as possible while considering the function in the present invention, but this may vary depending on the intention or custom of a person skilled in the art or the emergence of new technology. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in the description of the relevant invention. Therefore, we would like to clarify that the terms used in this specification should be interpreted based on the actual meaning of the term and the overall content of this specification, not just the name of the term.

Additionally, throughout this specification, neural network, neural network, and network function may be used with the same meaning. A neural network may consist of a set of interconnected computational units, which can generally be referred to as “nodes.” These “nodes” may also be referred to as “neurons.” A neural network is composed of at least two or more nodes. The nodes (or neurons) that make up neural networks may be interconnected by one or more “links.”

Figure 1 is a diagram for explaining a remote control device for heavy construction equipment according to the present invention.

As shown in FIG. 1, the remote control device 100 of the present invention can be connected to heavy construction equipment 200 for communication.

Here, the construction equipment 200 includes a control device 220 that controls the operation of the device based on data received from the power unit 210, and a PLC (Programmable Logic Controller) 230 that controls the control device 220. ) may include.

In addition, the PLC 230 of the construction equipment 200 is connected to the remote control device 100 of the present invention and can transmit a remote control command to the control device 220.

The remote control device 100 of the present invention may include other configurations for performing a computing environment, and only some of the disclosed configurations may configure the remote control device 100.

The remote control device 100 may include a processor 110, a memory 130, and a network unit 150.

In the present invention, the processor 110 assigns an identification number to a high-speed communication modem for communication connection with the control device 220 of the heavy construction equipment 200 to be remotely controlled, and the high-speed communication modem to which the identification number is assigned Matches the control device 220 of the heavy equipment 200, connects to the PLC 230 corresponding to the control device 220 through a high-speed communication modem, and displays information about the high-speed communication modem connected to the PLC 230. , generates a remote control command to remotely control the control device 220 of the heavy construction equipment 200, transmits the remote control command to the PLC 230 connected to the high-speed communication modem, and executes the remote control command information. can be received, and execution result information can be displayed.

As an example, the high-speed communication modem may include a high-speed communication modem capable of connecting to a mobile communication network including 5G or LTE, but this is only an example and is not limited thereto.

And, when assigning an identification number to a high-speed communication modem, the processor 110 checks whether the high-speed communication modem is a modem capable of high-speed communication, and if the high-speed communication modem is a modem capable of high-speed communication, mobile communication including 5G or LTE It is checked whether an identification number that can connect to the communication network exists, and if no identification number exists, an identification number that can connect to a mobile communication network including 5G or LTE can be assigned to the high-speed communication modem.

Here, when checking whether the high-speed communication modem is a modem capable of high-speed communication, the processor 110 obtains model information corresponding to the high-speed communication modem from an external server or internal memory, and determines the high-speed communication modem based on the obtained model information. You can check whether this modem is capable of high-speed communication.

Next, when assigning an identification number to a high-speed communication modem, if the identification number does not exist, the processor 110 connects to a preset communication service provider server and requests an identification number that can connect to a mobile communication network including 5G or LTE, and The identification number assigned by the server can be given to the high-speed communication modem.

In addition, when assigning an identification number to a high-speed communication modem, the processor 110 checks whether there are multiple high-speed communication modems for which no identification number exists, and if there are multiple high-speed communication modems for which no identification number exists, the processor 110 determines whether there are multiple high-speed communication modems for which no identification number exists. By accessing the telecommunication company server, each high-speed communication modem can request an identification number that can connect to a mobile communication network including 5G or LTE, and a different identification number assigned from the telecommunication company server can be assigned to each corresponding high-speed communication modem. .

Next, when matching the high-speed communication modem and the control device 220 of the heavy construction equipment 200, the processor 110 wants to remotely control the control devices 220 of the heavy construction equipment 200 through the high-speed communication modem. The matching information can be stored by selecting the control device 220 and matching the selected control device 220 with a high-speed communication modem for transmitting a remote control command to the selected control device 220.

As an example, the matching information may include identification information of the control device 220 to be remotely controlled and identification information of a high-speed communication modem that transmits a remote control command to the control device 220 to be remotely controlled. This is only an example and is not limited thereto.

Here, when matching the high-speed communication modem and the control device 220 of the heavy construction equipment 200, the processor 110 can match the high-speed communication modem and the control device 220 of the heavy construction equipment 200 on a one-to-one basis. .

In some cases, when matching the high-speed communication modem and the control device 220 of the heavy construction equipment 200, the processor 110 may match a plurality of control devices 220 to one high-speed communication modem.

Here, the plurality of control devices 220 matched to one high-speed communication modem may include control devices 220 that control equipment that performs the same operating function among the multiple operating functions of the heavy construction equipment 200. .

In addition, the processor 110 calculates the importance of the control devices 220 of the heavy construction equipment 200 when selecting a control device 220 to be remotely controlled through a high-speed communication modem, and based on the calculated importance, You can select the control device 220 you want to remotely control through a communication modem.

Here, when calculating the importance, the processor 110 may calculate the importance based on at least one condition among the operation time, operation risk, and operation error rate of the control device 220.

For example, when the processor 110 calculates the importance, the control device 220 with a long operating time may have a higher importance than the control device 220 with a small operating time.

As another example, when calculating the importance, the processor 110 may assign a higher importance to the control device 220 with a high operational risk than the control device 220 with a low operational risk.

As another example, when the processor 110 calculates the importance, the control device 220 with a high operation error rate may have a higher importance than the control device 220 with a low operation error rate.

In this way, the present invention can efficiently and quickly remotely control a control device by selecting the control device to be remotely controlled through a high-speed communication modem based on importance.

Next, when the processor 110 communicates with the PLC 230 through a high-speed communication modem, if there is only one control device 220 matched to the high-speed communication modem, the processor 110 communicates with the PLC 230 of the corresponding control device 220. If there are multiple control devices 220 that are connected and matched to high-speed communication modems, communication can be connected to multiple PLCs 230 of the corresponding control devices 220.

Here, when the processor 110 is connected to a plurality of PLCs 230 for communication, if there are a plurality of control devices 220 matched with a high-speed communication modem, the processor 110 sequentially connects the plurality of PLCs 230 according to a preset priority. Communication can be established.

For example, when setting priorities, the processor 110 determines the operation time, operation risk, operation error rate, and control devices 220 connected to each PLC 230 of the control devices 220 of the heavy construction equipment 200. Priority can be set based on at least one condition among the number.

That is, when the processor 110 is connected to a plurality of PLCs 230, the order of communication connection to the PLC 230 of the control device 220 with a large operating time is that of the control device 220 with a small operating time. It may be faster than the order of communication connection to the PLC (230).

In some cases, when the processor 110 is connected to a plurality of PLCs 230, the order in which the processor 110 is connected to the PLC 230 of the control device 220 with a high operational risk is changed to the control device 220 with a low operational risk. ) may be faster than the order of communication connection to the PLC (230).

In another case, when the processor 110 is connected to a plurality of PLCs 230, the order in which the control device 220 with a high operation error rate is connected to the PLC 230 is the order of the control device with a low operation error rate. It may be faster than the order of communication connection to the PLC (230) of (220).

As another case, when the processor 110 is connected to a plurality of PLCs 230, the order in which the processor 110 is connected to the PLC 230 connected to a large number of control devices 220 is a small number of control devices ( It may be faster than the order of communication connection to the PLC 230 connected to 220).

In this way, the present invention can efficiently and quickly perform data communication without communication traffic by setting priorities.

Next, the processor 110 determines whether the high-speed communication modem and the PLC 230 corresponding to the control device 220 of the heavy construction equipment 200 are connected to the high-speed communication modem when displaying information about the high-speed communication modem communicated with the PLC 230. Confirm, and when the PLC 230 corresponding to the control device 220 of the construction equipment 200 and the high-speed communication modem are connected, extract the identification information of the connected high-speed communication modem and the identification information of the control device 220. , the communication connection status between the high-speed communication modem including the extracted identification information and the PLC 230 of the control device 220 can be displayed on the output unit.

Here, when the processor 110 displays the communication connection status on the output unit, the processor 110 acquires the current location information of the heavy construction equipment 200 to be remotely controlled, and determines the corresponding location based on the current location information of the heavy construction equipment 200. Obtain communication network information, weather information, and past communication status history information, predict communication connection status based on the acquired communication network information, weather information, and past communication status history information, and predict result information corresponding to the predicted communication connection status. can be displayed in the output section.

As an example, when predicting the communication connection state, the processor 110 inputs communication network information, weather information, and past communication state history information corresponding to the current location information into a pre-trained neural network model to determine the current status of the heavy construction equipment 200. The communication connection status at the location can be predicted.

Here, the pre-trained neural network model analyzes communication network information, weather information, and past communication status history information corresponding to the current location information, and analyzes the high-speed communication modem and the PLC (PLC) corresponding to the control device 220 of the heavy construction equipment 200. 230) can be learned in advance to predict the communication connection status between

For example, when predicting the communication link state, processor 110 may use a normal level ranging from 0 to 4%, a caution level ranging from 5% to 29%, a warning level ranging from 30% to 69%, and It is possible to predict a communication connection state probability including any one communication connection state level among risk levels ranging from 70% to 100%, but this is only an example and is not limited thereto.

And, when generating a remote control command, the processor 110, when receiving a user input requesting remote control of the control device 220 of the heavy construction equipment 200, performs remote control corresponding to the user input or a preset setting value. You can create commands.

In some cases, when generating a remote control command, the processor 110 checks whether the set time for remotely controlling the control device 220 of the heavy construction equipment 200 has arrived and controls the heavy construction equipment 200. When the setting time for remotely controlling the device 220 arrives, a remote control command corresponding to the preset setting value may be generated.

Next, when the processor 110 transmits a remote control command to the PLC 230 that is connected to the high-speed communication modem, it checks the high-speed communication modem that matches the control device 220 corresponding to the generated remote control command, and matches it. The communication connection status between the matched high-speed communication modem and the PLC (230) corresponding to the control device 220 is checked, and as a result of checking the communication connection status, the matched high-speed communication modem and the PLC (230) corresponding to the control device 220 are confirmed. If a communication connection is established, remote control commands can be transmitted to the PLC (230) through a high-speed communication modem.

Next, when receiving the execution result information for the remote control command, the processor 110 may receive the execution result information for the remote control command from the operation status information provider of the heavy construction equipment 200.

Here, the execution result information for the remote control command includes a video signal captured by the control device 220 operated by the remote control command and a sensing signal that senses the operation of the control device 220 operated by the remote control command. It can be included.

As an example, the operating status information providing unit of the construction equipment 200 includes a camera unit that photographs the control device 220 operated by a remote control command, and a camera unit that senses the operation of the control device 220 operated by a remote control command. It may include a sensor unit and a communication unit that transmits an image signal captured from the camera unit and a sensing signal sensed from the sensor unit.

Additionally, when receiving execution result information for a remote control command, the processor 110 may receive the execution result information for the remote control command through a high-speed communication modem that transmitted the remote control command.

Here, when receiving execution result information for a remote control command, the processor 110 checks whether the high-speed communication modem is currently transmitting the remote control command, and if the high-speed communication modem is currently transmitting the remote control command, another high-speed communication modem is currently transmitting the remote control command. Check whether there is a communication modem, and if there is another high-speed communication modem, you can receive execution result information for the remote control command through another high-speed communication modem.

At this time, when checking whether there is another high-speed communication modem, the processor 110 checks whether the high-speed communication modem has completed transmitting the remote control command if there is no other high-speed communication modem, and when the high-speed communication modem has completed transmitting the remote control command, the processor 110 checks whether the high-speed communication modem has completed transmitting the remote control command. Execution result information for control commands can be received.

Next, when receiving execution result information for the remote control command, the processor 110 transmits the remote control command to the PLC 230 of the control device 220 and executes the remote control command for a preset time. If result information is not received, a request for execution result information may be transmitted to the operation status information provider of the heavy construction equipment 200.

Here, the preset setting time may include the time from the transmission completion point when the first remote control command is completely transmitted to the generation completion point when the next second remote control command is completely generated.

Additionally, the neural network model of the present invention described above may be a deep neural network. Throughout this specification, neural network, network function, and neural network may be used interchangeably. A deep neural network (DNN) may refer to a neural network that includes a plurality of hidden layers in addition to an input layer and an output layer. Deep neural networks allow you to identify latent structures in data. In other words, it is possible to identify the potential structure of a photo, text, video, voice, or music (e.g., what object is in the photo, what the content and emotion of the text are, what the content and emotion of the voice are, etc.) . Deep neural networks include convolutional neural network (CNN), recurrent neural network (RNN), restricted Boltzmann machine (RBM), and deep belief network (DBN). , may include Q network, U network, Siamese network, etc.

A convolutional neural network is a type of deep neural network and includes a neural network including a convolutional layer. Convolutional neural networks are a type of multilayer perceptrons designed to use minimal preprocessing. A CNN may consist of one or several convolutional layers and artificial neural network layers combined with them. CNN can utilize additional weights and pooling layers. Thanks to this structure, CNN can fully utilize input data with a two-dimensional structure. Convolutional neural networks can be used to recognize objects in images. A convolutional neural network can process image data by representing it as a matrix with dimensions. For example, in the case of image data encoded in RGB (red-green-blue), each R, G, and B color can be represented as a two-dimensional (for example, in the case of a two-dimensional image) matrix. That is, the color value of each pixel of image data can be a component of the matrix, and the size of the matrix can be the same as the size of the image. Therefore, image data can be represented as three two-dimensional matrices (three-dimensional data array).

In a convolutional neural network, the convolutional process (input and output of the convolutional layer) can be performed by moving the convolutional filter and multiplying the matrix components at each position of the image with the convolutional filter. The convolutional filter may be composed of an n*n matrix. A convolutional filter may consist of a fixed type of filter that is generally smaller than the total number of pixels in the image. In other words, when m*m images are input to a convolutional layer (for example, a convolutional layer with a convolutional filter size of n*n), a matrix representing n*n pixels including each pixel of the image This can be a convolutional filter and component product (i.e., product of each component of the matrix). A component that matches the convolutional filter can be extracted from the image by multiplying it with the convolutional filter. For example, a 3*3 convolutional filter to extract upper and lower straight line components from an image would be structured as [[0,1,0], [0,1,0], [0,1,0]] You can. When a 3*3 convolutional filter for extracting upper and lower straight line components from an image is applied to the input image, upper and lower straight line components that match the convolutional filter can be extracted and output from the image. The convolutional layer can apply a convolutional filter to each matrix for each channel representing the image (i.e., R, G, and B colors in the case of an R, G, and B coded image). The convolutional layer can apply a convolutional filter to the input image and extract features that match the convolutional filter from the input image. The filter value of the convolutional filter (i.e., the value of each element of the matrix) can be updated by backpropagation during the learning process of the convolutional neural network.

A subsampling layer is connected to the output of the convolutional layer, which simplifies the output of the convolutional layer and reduces memory usage and computation. For example, when inputting the output of a convolutional layer to a pooling layer with a 2*2 max pooling filter, the image can be compressed by outputting the maximum value included in each patch for each 2*2 patch in each pixel of the image. You can. The above-described pooling may be a method of outputting the minimum value in a patch or the average value of the patch, and any pooling method may be included in the present invention.

A convolutional neural network may include one or more convolutional layers and subsampling layers. A convolutional neural network can extract features from an image by repeatedly performing a convolutional process and a subsampling process (for example, the above-described max pooling, etc.). Through an iterative convolutional process and subsampling process, the neural network can extract global features of the image.

The output of the convolutional layer or subsampling layer can be input to a fully connected layer. A fully connected layer is a layer in which all neurons in one layer and all neurons in neighboring layers are connected. A fully connected layer may refer to a structure in a neural network where all nodes of each layer are connected to all nodes of other layers.

According to one embodiment of the present invention, the processor 110 may be composed of one or more cores, such as a central processing unit (CPU) of a remote control device, and a general purpose graphics processing unit (GPGPU). unit) and a processor for data analysis and deep learning, such as a tensor processing unit (TPU). The processor 110 may read a computer program stored in the memory 130 and perform data processing for machine learning according to an embodiment of the present invention. According to one embodiment of the present invention, the processor 110 may perform calculations for learning a neural network. The processor 110 performs neural network learning, such as processing input data for learning in deep learning (DL), extracting features from input data, calculating errors, and updating the weights of the neural network using backpropagation. calculations can be performed. At least one of the CPU, GPGPU, and TPU of the processor 110 may process learning of the network function. For example, CPU and GPGPU can work together to process learning of network functions and data classification using network functions. Additionally, in one embodiment of the present invention, the processors of a plurality of remote control devices can be used together to process learning of network functions and data classification using network functions. Additionally, the computer program executed in the remote control device according to an embodiment of the present invention may be a CPU, GPGPU, or TPU executable program.

According to one embodiment of the present invention, the memory 130 can store a computer program for providing remote control results of a remote control device, and the stored computer program can be read and driven by the processor 120. The memory 130 may store any type of information generated or determined by the processor 110 and any type of information received by the network unit 150.

According to one embodiment of the present invention, the memory 130 is a flash memory type, hard disk type, multimedia card micro type, or card type memory (e.g. For example, SD or It may include at least one type of storage medium among Read-Only Memory (Read-Only Memory), magnetic memory, magnetic disk, and optical disk.

The remote control device 100 may operate in connection with web storage that performs a storage function of the memory 130 on the Internet. The description of the memory described above is only an example and is not limited thereto.

The network unit 150 according to an embodiment of the present invention may be connected to the construction equipment 200 through wireless communication through a high-speed communication modem and wired communication through the communication unit.

In addition, the network unit 150 can transmit and receive remote control result information of the control device, etc. to other remote control devices and servers. In addition, the network unit 150 enables communication between a plurality of remote control devices so that operations for remote control of the control device or model learning can be distributed and performed in each of the plurality of remote control devices. The network unit 150 can enable communication between a plurality of remote control devices and allow distributed processing of operations for remote control of a control device or model learning using a network function.

The network unit 150 according to an embodiment of the present invention may operate based on any type of wired or wireless communication technology currently used and implemented, such as short-distance, long-distance, wired, and wireless, and other networks. It can also be used in

The remote control device 100 of the present invention may further include an output unit and an input unit.

The output unit according to an embodiment of the present invention may display a user interface (UI) for providing remote control results of the control device. The output unit may output any form of information generated or determined by the processor 110 and any form of information received by the network unit 150.

In one embodiment of the present invention, the output unit includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), and an organic light-emitting diode (OLED). , a flexible display, or a 3D display. Some of these display modules may be transparent or light-transmissive so that the outside can be viewed through them. This may be referred to as a transparent display module, and representative examples of the transparent display module include TOLED (Transparent OLED).

The input unit according to an embodiment of the present invention may receive user input. The input unit may include keys and/or buttons on a user interface or physical keys and/or buttons for receiving user input. A computer program for controlling the display according to embodiments of the present invention may be executed according to user input through the input unit.

The input unit according to embodiments of the present invention may receive a signal by detecting the user's button operation or touch input, or may receive a voice or movement of the user through a camera or microphone and convert it into an input signal. For this purpose, speech recognition technology or motion recognition technology can be used.

The input unit according to embodiments of the present invention may be implemented as an external input device connected to the remote control device 100. For example, the input device may be at least one of a touch pad, a touch pen, a keyboard, or a mouse for receiving user input, but this is only an example and is not limited thereto.

The input unit according to an embodiment of the present invention can recognize a user's touch input. The input unit according to an embodiment of the present invention may have the same configuration as the output unit. The input unit may be configured as a touch screen implemented to receive a user's selection input. The touch screen may be any one of a contact capacitive type, an infrared light sensing type, a surface ultrasonic (SAW) type, a piezoelectric type, and a resistive type. The detailed description of the touch screen described above is merely an example according to an embodiment of the present invention, and various touch screen panels may be employed in the remote control device 100. The input unit configured as a touch screen may include a touch sensor. The touch sensor may be configured to convert changes in pressure applied to a specific part of the input unit or capacitance generated in a specific part of the input unit into an electrical input signal. The touch sensor may be configured to detect not only the location and area of the touch, but also the pressure at the time of touch. When there is a touch input to the touch sensor, the corresponding signal(s) are sent to the touch controller. The touch controller may process the signal(s) and then transmit corresponding data to processor 110. Accordingly, the processor 110 can recognize which area of the input unit has been touched.

In one embodiment of the present invention, the server may include other components for performing the server environment of the server. A server can include any type of device. A server may be a digital device equipped with a processor and computing power with memory, such as a laptop computer, notebook computer, desktop computer, web pad, or mobile phone.

A server (not shown) that performs an operation to provide a user terminal displaying a remote control result of a control device according to an embodiment of the present invention may include a network unit, a processor, and memory.

The server may create a user interface according to embodiments of the present invention. A server may be a computing system that provides information to a client (eg, a user terminal) through a network. The server may transmit the created user interface to the user terminal. In this case, the user terminal may be any type of remote control device 100 that can access the server. The processor of the server may transmit the user interface to the user terminal through the network unit. The server according to embodiments of the present invention may be, for example, a cloud server. The server may be a web server that processes the service. The types of servers described above are only examples and are not limited thereto.

Therefore, the present invention transmits remote control commands through a high-speed communication modem connected to a PLC corresponding to the control device of heavy construction equipment and receives execution result information for the remote control command, so as to remote areas or remote areas where communication is impossible through general communication means. Heavy construction equipment equipped with PLC can be remotely controlled and monitored at high speeds without communication limitations even in a wide area.

In addition, the present invention monitors heavy construction equipment equipped with a PLC in remote areas or extensive areas where Wi-Fi communication networks cannot be used, and provides control based on preliminary analysis of malfunctions, localization (confirmation of work location information), and remote control requests. To support the performance of control tasks, etc.

Figure 2 is a schematic diagram showing a network function of a neural network model, according to an embodiment of the present invention.

Throughout this specification, computational model, neural network, network function, and neural network may be used interchangeably. A neural network can generally consist of a set of interconnected computational units, which can be referred to as nodes. These nodes may also be referred to as neurons. A neural network consists of at least one node. Nodes (or neurons) that make up neural networks may be interconnected by one or more links.

Within a neural network, one or more nodes connected through a link may form a relative input node and output node relationship. The concepts of input node and output node are relative, and any node in an output node relationship with one node may be in an input node relationship with another node, and vice versa. As described above, input node to output node relationships can be created around links. One or more output nodes can be connected to one input node through a link, and vice versa.

In a relationship between an input node and an output node connected through one link, the value of the data of the output node may be determined based on the data input to the input node. Here, the link connecting the input node and the output node may have a weight. Weights may be variable and may be varied by the user or algorithm in order for the neural network to perform the desired function. For example, when one or more input nodes are connected to one output node by respective links, the output node is set to the values input to the input nodes connected to the output node and the links corresponding to each input node. The output node value can be determined based on the weight.

As described above, in a neural network, one or more nodes are interconnected through one or more links to form an input node and output node relationship within the neural network. The characteristics of the neural network can be determined according to the number of nodes and links within the neural network, the correlation between the nodes and links, and the value of the weight assigned to each link. For example, if the same number of nodes and links exist and two neural networks with different weight values of the links exist, the two neural networks may be recognized as different from each other.

A neural network may consist of a set of one or more nodes. A subset of nodes that make up a neural network can form a layer. Some of the nodes constituting the neural network may form one layer based on the distances from the first input node. For example, a set of nodes with a distance n from the initial input node may constitute n layers. The distance from the initial input node can be defined by the minimum number of links that must be passed to reach the node from the initial input node. However, this definition of a layer is arbitrary for explanation purposes, and the order of a layer within a neural network may be defined in a different way than described above. For example, a layer of nodes may be defined by distance from the final output node.

The initial input node may refer to one or more nodes in the neural network through which data is directly input without going through links in relationships with other nodes. Alternatively, in a neural network network, in the relationship between nodes based on links, it may mean nodes that do not have other input nodes connected by links. Similarly, the final output node may refer to one or more nodes that do not have an output node in their relationship with other nodes among the nodes in the neural network. Additionally, hidden nodes may refer to nodes constituting a neural network other than the first input node and the last output node.

The neural network according to an embodiment of the present invention is a neural network in which the number of nodes in the input layer may be the same as the number of nodes in the output layer, and the number of nodes decreases and then increases again as it progresses from the input layer to the hidden layer. You can. In addition, the neural network according to another embodiment of the present invention may be a neural network in which the number of nodes in the input layer may be less than the number of nodes in the output layer, and the number of nodes decreases as it progresses from the input layer to the hidden layer. there is. In addition, the neural network according to another embodiment of the present invention is a neural network in which the number of nodes in the input layer may be greater than the number of nodes in the output layer, and the number of nodes increases as it progresses from the input layer to the hidden layer. You can. The neural network according to another embodiment of the present invention may be a neural network that is a combination of the above-described neural networks.

A deep neural network (DNN) may refer to a neural network that includes multiple hidden layers in addition to the input layer and output layer. Deep neural networks allow you to identify latent structures in data. In other words, it is possible to identify the potential structure of a photo, text, video, voice, or music (e.g., what object is in the photo, what the content and emotion of the text are, what the content and emotion of the voice are, etc.) . Deep neural networks include convolutional neural networks (CNN), recurrent neural networks (RNN), auto encoders, generative adversarial networks (GAN), and restricted Boltzmann machines (RBM). machine), deep belief network (DBN), Q network, U network, Siamese network, Generative Adversarial Network (GAN), etc. The description of the deep neural network described above is only an example and is not limited thereto.

In one embodiment of the present invention, the network function may include an autoencoder. An autoencoder may be a type of artificial neural network to output output data similar to input data. The autoencoder may include at least one hidden layer, and an odd number of hidden layers may be placed between input and output layers. The number of nodes in each layer may be reduced from the number of nodes in the input layer to an intermediate layer called the bottleneck layer (encoding), and then expanded symmetrically and reduced from the bottleneck layer to the output layer (symmetrical to the input layer). Autoencoders can perform nonlinear dimensionality reduction. The number of input layers and output layers can be corresponded to the dimension after preprocessing of the input data. In an auto-encoder structure, the number of nodes in the hidden layer included in the encoder may have a structure that decreases as the distance from the input layer increases. If the number of nodes in the bottleneck layer (the layer with the fewest nodes located between the encoder and decoder) is too small, not enough information may be conveyed, so if it is higher than a certain number (e.g., more than half of the input layers, etc.) ) may be maintained.

A neural network may be trained in at least one of supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Learning of a neural network may be a process of applying knowledge for the neural network to perform a specific operation to the neural network.

Neural networks can be trained to minimize output errors. In neural network learning, learning data is repeatedly input into the neural network, the output of the neural network and the error of the target for the learning data are calculated, and the error of the neural network is transferred from the output layer of the neural network to the input layer in the direction of reducing the error. This is the process of updating the weight of each node in the neural network through backpropagation. In the case of teacher learning, learning data in which the correct answer is labeled in each learning data is used (i.e., labeled learning data), and in the case of non-teacher learning, the correct answer may not be labeled in each learning data. That is, for example, in the case of teacher learning regarding data classification, the learning data may be data in which each learning data is labeled with a category. Labeled training data is input to the neural network, and the error can be calculated by comparing the output (category) of the neural network with the label of the training data. As another example, in the case of non-teachable learning for data classification, the error can be calculated by comparing the input training data with the neural network output. The calculated error is backpropagated in the reverse direction (i.e., from the output layer to the input layer) in the neural network, and the connection weight of each node in each layer of the neural network can be updated according to backpropagation. The amount of change in the connection weight of each updated node may be determined according to the learning rate. The neural network's calculation of input data and backpropagation of errors can constitute a learning cycle (epoch). The learning rate may be applied differently depending on the number of repetitions of the learning cycle of the neural network. For example, in the early stages of neural network training, a high learning rate can be used to increase efficiency by allowing the neural network to quickly achieve a certain level of performance, and in the later stages of training, a low learning rate can be used to increase accuracy.

In the learning of neural networks, the training data can generally be a subset of real data (i.e., the data to be processed using the learned neural network), and thus the error for the training data is reduced, but the error for the real data is reduced. There may be an incremental learning cycle. Overfitting is a phenomenon in which errors in actual data increase due to excessive learning on training data. For example, a phenomenon in which a neural network that learned a cat by showing a yellow cat fails to recognize that it is a cat when it sees a non-yellow cat may be a type of overfitting. Overfitting can cause errors in machine learning algorithms to increase. To prevent such overfitting, various optimization methods can be used. To prevent overfitting, methods such as increasing the learning data, regularization, dropout to disable some of the network nodes during the learning process, and use of a batch normalization layer can be applied. You can.

Figure 3 is a flowchart for explaining the remote control method for heavy construction equipment according to the present invention.

As shown in Figure 3, the present invention can assign an identification number to a high-speed communication modem for communication connection with a control device of heavy construction equipment to be remotely controlled (S210).

Here, the present invention checks whether the high-speed communication modem is a modem capable of high-speed communication, and if the high-speed communication modem is a modem capable of high-speed communication, checks whether an identification number that can be connected to a mobile communication network including 5G or LTE exists, and identifies If the number does not exist, an identification number capable of accessing a mobile communication network including 5G or LTE can be assigned to the high-speed communication modem.

And, the present invention can match the high-speed communication modem to which an identification number is assigned and the control device of heavy construction equipment (S220).

Here, the present invention selects a control device to be remotely controlled through a high-speed communication modem among the control devices of heavy construction equipment, and matches the selected control device with a high-speed communication modem for transmitting a remote control command to the selected control device. Information can be saved.

In some cases, the present invention may calculate the importance of control devices of heavy construction equipment when selecting a control device, and select a control device to be remotely controlled through a high-speed communication modem based on the calculated importance.

Next, the present invention can be connected to a PLC (Programmable Logic Controller) corresponding to the control device through a high-speed communication modem (S230).

Here, in the present invention, if there is one control device matched to a high-speed communication modem, communication is connected to the PLC of the corresponding control device, and if there are multiple control devices matched to the high-speed communication modem, communication is connected to multiple PLCs of the corresponding control device. can be connected

In some cases, according to the present invention, if there are multiple control devices matched to a high-speed communication modem, multiple PLCs can be connected sequentially according to a preset priority.

Next, the present invention can display information on a high-speed communication modem connected to the PLC (S240).

Here, the present invention obtains the current location information of heavy construction equipment to be remotely controlled, obtains communication network information, weather information, and past communication status history information of the location based on the current location information of the heavy construction equipment, and obtains the obtained communication network The communication connection status can be predicted based on information, weather information, and past communication status history information, and prediction result information corresponding to the predicted communication connection status can be displayed on the output unit.

As an example, when predicting the communication connection state, the present invention inputs communication network information, weather information, and past communication status history information corresponding to the current location information into a pre-learned neural network model to connect communication at the current location of heavy construction equipment. Status can be predicted.

And, the present invention can generate a remote control command for remotely controlling the control device of heavy construction equipment (S250).

Here, the present invention can generate a remote control command corresponding to the user input or a preset setting value when a user input requesting remote control of the control device of heavy construction equipment is received.

In some cases, the present invention checks whether the set time for remotely controlling the control device of heavy construction equipment has arrived, and when the set time for remotely controlling the control device of heavy construction equipment arrives, the remote control device corresponding to the preset setting value is used. Control commands can also be created.

Next, the present invention can transmit a remote control command to a PLC connected to a high-speed communication modem (S260).

Next, the present invention can receive execution result information for the remote control command (S270).

Here, in the present invention, when receiving execution result information for a remote control command, execution result information for the remote control command can be received from an operation status information provider of heavy construction equipment.

As an example, the execution result information for the remote control command may include an image signal of a control device operated by the remote control command and a sensing signal that senses the operation of the control device operated by the remote control command.

And, the present invention can display execution result information (S280).

As such, the present invention transmits remote control commands through a high-speed communication modem that is connected to a PLC corresponding to the control device of heavy construction equipment and receives execution result information for the remote control commands, thereby enabling communication in remote areas where communication is impossible through general communication means. Alternatively, heavy construction equipment equipped with PLC can be remotely controlled and monitored at high speeds without communication limitations even in a wide area.

In addition, the present invention monitors heavy construction equipment equipped with a PLC in remote areas or extensive areas where Wi-Fi communication networks cannot be used, and provides control based on preliminary analysis of malfunctions, localization (confirmation of work location information), and remote control requests. To support the performance of control tasks, etc.

The features, structures, effects, etc. described in the present inventions above are included in at least one embodiment of the present invention, and are not necessarily limited to only one embodiment. Furthermore, the features, structures, effects, etc. illustrated in each embodiment can be combined or modified and implemented in other embodiments by a person with ordinary knowledge in the field to which the embodiments belong. Therefore, contents related to such combinations and modifications should be construed as being included in the scope of the present invention.

In addition, although the above description has been made focusing on the examples, this is only an example and does not limit the present invention, and those skilled in the art will understand the above examples without departing from the essential characteristics of the present embodiment. You will be able to see that various modifications and applications are possible. For example, each component specifically shown in the examples can be modified and implemented. And these variations and differences in application should be construed as being included in the scope of the present invention as defined in the appended claims.

100: remote control device
110: processor
130: memory
150: Network unit
200: Heavy construction equipment
210: Power unit
220: Control device
230: PLC

Claims (10)

In the construction heavy equipment remote control method of the heavy construction equipment remote control device including a processor that controls the network unit and the output unit,
Allocating, by the processor, an identification number to a high-speed communication modem for communication connection with a control device of heavy construction equipment to be remotely controlled;
Matching, by the processor, a high-speed communication modem to which the identification number is assigned and a control device of the heavy construction equipment;
Controlling the network unit so that the processor communicates with a PLC (Programmable Logic Controller) corresponding to the control device through the high-speed communication modem;
Controlling, by the processor, the output unit to display information on a high-speed communication modem connected to the PLC;
Generating, by the processor, a remote control command to remotely control a control device of the heavy construction equipment;
Controlling the network unit so that the processor transmits the remote control command to a PLC connected to the high-speed communication modem;
Receiving, by the processor, execution result information for the remote control command through the network unit; and
A heavy construction equipment remote control method comprising the step of controlling the output unit to display the execution result information by the processor. According to claim 1,
The step of matching the high-speed communication modem and the control device of the heavy construction equipment is,
Among the control devices of the heavy construction equipment, select a control device to be remotely controlled through the high-speed communication modem, match the selected control device with a high-speed communication modem for transmitting a remote control command to the selected control device, and provide matching information. A remote control method for heavy construction equipment characterized by storing. According to clause 2,
The step of selecting a control device to be remotely controlled through the high-speed communication modem is:
A remote control method for heavy construction equipment, characterized in that calculating the importance of the control devices of the heavy construction equipment and selecting a control device to be remotely controlled through the high-speed communication modem based on the calculated importance. According to claim 1,
The step of communicating and connecting the PLC and the high-speed communication modem is,
If there is one control device matched to the high-speed communication modem, communication is connected to the PLC of the corresponding control device,
A remote control method for heavy construction equipment, characterized in that if there are multiple control devices matched to the high-speed communication modem, communication is connected to multiple PLCs of the corresponding control devices. According to claim 1,
The step of displaying information on a high-speed communication modem connected to the PLC is,
Check whether the PLC corresponding to the control device of the heavy construction equipment and the high-speed communication modem are connected through communication, and if the PLC corresponding to the control device of the heavy construction equipment and the high-speed communication modem are connected through communication, the identification information of the connected high-speed communication modem is provided. A remote control method for heavy construction equipment, characterized in that extracting identification information of a control device and displaying the communication connection status between a high-speed communication modem including the extracted identification information and the PLC of the control device on the output unit. According to clause 5,
The step of displaying information on a high-speed communication modem connected to the PLC is,
When the communication connection status is displayed on the output unit, the current location information of the heavy construction equipment to be remotely controlled is obtained, and the communication network information, weather information, and past communication status history of the location are obtained based on the current location information of the heavy construction equipment. Obtaining information, predicting the communication connection status based on the acquired communication network information, weather information, and past communication status history information, and displaying prediction result information corresponding to the predicted communication connection status on the output unit. How to remotely control heavy construction equipment. According to claim 1,
The step of generating the remote control command is,
A remote control method for heavy construction equipment, characterized in that when a user input requesting remote control of the control device of the heavy construction equipment is received, a remote control command corresponding to the user input or a preset setting value is generated. According to claim 1,
The step of receiving execution result information for the remote control command is,
A remote control method for heavy construction equipment, characterized in that receiving execution result information for the remote control command from an operation status information provider of the heavy construction equipment. According to clause 8,
The execution result information for the remote control command is,
A remote control method for heavy construction equipment, comprising an image signal captured by a control device operated by the remote control command and a sensing signal that senses the operation of the control device operated by the remote control command. As a remote control device for heavy construction equipment to provide a remote control method for heavy construction equipment,
A processor including one or more cores; and
contains memory,
The processor,
Assign an identification number to a high-speed communication modem for communication connection with the control device of heavy construction equipment to be remotely controlled,
Matching the high-speed communication modem to which the identification number is assigned and the control device of the heavy construction equipment,
A communication connection is made between a PLC (Programmable Logic Controller) corresponding to the control device and the high-speed communication modem,
Displays information on a high-speed communication modem connected to the PLC,
Generating a remote control command to remotely control the control device of the heavy construction equipment,
Transmitting the remote control command to a PLC connected to the high-speed communication modem,
Receiving execution result information for the remote control command,
A remote control device for heavy construction equipment, characterized in that it displays the execution result information.

Images