KR102292276B1 - System Operating Method For Broadcast Control And System of Thereof - Google Patents

Abstract

The following embodiment relates to a method of operating a system for remote control. An operating method of a platform device for remote control according to an embodiment includes: receiving collection information of a remote control target device; storing the collected information; diagnosing the state of the remote control target equipment by analyzing the collected information through the previously learned machine learning-based analysis model; and transmitting a notification to the user terminal based on the diagnostic status of the remote control target device.

Description

A system operating method for remote control and a system for performing the same

The following embodiments relate to a method of operating a system for remote control and a system for performing the same.

In general, a system for remotely controlling equipment is applied to construction machinery and agricultural products and sold. These products are equipped with a transmitter and a receiver and are directly controlled by a person from a distance.

Representative products of construction equipment that remotely control equipment include cranes and concrete pump cars that transport heavy objects. These vehicles are remote, but workers are far away from the vehicle and cannot see the work situation, so remote control equipment is installed. In this way, the driver controls the vehicle by operating the transmitter at a remote workshop.

On the other hand, operation control technology is a remote control method using web technology such as IoT after product design and product development through 2D/3D CAD (CAD), and development and operation are separated. have.

In an embodiment, the present invention intends to provide a function capable of developing, managing, monitoring, and diagnosing unified equipment through a digital twin.

A method of operating a platform device for remote control, the method comprising: receiving collection information of a remote control target device; storing the collected information; diagnosing the state of the remote control target equipment by analyzing the collected information through a pre-learned machine learning-based analysis model; and transmitting a notification to a user terminal based on the diagnostic status of the remote control target device.

the remote control target equipment is a tractor, and transmitting first data for generating a virtual model of the tractor among the collected information to a digital twin; and transmitting, to a user terminal, second data for generating an augmented reality view targeting the tractor among the collected information according to the diagnostic status of the tractor, wherein the tractor analysis model is based on the collected information. It may be a model trained to diagnose the condition of the tractor based on it.

The digital twin may provide an environment for simulating the operation of the tractor in virtual reality by modeling the tractor.

The remote control target equipment is a tractor, and the step of receiving the collection information of the tractor may include the step of receiving the collection information converted from a Controlled Area Network (CAN) message generated from the tractor from the tractor. .

The remote control target equipment is a tractor, and the machine learning-based tractor analysis model may include: distinguishing sounds generated from the tractor and sounds generated from the clutch of the tractor among the sounds; classifying at least one failure cause sound, which is a cause of the failure of the clutch, from among the sounds generated by the clutch; and learning the failure cause and the at least one failure cause sound.

The sound generated by the tractor is an audio file, and the step of learning the failure cause and the at least one failure cause sound includes learning the failure cause and an audio file of the failure cause sound corresponding to the failure cause can do.

The remote control target equipment is a tractor, the collection information includes an audio file of a sound generated by the tractor, and analyzes the collection information through the previously learned machine learning-based tractor analysis model to determine the state of the tractor The diagnosing may include analyzing the audio file and determining whether an abnormality occurs in the clutch of the tractor.

The remote control target equipment may be a tractor, and the method may further include storing time information and simulation information related to a diagnosis state of the tractor as big data.

A method of operating a digital twin for remote control, wherein the collected information of the remote control target device and the diagnostic status of the remote control target device in which the collected information is analyzed through a pre-learned machine learning-based analysis model are received from a platform device to do; updating the model of the remote control target device based on the collected information; and providing a virtual reality simulation of the remote control target device based on the updated model.

The remote control target equipment may be a tractor, and the method may further include transmitting an alarm about the occurrence of a failure of the tractor to a user terminal based on a diagnosis state of the tractor.

the remote control target equipment is a tractor, and obtaining time information about occurrence of an abnormality in the clutch of the tractor through a virtual reality simulation of the tractor; and transmitting time information on the occurrence of an abnormality in the clutch of the tractor as big data of the platform device.

Development of unified equipment through digital twin, a digital asset of the same form as a physical asset, using technologies such as 2D/3D CAD, IoT, big data, artificial intelligence, and augmented reality through an embodiment of the present invention; It can provide functions that enable management, monitoring, and diagnosis.

1 is a view for explaining a system for controlling a tractor according to an embodiment.
2 is a flowchart for explaining an operating method of a system for controlling a tractor according to an embodiment.
3 is a diagram for explaining a method of learning a machine learning-based tractor analysis model for tractor control in one embodiment.
4 is a diagram for explaining the configuration of a platform device and a digital twin among systems for tractor control according to an embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Specific structural or functional descriptions disclosed in this specification are merely illustrative for the purpose of describing embodiments according to technical concepts, and the embodiments may be embodied in various other forms and are limited to the embodiments described herein. doesn't happen

Terms such as first or second may be used to describe various elements, but these terms should be understood only for the purpose of distinguishing one element from another element. For example, a first component may be termed a second component, and similarly, a second component may also be termed a first component.

When a component is referred to as being “connected” or “connected” to another component, it is understood that the other component may be directly connected or connected to the other component, but other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that no other element is present in the middle. Expressions describing the relationship between elements, for example, “between” and “between” or “neighboring to” and “directly adjacent to”, etc. should be interpreted similarly.

The singular expression includes the plural expression unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, and includes one or more other features or numbers, It should be understood that the possibility of the presence or addition of steps, operations, components, parts or combinations thereof is not precluded in advance.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present specification. does not

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Like reference numerals in each figure indicate like elements.

1 is a view for explaining a system for controlling a tractor according to an embodiment.

In the embodiment, the communication device 10 is connected to a tractor, which is a remote control target equipment, and terminates a CAN (Controlled Area Network) message generated from the tractor, and a protocol that enables wireless communication capable of communicating the message, for example, MQTT It may serve to convert to a protocol or the like and transmit it to the platform device 20 . The message should be described as the collection information below. To this end, the communication device may include a function for 4G or 5G communication. A message generated from the platform device 20 may be received through the corresponding communication function.

The communication device 10 may acquire GPS information of the tractor and transmit the corresponding information to the platform device 20 . In embodiments, the GPS information may be obtained through a GPS device, such as a precision GPS.

The communication device 10 may acquire the collection information of the tractor according to a predetermined cycle or a collection command, and may transmit the collection information to the platform device 20 .

The platform device 20 may acquire collection information about the tractor transmitted from the communication device 10 . In an embodiment, the platform device 20 may be configured to include an IoT platform for communication, a big data platform for storing big data, and an artificial intelligence platform for diagnosing the condition of a tractor.

In an embodiment, it is possible to store the collected information about the tractor transmitted to the big data platform in the device.

The platform device 20 mines and analyzes the stored information about the tractor, and can diagnose the state of the tractor using the analyzed information.

In an embodiment, the platform device 20 may include a pre-learned machine learning-based tractor analysis model, and diagnose the status of the tractor, for example, whether a failure occurs in the tractor by analyzing the collected information through the model. can do.

The machine learning-based tractor analysis model according to the embodiment may be built by learning the data generated in the tractor and the state of the tractor corresponding to the data. The tractor analysis model will be described in detail later with reference to FIG. 3 .

The platform device 20 may transmit a warning notification to the digital twin 30 and/or the user terminal 40 based on the analyzed diagnostic status of the tractor.

In an embodiment, the platform device 20 may receive collection information including an audio file of the sound generated by the tractor, and analyze the audio file through the tractor analysis model learned for the sound generated by the tractor It is possible to determine whether a failure occurs in the tractor, for example, whether an abnormality has occurred in the clutch of the tractor.

The platform device 20 includes information about the diagnostic status of the tractor, for example, time information related to whether or not a clutch abnormality occurs, such as the abnormal occurrence time, the elapsed time since the release of the tractor, information on the expected failure time, etc., and prediction information through simulation, etc. can be stored as big data. In addition, a warning notification including the corresponding information may be transmitted to the user terminal 40 and the digital twin 30, for example, if a clutch failure occurs despite the predicted failure time of the clutch not arriving, the relevant information By sending a notification to the user, it is possible to analyze the cause so that it can be utilized for product improvement.

The digital twin 30 is to provide a simulation environment by virtualizing a physical thing, and in an embodiment, the tractor may be modeled in 3D to provide an environment for simulating the operation of the tractor.

The digital twin 30 can model a physical tractor as a 2D/3D digital tractor, and can provide virtual reality with a real-time view based on actual operation data by interworking with the platform device 20, and simulation of the tractor's operation It is possible to provide an environment for analyzing the operation of the tractor.

The digital twin 30 according to the embodiment is linked with a Geographic Information System (GIS) to manage location management.

In an embodiment, the digital twin 30 may acquire information on the operation of the tractor through simulation using 2D/3D CAD/CAE. For example, in the digital twin 30, the clutch is worn in response to the number of shifts of the tractor, and the estimated time required until clutch failure occurs due to the wear of the clutch, or the hardened wear of the clutch lining due to the lack of lubricating oil of the tractor. Results can be obtained through simulation for the estimated time required until the clutch failure occurs, and information such as bearing wear information according to the simulation can be obtained together. Information obtained through the simulation may be transmitted to the platform device 20 and stored as big data.

The user terminal 40 may include a smartphone, a tablet, a PC, and the like, and may include a camera and specifications for executing a user app.

The user terminal 40 according to the embodiment may be used as a means for assisting in diagnosing a malfunction of a tractor using an augmented reality function provided by a provided camera and an installed app. When the tractor is photographed through the camera of the user terminal 40, an augmented reality view for assisting in diagnosing the failure of the tractor may be provided through the captured image. The user terminal 40 may be interlocked with the platform device 20 and/or the digital twin 30 to receive and execute a warning notification about the occurrence of a failure of the tractor.

2 is a flowchart for explaining an operating method of a system for controlling a tractor according to an embodiment.

In step 201 , the communication device may transmit the collected information about the tractor to the platform device.

The collection information according to the embodiment is a message in which the protocol of the CAN message generated by the tractor is converted, for example, includes an audio file for the sound of the tractor, includes precise GPS information of the tractor, or in the form of an image, video, etc. files can be included.

In step 202, the platform device may store the collected information in the big data platform.

In step 203, the platform device may update the collected information to a digital twin. In embodiments, when the collected information is updated to a digital twin, data for generating a 2D/3D model of a virtual reality simulation of a tractor may be transmitted via the digital twin, wherein the data corresponds to some or all of the collected information can do.

In step 204, the digital twin may update the 2D/3D model information for the simulation based on the collected information.

In an embodiment, based on changes in the tractor included in the collected information, changes may be applied to the simulation provided by the digital twin.

In step 205, the platform device may diagnose the condition of the tractor and request a notification.

In an embodiment, the platform device may determine whether a failure has occurred in the tractor based on the collected information, and when it is determined that a failure has occurred in the tractor, a warning notification is sent to at least one of a digital twin and a user terminal for a tractor failure can make it To this end, data corresponding to some or all of the collected information in the platform device may be input into the tractor analysis model, and the state of the tractor may be diagnosed in the tractor analysis model.

For example, if the collection information includes an audio file for the sound generated by the tractor, the platform device determines whether the audio file is a sound related to the failure of the tractor based on the previously learned and stored tractor analysis model. The failure may be diagnosed or predicted, and the platform device may transmit a warning notification to at least one of the digital twin and the user terminal in response to the failure diagnosis or prediction situation of the tractor.

In step 206, the platform device may send an alert notification to the digital twin.

In step 207, the digital twin may expose a corresponding alert notification. In an embodiment, a warning notification may be transmitted to the user terminal or a warning may be outputted in a predetermined manner to an administrator designated in advance in the corresponding system.

In step 208, the platform device may transmit a warning notification to the user terminal.

In an embodiment, the order in which alert notifications are sent in steps 206 and 208 is not limited to the drawings.

In step 209, the user terminal may run the app in response to the warning notification, and tag the tractor through the app.

In an embodiment, the app may provide augmented reality for the tractor by using a camera or the like provided in the user terminal. To this end, the tractor is tagged through an app running on the user terminal, but the tractor may be tagged through a method such as tagging a code provided to a predetermined location of the physical tractor or photographing a part of the tractor.

In step 210, the user terminal may collect tractor information from the platform device.

In the embodiment, information necessary to provide augmented reality of the tractor such as tractor information stored in the big data platform in the platform device, for example, image information corresponding to the appearance and each part of the tractor, parts information, operation information, etc. can receive The user terminal may request the tractor information from the platform device to collect the tractor information.

Alternatively, the user terminal may receive data for generating an augmented reality view for the tractor including a part of the collected information from the platform device.

In step 211, the user terminal may provide an augmented reality view.

In step 212, the digital twin can take action and support resolution through the same view for situations such as failures occurring in the tractor through interworking with the user terminal.

In embodiments, an augmented reality view may be provided for the tractor corresponding to the alert notification sent from the platform device. For example, an augmented reality view for delivering additional information to a user in conjunction with a digital twin based on a captured image of an actual tractor may be provided. Due to this, the augmented reality view can be provided in various forms, such as a function that notifies the part where a failure or abnormality has occurred in the captured image of the actual tractor, repairs or replaces the part in conjunction with the digital twin, or guides emergency treatment. .

In embodiments, the digital twin allows users and administrators to deal with issues such as malfunctions in the tractor through the same view of the augmented reality view and virtual reality view by remotely requesting access to the user app, or wired/wireless voice connection service may be provided.

For example, the digital twin identifies issues such as malfunctions in the tractor through AI using the collected information or engineering information, and sends a notification to the user app, and the user app provides an augmented reality view through the notification. can do. The user app can recognize the tractor by utilizing the model target (using CAD data) technology applied for tractor recognition, connect with the digital twin, exchange necessary information, and provide the faulty area in augmented reality.

In addition, in the control screen of the digital twin, in conjunction with augmented reality, augmented reality extended tracking technology (when a pen is drawn on a real object from a distance, image and cloud point information are utilized to provide camera view, augmented information, pen drawing information, etc.) The same CAD view and augmented reality can be viewed by utilizing the technology to maintain

In embodiments, the digital twin may use CAD assembly data to provide a virtual reality view identical to the tractor physical asset. The digital twin converts information such as ECU (engine), TCU (transmission), BCU (brake), HCU (electro-hydraulic), etc. into MQTT or other protocols through CAN messages from the tractor and collects it through a mobile network, and then collects the 3D of the tractor. It can be visualized and displayed in the information area to be displayed on the CAD view.

3 is a diagram for explaining a method of learning a machine learning-based tractor analysis model for tractor control in one embodiment.

In the embodiment, a method for learning a model for determining the presence or absence of a failure of the tractor and a cause of failure by learning the sound generated by the tractor will be described.

The learning method according to the embodiment is based on supervised learning-based machine learning 300 . In order to learn the tractor analysis model, the tractor analysis model may be learned by classifying the tractor sound 301 and the sound causing the clutch failure corresponding to the tractor sound 301 .

In an embodiment, the tractor sound 301 may be converted into an audio file such as a PCM file and transmitted to the learning model for learning the sound generated by the tractor, or may be converted in the learning model for the learning process.

In the embodiment, classifying the clutch sound among the tractor sounds 301 through machine learning 300, determining whether there is an abnormality in the clutch sound, classifying a sound that causes an abnormality in the clutch, and a clutch failure occurs with respect to the sound cause can be determined.

For example, to classify sounds that cause failure of the tractor, such as a sound that causes clutch failure due to wear 301 of the clutch release bearing, and a sound that causes clutch failure due to hard wear and tear of the clutch lining 302 can learn

According to an embodiment, when receiving various sounds of the tractor, for example, a clutch shift sound, a sound generated when the gear gums are worn, an engine sound, etc., it is determined that the sound is generated by the clutch of the electric transmission device, to determine the cause of the failure.

By using the tractor analysis model learned through the process of FIG. 3 , the platform device may determine whether the tractor is faulty by using the tractor collection information transmitted from the communication device.

4 is a diagram for explaining the configuration of a platform device and a digital twin among systems for tractor control according to an embodiment.

The tractor control system according to the embodiment may include a platform device 400 and a digital twin 500 .

In an embodiment, the platform device 400 is configured to include a memory 410 and a processor 420 , and the method of operating the platform device for tractor control stored in the memory 410 may be executed through the processor 520 . .

The digital twin 500 includes a memory 510 and a processor 520 , and the operation method of the digital twin for tractor control stored in the memory 510 may be executed through the processor 520 .

The platform device 400 may mine and analyze the stored information on the tractor, and diagnose the state of the tractor using the analyzed information.

In an embodiment, the platform device 400 may include a pre-learned machine learning-based tractor analysis model, and diagnose the state of the tractor, for example, whether a failure occurs in the tractor by analyzing the collected information through the model. can do. The tractor analysis model according to the embodiment is a machine learning model previously learned according to the description of FIG. 3 .

The machine learning-based tractor analysis model according to the embodiment may be built by learning the data generated in the tractor and the state of the tractor corresponding to the data. For this, the audio file of the sound generated by the tractor may be learned.

For example, if the collection information includes an audio file for the sound generated by the tractor, the platform device 400 determines whether the audio file is a sound related to the failure of the tractor based on the previously-learned and stored tractor analysis model By doing so, it is possible to diagnose or predict the failure of the tractor.

The platform device 400 provides a diagnostic status for the tractor, for example, time information related to whether or not a clutch abnormality has occurred, such as an abnormal occurrence time, an elapsed time since the release of the tractor, information on an expected failure time, etc., and prediction information through simulation, etc. can be stored as big data.

In an embodiment, the platform device 400 may transmit a warning notification to the digital twin 500 and/or the user terminal based on the analyzed diagnostic status of the tractor.

In response to the tractor failure diagnosis or prediction situation, the platform device 400 may transmit a warning notification to at least one of the digital twin 500 and the user terminal.

For example, the warning notification may include information such as voice and text for warning notification.

Alternatively, when a clutch failure occurs even though the predicted clutch failure time has not arrived, a notification may be transmitted to allow the user to analyze the cause and utilize it for product improvement.

The digital twin 500 is to provide a simulation environment by virtualizing a physical thing. In an embodiment, the tractor may be modeled in 3D to provide an environment for simulating the operation of the tractor.

The digital twin 500 can model a physical tractor as a 2D/3D digital tractor, and can provide augmented reality with a real-time view based on actual operation data in conjunction with the platform device 400, and simulate the operation of the tractor It is possible to provide an environment for analyzing the operation of the tractor.

The digital twin 500 according to the embodiment is linked with a Geographic Information System (GIS) to manage location management.

In an embodiment, the digital twin 500 may acquire information on the operation of the tractor through simulation using 2D/3D CAD/CAE. For example, in the digital twin 30, the clutch is worn in response to the number of shifts of the tractor, and the estimated time required until clutch failure occurs due to the wear of the clutch, or the hardened wear of the clutch lining due to the lack of lubricating oil of the tractor. Results can be obtained through simulation for the estimated time required until the clutch failure occurs, and information such as bearing wear information according to the simulation can be obtained together. Information obtained through the simulation may be transmitted to the platform device 400 and stored as big data.

The embodiments described above may be implemented by a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the apparatus, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA) array), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and carry out program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with reference to the limited drawings, those skilled in the art may apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Claims (23)

In the operating method of a platform device for remote control,
transmitting first data to the digital twin for obtaining collected information about the remote control target equipment, wherein the collected information includes information for generating a virtual model;
receiving collection information on the remote control target device from the remote control target device and the digital twin;
storing the collected information;
diagnosing the state of the remote control target equipment by analyzing the collected information through a pre-learned machine learning-based analysis model; and
Transmitting a notification to a user terminal based on the diagnostic status of the remote control target device
including,
In the digital twin,
An environment is provided for simulating the operation of the remote control target device in virtual reality by modeling the remote control target device,
Prediction information about the occurrence of a failure of the remote control target equipment can be obtained through simulation of the occurrence of a failure of the parts of the remote control target equipment,
The step of diagnosing the state of the remote control target equipment comprises:
diagnosing a state of failure of a component of the remote control target equipment based on the collection information; and
Analyzing a cause of a failure using predictive information on the occurrence of a failure of a part of the remote control target equipment transmitted from the digital twin based on the simulation and a diagnosed state for the failure
containing,
How the platform device works.
According to claim 1,
The remote control target equipment is a tractor,
The step of transmitting a notification to the user terminal based on the diagnostic status of the remote control target device comprises:
Transmitting, in the platform device, second data for generating an augmented reality view targeting the tractor among the collected information to a user terminal according to the diagnostic status of the tractor;
includes,
The analysis model is a model trained to diagnose the condition of the tractor based on the collected information,
How the platform device works.
delete According to claim 1,
The remote control target equipment is a tractor,
Receiving the collection information of the tractor,
Receiving the collection information converted from the CAN (Controlled Area Network) message generated from the tractor from the tractor
containing,
How the platform device works.
According to claim 1,
The remote control target equipment is a tractor,
The machine learning-based tractor analysis model is,
distinguishing sounds generated by the tractor and sounds generated by the clutch of the tractor from among the sounds;
classifying at least one failure cause sound, which is a cause of the failure of the clutch, from among the sounds generated by the clutch; and
learning the failure cause and the at least one failure cause sound
learned through
How the platform device works.
6. The method of claim 5,
The sound generated by the tractor is an audio file,
The step of learning the failure cause and the at least one failure cause sound comprises:
learning the cause of the failure and the audio file of a sound of a failure cause corresponding to the cause of the failure
containing,
How the platform device works.
According to claim 1,
The remote control target equipment is a tractor,
The collection information includes an audio file of the sound generated by the tractor,
The step of diagnosing the state of the tractor by analyzing the collected information through the previously learned machine learning-based tractor analysis model,
Analyzing the audio file to determine whether an abnormality occurs in the clutch of the tractor
containing,
How the platform device works.
According to claim 1,
The remote control target equipment is a tractor,
Storing time information and simulation information related to the diagnostic state of the tractor as big data
further comprising,
How the platform device works.
In the operating method of a digital twin for remote control,
receiving, from a platform device, a diagnostic status of the remote control target device, in which the collected information is analyzed, through the collected information of the remote control target device and a previously-learned machine learning-based analysis model;
updating the model of the remote control target device based on the collected information;
providing a virtual reality simulation of the remote control target device based on the updated model;
acquiring predictive information on the occurrence of a failure of the remote control target device through a virtual reality simulation on the occurrence of a failure of a component of the remote control target device; and
transmitting the prediction information to the platform device
including,
through the platform device,
In response to the occurrence of a failure of a component of the remote control target equipment, the cause of the failure is analyzed using the state diagnosed with the failure and the prediction information,
How digital twins work.
10. The method of claim 9,
The remote control target equipment is a tractor,
The step of transmitting a notification to the user terminal based on the diagnostic status of the remote control target device comprises:
Transmitting an alarm about the occurrence of a failure of the tractor to a user terminal based on the diagnostic status of the tractor
containing,
How digital twins work.
10. The method of claim 9,
The remote control target equipment is a tractor,
acquiring time information about occurrence of abnormality in the clutch of the tractor through virtual reality simulation of the tractor; and
Transmitting time information about the occurrence of an abnormality in the clutch of the tractor as big data of the platform device
further comprising,
How digital twins work.
12. A computer program stored in a medium in combination with hardware to execute the method of any one of claims 1, 2 and 4 to 11.
A platform device for remote control, comprising:
one or more processors;
Memory; and
one or more programs stored in the memory and configured to be executed by the one or more processors;
The program is
transmitting first data to the digital twin for obtaining collected information about the remote control target equipment, wherein the collected information includes information for generating a virtual model;
receiving collection information on the remote control target device from the remote control target device and the digital twin;
storing the collected information;
diagnosing the state of the remote control target equipment by analyzing the collected information through a pre-learned machine learning-based analysis model; and
Transmitting a notification to a user terminal based on the diagnostic status of the remote control target device
run,
In the digital twin,
An environment is provided for simulating the operation of the remote control target device in virtual reality by modeling the remote control target device,
Prediction information about the occurrence of a failure of the remote control target equipment can be obtained through simulation of the occurrence of a failure of the parts of the remote control target equipment,
The step of diagnosing the state of the remote control target equipment comprises:
diagnosing a state of failure of a component of the remote control target equipment based on the collection information; and
Analyzing a cause of a failure using predictive information on the occurrence of a failure of a part of the remote control target equipment transmitted from the digital twin based on the simulation and a diagnosed state for the failure
containing,
platform device.
14. The method of claim 13,
The remote control target equipment is a tractor,
The step of transmitting a notification to the user terminal based on the diagnostic status of the remote control target device comprises:
Transmitting, in the platform device, second data for generating an augmented reality view targeting the tractor among the collected information to a user terminal according to the diagnostic status of the tractor;
run more,
The analysis model is a model trained to diagnose the condition of the tractor based on the collected information,
platform device.
delete 14. The method of claim 13,
The remote control target equipment is a tractor,
In the step of receiving the collection information of the tractor,
Receiving the collection information converted from the CAN (Controlled Area Network) message generated from the tractor from the tractor
to run,
platform device.
14. The method of claim 13,
The remote control target equipment is a tractor,
The machine learning-based tractor analysis model is,
distinguishing sounds generated by the tractor and sounds generated by the clutch of the tractor from among the sounds;
classifying at least one failure cause sound, which is a cause of the failure of the clutch, from among the sounds generated by the clutch; and
learning the failure cause and the at least one failure cause sound
learned through
platform device.
18. The method of claim 17,
The sound generated by the tractor is an audio file,
The step of learning the failure cause and the at least one failure cause sound comprises:
learning the cause of the failure and the audio file of a sound of a failure cause corresponding to the cause of the failure
containing,
platform device.
14. The method of claim 13,
The remote control target equipment is a tractor,
The collection information includes an audio file of the sound generated by the tractor,
In the step of diagnosing the state of the tractor by analyzing the collected information through the previously learned machine learning-based tractor analysis model,
Analyzing the audio file to determine whether an abnormality occurs in the clutch of the tractor
to run,
platform device.
14. The method of claim 13,
The remote control target equipment is a tractor,
Storing time information and simulation information related to the diagnostic state of the tractor as big data
to run more,
platform device.
In the digital twin for remote control,
one or more processors;
Memory; and
one or more programs stored in the memory and configured to be executed by the one or more processors;
The program is
receiving, from a platform device, a diagnostic status of the remote control target device, in which the collected information is analyzed, through the collected information of the remote control target device and a previously-learned machine learning-based analysis model;
updating the model of the remote control target device based on the collected information; and
providing a virtual reality simulation of the remote control target device based on the updated model;
acquiring predictive information on the occurrence of a failure of the remote control target device through a virtual reality simulation on the occurrence of a failure of a component of the remote control target device; and
transmitting the prediction information to the platform device
run,
through the platform device,
In response to the occurrence of a failure of a component of the remote control target equipment, the cause of the failure is analyzed using the state diagnosed with the failure and the prediction information,
digital twin.
22. The method of claim 21,
The remote control target equipment is a tractor,
The step of transmitting a notification to the user terminal based on the diagnostic status of the remote control target device comprises:
Transmitting an alarm about the occurrence of a failure of the tractor to a user terminal based on the diagnostic status of the tractor
to run,
digital twin.
22. The method of claim 21,
The remote control target equipment is a tractor,
acquiring time information about occurrence of abnormality in the clutch of the tractor through virtual reality simulation of the tractor; and
Transmitting time information about the occurrence of an abnormality in the clutch of the tractor as big data of the platform device
to run more,
digital twin.

Images