KR20230042158A - A system that assists in autonomous driving of mechanical tractors via human machine interface - Google Patents

Abstract

The present invention relates to a system for assisting autonomous driving of a mechanical tractor through a human machine interface (HMI) implemented using a computing device including one or more processors and one or more memories storing instructions executable by the processor. The system comprises: a power information receiving unit which receives engine information and hitch information of the mechanical tractor through an HMI display, when the mechanical tractor starts work while the HMI display is connected to a connection terminal positioned in one area of the mechanical tractor; a farmland information receiving unit which receives farmland information of a farmland in which the mechanical tractor is located through the HMI display, when the mechanical tractor starts the work, while the mechanical tractor and the HMI display are connected; a work data generating unit which generates work pattern data of the mechanical tractor by analyzing the engine information, the hitch information, and the farmland information, when reception of the engine information, the hitch information, and the farmland information is completed, based on a pre-stored algorithm, and transmits the work pattern data to the HMI display; and a work progress unit which allows the mechanical tractor to drive autonomously by the HMI display, by requesting the HMI display to control the mechanical tractor through the received work pattern data, when the HMI display receives the work pattern data. Various other embodiments identified through the present document are possible. According to the present invention, the problems of labor shortage and agricultural labor costs can be solved.

Description

A system that assists autonomous driving of mechanical tractors through human machine interfaces

The present invention relates to a system for assisting autonomous driving of a mechanical tractor through a human machine interface, and more specifically, when a human machine interface (HMI) display is connected to a mechanical tractor, engine information of the mechanical tractor from the HMI display, hitch ) receive information and farmland information, analyze the received engine information, hitch information, and farmland information based on pre-stored algorithms to create work pattern data of mechanical tractors, and generate work pattern data through the HMI display. Based on this, it relates to a technology that allows a mechanical tractor to autonomously drive to harvest crops in farmland.

Tractor is a word that refers to a special vehicle that pulls heavy loads or agricultural machinery. Tractors are used in various industrial fields, and are mainly used in agriculture. However, the sales rate of tractors is rapidly declining due to the decrease in manpower due to the aging of rural communities and the decrease in yield due to climate change. Accordingly, in order to improve the sales rate of tractors, the agricultural machinery industry is releasing a large number of economical tractors with improved cost-effectiveness. However, in the case of an economic tractor, a function providing convenience to the user is excluded, causing inconvenience to the operator. Accordingly, the agricultural machinery industry is developing various technologies that combine "high-tech" technology that guarantees user convenience to tractors.

As an example, Korean Patent Registration No. 10-1339750 (autonomous tractor) discloses a technology for generating an autonomous driving path for a tractor and checking the location of the tractor in real time so that the tractor drives along the autonomous driving path.

However, in the above-described prior art, only a technology for driving a tractor according to a previously created autonomous driving path is disclosed, and based on a pre-stored algorithm, machine learning is performed on various information received from the tractor to determine the farmland where the tractor is located. A technology for autonomously driving a tractor in consideration of factors such as the environment of the field, for example, the slope of the ground and the climate change of farmland, has not been disclosed, and the need for a technology capable of solving this problem is emerging.

Accordingly, in the present invention, when a human machine interface (HMI) display is connected to a mechanical tractor, engine information, hitch information, and farmland information of the mechanical tractor are received from the HMI display, and the received engine information and hitch information are received. and farmland information is analyzed based on pre-stored algorithms to generate work pattern data of the mechanical tractor, and based on the work pattern data generated by the HMI display, the mechanical tractor is controlled to autonomously drive in consideration of the farmland environment. Its purpose is to provide work convenience to workers by autonomously performing work suitable for the farmland environment.

In a system for assisting autonomous driving of a mechanical tractor through a human machine interface implemented as a computing device including one or more processors and one or more memories storing instructions executable by the processors according to an embodiment of the present invention, HMI (human machine interface) Reception for receiving engine information and hitch information of the mechanical tractor through the HMI display when the mechanical tractor starts working in a state in which a display is connected to a connection terminal located in one area of the mechanical tractor a power information receiving unit; a farmland information receiver configured to receive farmland information of a farmland where the mechanical tractor is located through the HMI display when the mechanical tractor starts working while the mechanical tractor and the HMI display are connected; When the reception of the engine information, the hitch information, and the farmland information is completed, based on a pre-stored algorithm, the engine information, the hitch information, and the farmland information are analyzed to generate work pattern data of the mechanical tractor a work data generating unit transmitting to the HMI display; and when the HMI display receives the work pattern data, requesting the HMI display to control the mechanical tractor through the received work pattern data so that the mechanical tractor autonomously drives by the HMI display. It is characterized in that it includes;

The engine information is information corresponding to information output to the instrument panel of the mechanical tractor as the mechanical tractor starts driving, and includes information on engine rotation speed, gear condition information, transmission oil temperature information, and coolant temperature of the mechanical tractor. information and at least one of battery voltage information, wherein the hitch information is information measured by a hitch sensor of the mechanical tractor as the mechanical tractor starts driving, and applied to the hitch of the mechanical tractor It is preferable to include at least one of pressure information about a losing pressure, flow rate information about the flow rate of the hitch, hitch location information about the location of the hitch, and hitch horizontality information about the horizontality of the hitch.

The system performs CAN (controller area network) communication between the mechanical tractor and the HMI display, and performs cellular communication between the HMI display and a management server that manages the HMI display and the mechanical tractor. When receiving of the engine information, the hitch information, and the farmland information is completed, the engine information, the hitch information, and the farmland information may be stored in a database of the management server.

The farmland information receiving unit may include a location confirmation unit for confirming a location of the farmland where the mechanical tractor is located based on the received farmland information when the farmland information is received from the HMI display; And when the location confirmation of the farmland where the mechanical tractor is located is completed by the location confirmation unit, receiving climate information corresponding to the confirmed location from the Meteorological Administration server to check the precipitation value, air temperature value, and air volume value of the farmland A climate checking unit; however, the farmland information preferably includes location information of the farmland where the mechanical tractor is located and crop information about crops being cultivated in the farmland.

The work data generating unit, when the reception of the engine information is completed, based on the received engine information, the engine speed information, the gear condition information, the transmission oil temperature information, a driving state determiner configured to determine a driving state of the mechanical tractor by analyzing the coolant temperature information and the battery voltage information; When the reception of the hitch information is completed, based on the received hitch information, the pressure information, the flow rate information, the hitch position information, and the hitch horizontality information at a first time in the specified time range are analyzed, a working state determining unit for determining a working state of the mechanical tractor; When the reception of the farmland information is completed, a climate state determination unit for determining a climate state during operation of the mechanical tractor by analyzing the farmland location information, the climate information, and the crop information at a first time in the designated time range. ; and obtaining first operation information for the mechanical tractor at the first time based on the pre-stored algorithm when the determination of the driving state, the working state, and the weather state at the first time is completed. It is possible to include; operation information acquisition unit.

The operation information acquisition unit may, when the acquisition of operation information for each of a plurality of times included in the designated time range is completed due to the performance of the driving state determination unit, the work state determination unit, and the climate state determination unit, Based on operation information for each of a plurality of times, it is possible to generate work pattern data for operation of the mechanical tractor.

The pre-stored algorithm is an algorithm for generating the work pattern data, and machine learning is performed on the generated work pattern data to obtain other work pattern data for different driving conditions, different working conditions, and different climate conditions. It is possible to update to generate.

When the HMI display receives the work pattern data from the work progress unit, based on the received work pattern data, the HMI display controls the mechanical tractor connected through the connection terminal to autonomously drive to harvest crops in the farmland. It is possible to transmit the engine information, the hitch information, and the farmland information obtained as the mechanical tractor is driven by controlling the mechanical tractor by the driving start unit, to the management server in real time. .

A system for assisting autonomous driving of a mechanical tractor through a human machine interface according to the present invention learns the tractor operation method of a passenger riding a mechanical tractor through machine learning to generate work pattern data, and HMI display generated work pattern data Based on this, the mechanical tractor can be autonomously driven unmanned by controlling the mechanical tractor.

Accordingly, the system for assisting autonomous driving of a mechanical tractor through the human machine interface of the present invention can solve the problem of labor shortage and labor cost due to the aging of rural areas by allowing the mechanical tractor to perform surrogate farming.

In addition, as a pre-stored algorithm for generating work pattern data is automatically updated, new work pattern data of a mechanical tractor suitable for various farmland environments and climatic environments may be generated.

1 is a conceptual diagram of a system for assisting autonomous driving of a mechanical tractor through a human machine interface according to an embodiment of the present invention.
2 is a diagram for explaining a power information receiving unit of a system for assisting autonomous driving of a mechanical tractor through a human machine interface according to an embodiment of the present invention.
3 is a diagram for explaining a farmland information receiving unit of a system for assisting autonomous driving of a mechanical tractor through a human machine interface according to an embodiment of the present invention.
4 is a view for explaining a work progressing part of a system for assisting autonomous driving of a mechanical tractor through a human machine interface according to an embodiment of the present invention.
5 is a view for explaining an operation information acquisition unit of a system for assisting autonomous driving of a mechanical tractor through a human machine interface according to an embodiment of the present invention.
6 is another diagram for explaining a work progressing part of a system for assisting autonomous driving of a mechanical tractor through a human machine interface according to an embodiment of the present invention.
7 is a diagram for explaining an example of an internal configuration of a computing device according to an embodiment of the present invention.

In the following, various embodiments and/or aspects are disclosed with reference now to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth in order to facilitate a general understanding of one or more aspects. However, it will also be appreciated by those skilled in the art that such aspect(s) may be practiced without these specific details. The following description and accompanying drawings describe in detail certain illustrative aspects of one or more aspects. However, these aspects are exemplary and some of the various methods in principle of the various aspects may be used, and the described descriptions are intended to include all such aspects and their equivalents.

References to “embodiment,” “example,” “aspect,” “example,” etc., used in this specification should not be construed as indicating that any aspect or design described is preferable to or advantageous over other aspects or designs. .

Also, the terms "comprises" and/or "comprising" mean that the feature and/or element is present, but excludes the presence or addition of one or more other features, elements and/or groups thereof. It should be understood that it does not.

In addition, terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

In addition, in the embodiments of the present invention, unless otherwise defined, all terms used herein, including technical or scientific terms, are those commonly understood by those of ordinary skill in the art to which the present invention belongs. have the same meaning. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the embodiments of the present invention, an ideal or excessively formal meaning not be interpreted as

1 is a conceptual diagram of a system for assisting autonomous driving of a mechanical tractor through a human machine interface according to an embodiment of the present invention.

Referring to FIG. 1, a system 100 for assisting autonomous driving of a mechanical tractor through a human machine interface implemented as a computing device including one or more processors and one or more memories storing instructions executable by the processors (hereinafter, referred to as an auxiliary system) may include a management server 101 , an HMI display 103 and a mechanical tractor 105 .

According to an embodiment, the management server 101 is a component that manages the HMI display 103, and controls the HMI display 103 to allow the HMI display to control the mechanical tractor 105. there is.

According to one embodiment, the HMI display 103 may be a display 103 providing a human machine interface function. The human machine interface may be an interface that allows a user to communicate with industrial and manufacturing processes and machines, computer programs or systems. That is, the HMI display may be a touch screen used by a user to operate a machine in an automation field.

According to one embodiment, the mechanical tractor 105 is a vehicle that performs various tasks by using traction force, and may be a vehicle that performs various tasks by attaching and detaching various work machines. That is, the auxiliary system 100 of the present invention controls the management server 101 to autonomously drive the mechanical tractor 105 in a state in which the HMI display 103 is connected to the mechanical tractor 105 could be an invention.

According to one embodiment, the management server 101 may include a power information receiving unit 101a, a farmland information receiving unit 101b, a work data generating unit 101c and a work progressing unit 101d.

According to an embodiment, the power information receiving unit 101a is configured to operate when the mechanical tractor 105 starts working in a state in which the HMI display 103 is connected to a connection terminal located in one area of the mechanical tractor 105. In this case, engine information and hitch information of the mechanical tractor 105 may be received through the HMI display 103 . The meaning that the mechanical tractor 105 starts working may mean that the mechanical tractor 105 simultaneously performs driving and harvesting of crops.

According to one embodiment, the HMI display 103 can perform CAN (controller area network) communication when connected to the mechanical tractor 105 through the connection terminal. In addition, the management server 101 may perform cellular communication with the HMI display 103 .

According to one embodiment, when the HMI display 103 is connected to the mechanical tractor 105 through a connection terminal of the mechanical tractor 105, an electronic control unit (ECU) installed in the mechanical tractor 105 and can be interlocked. The HMI display 103 may receive the engine information from the ECU. The ECU receives input values from various sensors (eg, temperature sensor, oil sensor, engine sensor, hitch sensor, etc.) installed in the mechanical tractor 105, and outputs the received input values to the instrument panel of the mechanical tractor 105. can do. That is, when the HMI display 103 is connected to the mechanical tractor 105, control devices (eg, engine control device, ECU, hitch control device, etc.) of the mechanical tractor 105 are displayed through a human machine interface. You can control it.

According to one embodiment, the engine information is information corresponding to information output on the dashboard of the mechanical tractor 105 as the mechanical tractor 105 starts driving, and the engine rotation of the engine located in the mechanical tractor 105 The information may include at least one of number information, gear state information, transmission oil temperature information, coolant temperature information, and battery voltage information.

That is, the power information receiver 101a performs cellular communication with the HMI display 103 when the HMI display 103 completes receiving the engine information from the mechanical tractor 105 through CAN communication. , the engine information can be received from the HMI display 103.

According to one embodiment, the hitch information is information measured by a hitch sensor of the mechanical tractor 105 as the mechanical tractor 105 starts driving, and the hitch of the mechanical tractor 105 It may include at least one of pressure information about pressure applied to the hitch, flow rate information about the flow rate of the hitch, location information about the location of the hitch, and hitch horizontality information about the horizontality of the hitch. The hitch may have a configuration in which a working machine mounted on the mechanical tractor 105 is substantially coupled.

According to one embodiment, the ECU of the mechanical tractor 105 may receive an input value from the hitch sensor. Accordingly, the ECU of the mechanical tractor 105 may output various values related to the hitch to the instrument panel. That is, various values related to the hitch output to the instrument panel may be included in the hitch information.

According to an embodiment, the power information receiving unit 101a may receive the hitch information in the same manner as receiving the engine information from the HMI display 103 .

According to one embodiment, the farmland information receiving unit 101b is connected to the mechanical tractor 105 and the HMI display 103, when the mechanical tractor 105 starts working, the HMI display 103 ) It is possible to receive farmland information of the farmland where the mechanical tractor 105 is located.

According to an embodiment, the farmland information receiver 101b receives the farmland information from the HMI display 103 in the same manner as the power information receiver 101a receives the engine information and the hitch information. can do.

According to one embodiment, the farmland information receiver 101b may include a location checker (not shown) and a climate checker (not shown).

According to an embodiment, when the farmland information is received from the HMI display 103, the location confirmation unit may determine the location of the farmland where the mechanical tractor 105 is located based on the received farmland information.

According to an embodiment, the farmland information may include location information where the mechanical tractor 105 is located and crop information about crops being cultivated in the farmland. The location information may be information obtained through a location tracking module included in at least one of the HMI display 103 and the mechanical tractor 105 .

According to another embodiment, the crop information about the crops may include the name of the crops, a harvesting method of the crops, an appropriate temperature required for the growth of the crops, an appropriate amount of precipitation required for the growth of the crops, an appropriate growth period of the crops, and the like. In addition, the farmland information may be acquired through other electronic devices. For example, the other electronic device may include a smart phone and a desktop, and may be obtained through a vehicle such as an unmanned drone. That is, when the management server 101 or the HMI display 103 acquires image or video information on crops through other electronic devices, the obtained image or video information on crops is analyzed, and in the crops database It is possible to obtain crop information on crops based on the analysis result.

According to another embodiment, the HMI display 103 may further include a separate camera module. The HMI display 103 may provide the location information to the location confirmation unit in real time and simultaneously provide image information on farmland received through the camera module. Accordingly, the positioning unit may grasp information on farmland where the tractor is working (eg, slope of farmland, area of farmland, growth state of cultivated crops, etc.) through the received image information.

According to an embodiment, the crop information is information on crops previously input to the HMI display 103, and may include information such as the name, variety, and harvest time of the crop. At this time, the crop information may also include image information about the crops received through the camera module of the HMI display 103 .

That is, the location confirmation unit receives map data for the farmland from an outside (eg, public institution server, private company server (Google Earth server, Naver map server)), and maps the location information on the map data to obtain the mechanical The exact position of the tractor 105 can be checked in real time. In this case, the outside that provided the map data may be an outside contracted with the management server 101 .

According to another embodiment, the positioning unit provides image data corresponding to the image to the map data when the topography located in the image received through the HMI display 103 is different from the topography in the map data from the outside. can be provided externally.

According to one embodiment, the climate confirmation unit receives climate information corresponding to the confirmed location from the Meteorological Administration server when the location confirmation of the farmland where the mechanical tractor 105 is located is completed by the location confirmation unit, You can check the precipitation value, temperature value, and wind volume value.

For example, if the location of the farmland confirmed by the location confirmation unit is "Farmland No. 4 located in Jakcheon-myeon, Gangjin-gun, Jeollanam-do", the weather confirmation unit may receive climate information about "Jakcheon-myeon, Gangjin-gun, Jeollanam-do" from the Meteorological Administration server. there is.

According to an embodiment, the work data generation unit 101c, when receiving the engine information, the hitch information, and the farmland information is completed, based on a pre-stored algorithm, the engine information, the hitch information, and the farmland information. By analyzing the information, work pattern data of the mechanical tractor 105 may be generated and transmitted to the HMI display 103 .

According to an embodiment, the work pattern data may be data for allowing the HMI display 103 to autonomously drive the mechanical tractor 105 to harvest crops in farmland. That is, the work pattern data may be information for operating the mechanical tractor 105 that is changed according to the climate and situation (eg, slope) of farmland. A detailed description related to the work data generator 101c will be described with reference to FIG. 4 .

According to an embodiment, when the HMI display 103 receives the work pattern data, the work progress unit 101d causes the HMI display 103 to perform the mechanical tractor (through the received work pattern data). 105) can be requested to control.

According to an embodiment, the work progressing unit 101d requests the HMI display 103 to control the mechanical tractor 105, so that the mechanical tractor 105 is driven autonomously by the HMI display 103. can be made

2 is a diagram for explaining a power information receiving unit of a system for assisting autonomous driving of a mechanical tractor through a human machine interface according to an embodiment of the present invention.

Referring to FIG. 2, a system 100 for assisting autonomous driving of a mechanical tractor through a human machine interface implemented as a computing device including one or more processors and one or more memories storing instructions executable by the processors (hereinafter, Referred to as an auxiliary system) is a management server (e.g. management server 101 in FIG. 1), an HMI display 203 (e.g. HMI display 103 in FIG. 1) and a mechanical tractor (e.g. a mechanical tractor in FIG. 1 ( 105)).

According to one embodiment, the management server may include a power information receiving unit 201 (eg, the power information receiving unit 101a of FIG. 1).

According to an embodiment, the power information receiver 201 may receive engine information and hitch information through the HMI display 203 . For a detailed description of the engine information and the hitch information, refer to FIG. 1 .

According to an embodiment, the power information receiving unit 201 is configured to display the HMI display 203 when the mechanical tractor starts working in a state in which the HMI display 203 is connected to a connection terminal located in one area of the mechanical tractor ( Through 203), engine information and hitch information of the mechanical tractor may be received.

According to an embodiment, the power information receiving unit 201 may perform CAN (controller area network) communication with the mechanical tractor and receive engine information and hitch information when the HMI display 203 receives engine information and hitch information. The engine information and the hitch information may be received from the HMI display by performing cellular communication with the HMI display.

According to one embodiment, when the HMI display 203 is connected to the mechanical tractor through a connection terminal of the mechanical tractor, it may be interlocked with an electronic control unit (ECU) 205 installed in the mechanical tractor.

The ECU 205 receives input values from various sensors (eg, temperature sensor 205a, oil sensor 205b, engine sensor 205c, hitch sensor 205d, gear position sensor, etc.) installed in the mechanical tractor. received, and the received input value can be output to the instrument panel of the mechanical tractor. In addition, the ECU 205 may receive various engine-related input values (eg, engine speed) from an engine management system electronic control unit (EMS ECU) located in the mechanical tractor.

According to an embodiment, the ECU 205 may generate a pressure value for pressure applied to a hitch of a mechanical tractor from the hitch sensor 205d, a flow rate value for a flow rate of the hitch, and a location of the hitch. A hitch value including at least one of a position value and a hitch horizontality value relative to the horizontality of the hitch may be received.

That is, the HMI display 203 may generate engine information and hitch information based on various input values input to the ECU 205 and provide the generated engine information and hitch information to the power information receiver 201 .

3 is a diagram for explaining a farmland information receiving unit of a system for assisting autonomous driving of a mechanical tractor through a human machine interface according to an embodiment of the present invention.

Referring to FIG. 3 , a system 100 for assisting autonomous driving of a mechanical tractor through a human machine interface implemented as a computing device including one or more processors and one or more memories storing instructions executable by the processors (hereinafter, Referred to as an auxiliary system) may include a management server (eg, management server 101 of FIG. 1 ) and a mechanical tractor 303 (eg, mechanical tractor 105 of FIG. 1 ).

According to one embodiment, the mechanical tractor 303 may be connected to an HMI display (eg, the HMI display 103 of FIG. 1 ) through a connection terminal.

According to one embodiment, the farmland information receiving unit 301 is connected to the mechanical tractor 303 and the HMI display, when the mechanical tractor 303 starts working, the mechanical tractor through the HMI display Farmland information of farmland where 303 is located may be received.

According to one embodiment, the farmland information receiving unit 301 may include a location confirmation unit (not shown) and a climate confirmation unit (not shown).

According to an embodiment, when receiving the farmland information from the HMI display, the positioning unit may determine the location of the farmland where the mechanical tractor 303 is located based on the received farmland information.

According to an embodiment, the farmland information may include location information where the mechanical tractor 303 is located and crop information about crops being cultivated in the farmland. The location information may be information obtained through a location tracking module 303a included in the mechanical tractor 303. For another example, the location information may be information obtained through a location tracking module included in the HMI display.

That is, the positioning unit can determine the location of the farmland where the mechanical tractor 303 is located based on the received location information, and can more accurately determine the location of the mechanical tractor 303 .

According to an embodiment, the cultivated crop information may be information about cultivated crops previously input to the HMI display, and may include information such as the name, variety, and harvest time of the cultivated crop. At this time, the location confirmation unit receives map data for the farmland from an outside (eg, public institution server, private company server (Google Earth server, Naver map server)), maps the location information on the map data, and The exact position of the mechanical tractor 303 can be checked in real time.

According to an embodiment, when the location confirmation of the farmland where the mechanical tractor 303 is located is completed by the location confirmation unit, the weather confirmation unit transmits climate information 305a corresponding to the confirmed location to the Korea Meteorological Administration server 305. It is received from , and it is possible to check the value of precipitation, temperature, and wind volume of the farmland.

For example, if the location of the farmland confirmed by the location confirmation unit is "Farmland No. 4 located in Jakcheon-myeon, Gangjin-gun, Jeollanam-do", the climate confirmation unit receives climate information 305a for "Jakcheon-myeon, Gangjin-gun, Jeollanam-do" from the Korea Meteorological Administration server. can receive

According to another embodiment, when the reception of the climate information 305a is completed, the farmland information receiving unit 301 operates a method of operating the mechanical tractor 303 suitable for the farmland environment based on the received climate information 305a. Information may be extracted from a pre-stored operation table. In general, the mechanical tractor 303 needs to be directly operated for a certain period of time while the user is on board. can be extracted from and output through the HMI display. In this case, the operation method information may be information generated as a pre-stored algorithm performs machine learning.

4 is a view for explaining a work progressing part of a system for assisting autonomous driving of a mechanical tractor through a human machine interface according to an embodiment of the present invention.

Referring to FIG. 4, a system 100 for assisting autonomous driving of a mechanical tractor through a human machine interface implemented as a computing device including one or more processors and one or more memories storing instructions executable by the processors (hereinafter, Referred to as an auxiliary system) may include a management server (eg, the management server 101 of FIG. 1). The management server may include a task progressing unit 400 (eg, the task progressing unit 101d in FIG. 1 ). For a brief description of the work proceeding unit 400, refer to FIG. 1.

According to an embodiment, the work progress unit 400 may include a driving state determination unit 401, a work state determination unit 403, a climate state determination unit 405, and an operation information acquisition unit 407. .

According to an embodiment, when engine information is received by the power information receiver (eg, the power information receiver 101a of FIG. 1 ), the driving state determination unit 401, based on the received engine information, Engine speed information, gear condition information, transmission oil temperature information, coolant temperature information, and battery voltage information at a first time within a specified time range may be analyzed. For a detailed description of the engine information, refer to FIG. 1 .

According to an embodiment, the driving state determination unit 401 determines the mechanical tractor based on a result of analyzing the engine speed information, the gear state information, the transmission oil temperature information, the coolant temperature information, and the battery voltage information. (eg, the driving state of the mechanical tractor 105 of FIG. 1) can be determined.

According to one embodiment, the driving state, when the mechanical tractor is running at the first time, the state of the gear, brake and accelerator operated by the user riding on the mechanical tractor and the gear, brake and accelerator operated by the user It may be information including all of the mission state, the coolant state, and the battery state according to the state of the .

According to one embodiment, the work state determining unit 403, when receiving the hitch information by the power information receiving unit is completed, based on the received hitch information, the pressure information at the first time of the designated time range , flow information, hitch location information and hitch level information can be analyzed. For a detailed description of the hitch information, refer to FIG. 1 .

According to an embodiment, the work state confirmation unit 403 may determine the work state of the mechanical tractor based on a result of analyzing the pressure information, the flow rate information, the hitch position information, and the hitch level information. .

The hitch is configured to be substantially connected to a work machine mounted on a mechanical tractor, and the power information receiver may receive hitch information about an external force applied to the hitch through a hitch sensor.

According to one embodiment, in the working state, when the mechanical tractor is working at the first time, the working machine connected to the hitch is moved as the mechanical tractor is operated by a user riding the mechanical tractor. At this time, the hitch is loaded. It may be information that includes all conditions of the applied pressure, the flow rate of the hitch, the position of the hitch, and the horizontality of the hitch. That is, the work state may be information capable of determining the state of the work machine based on the hitch information. For example, when it is confirmed that the horizontality of the hitch is not horizontal but forming a certain angle, it can be confirmed that the mechanical tractor is currently working on an inclined area. For another example, when the pressure of the hitch exceeds a specified pressure value, it may be determined that the mechanical tractor's work machine is caught in an obstacle.

According to an embodiment, the climatic state determining unit 405, when receiving farmland information by the farmland information receiving unit (eg, the farmland information receiving unit 101b of FIG. 1 ) is completed, at a first time in the designated time range of farmland location information, climate information, and crop information can be analyzed. For a detailed description related to the farmland information, refer to FIG. 1 . The climate state determination unit 405 may check the climate state during operation of the mechanical tractor based on the analysis result.

According to an embodiment, the climate condition may be information for confirming the climate of an area where the mechanical tractor is located while the mechanical tractor is driving or working.

According to an embodiment, when the driving state, the working state, and the climate state at the first time are determined, the manipulation information acquisition unit 407 performs a mechanical operation at the first time based on a pre-stored algorithm. First operation information on the tractor may be acquired. The pre-stored algorithm may be an algorithm for controlling the mechanical tractor through an HMI display (eg, the HMI display 103 of FIG. 1 ) to obtain manipulation information necessary for the mechanical tractor to autonomously drive.

According to an embodiment, when the first manipulation information is generated through the pre-stored algorithm, the manipulation information acquisition unit 407 includes not only the driving state, working state and climate state, but also crop information and mechanical information input to the HMI display. By analyzing the operation image information of the tractor, it is possible to generate operation information suitable for autonomous driving of the mechanical tractor according to the farmland environment.

For example, the operation information acquisition unit 407 checks the driving state and confirms that the accelerator and brake of the mechanical tractor are currently operated while being crossed, but the degree of pressing the accelerator is gradually increasing. . In addition, the operation information acquisition unit 407 may check the driving state and confirm that the gear state of the mechanical tractor is "drive mode". The manipulation information acquisition unit 407 may check the working state and confirm that the hitch of the mechanical tractor applies a pressure of 40P in the right direction. In addition, the operation information acquisition unit 407 may confirm that the horizontality of the hitch is formed downward by checking the working state. The operation information acquisition unit 407 may check the climate condition and confirm that rain is falling on the farmland where the mechanical tractor is located.

Accordingly, the manipulation information acquisition unit 407 may generate first manipulation information based on the determined driving state, working state, and climate state through a pre-stored algorithm. The operation information acquisition unit 407 can confirm that a part of the work machine is stuck in the ground (the level is lowered down) because the ground is not strong because it is currently raining on the farmland through the pre-stored algorithm, and mechanical It may be determined that an occupant riding the tractor alternately operates the brake and the accelerator to overcome the pressure caused by the ground and crops in contact with the work machine, and gradually presses the accelerator hard to increase the output of the engine.

That is, the operation information acquisition unit 407 may generate first operation information based on a tractor operation method of a passenger riding a mechanical tractor in rainy weather, based on the pre-stored algorithm, in consideration of the determination. .

At this time, the manipulation information acquisition unit 407 may analyze the crop information input to the HMI display and the work image information of the mechanical tractor based on a pre-stored algorithm, and generate first manipulation information reflecting the analyzed content. . The first operation information may include operation information of the mechanical tractor at the first time (information on states of operating components to perform work and travel). That is, the first operation information may refer to operation information of a mechanical tractor suitable for farmland conditions at the first time.

5 is a view for explaining an operation information acquisition unit of a system for assisting autonomous driving of a mechanical tractor through a human machine interface according to an embodiment of the present invention.

Referring to FIG. 5, a system 100 for assisting autonomous driving of a mechanical tractor through a human machine interface implemented as a computing device including one or more processors and one or more memories storing instructions executable by the processors (hereinafter, Referred to as an auxiliary system) may include a manipulation information obtaining unit 501 (eg, the manipulation information obtaining unit 407 of FIG. 4 ).

According to an embodiment, the operation information acquisition unit 501 includes a driving state determination unit (eg, the driving state determination unit 401 of FIG. 4 ), a work state determination unit (eg, the work state determination unit 403 of FIG. 4 ) )) and operation information for each of a plurality of times 503a, 503b, and 503c included in the designated time range 503 due to the performance of the function of the climate state determination unit (eg, the climate state determination unit 405 of FIG. 4) When acquisition of is completed, work pattern data for manipulation of the mechanical tractor may be generated based on manipulation information for each of the plurality of times 503a, 503b, and 503c.

According to an embodiment, the manipulation information acquisition unit 501 may acquire first manipulation information 503a at a first time in the designated time range 503 . For a detailed description related to the manipulation information, refer to FIG. 4 . When the acquisition of the first manipulation information 503a is completed, the manipulation information acquisition unit 501 may obtain second manipulation information 503b at a second time in the designated time range 503 . In addition, when the acquisition of the second manipulation information 503b is completed, the manipulation information acquisition unit 501 may acquire third manipulation information 503c at a third time in the designated time range 503. .

According to one embodiment, the manipulation information acquisition unit 501 arranges the first manipulation information 501a, the second manipulation information 501b, and the third manipulation information 501c in chronological order, and sets the specified time It is possible to generate work pattern data of mechanical tractors that have been driven and worked while driving. The work pattern data may be autonomous driving and autonomous work data necessary for an HMI display (eg, the HMI display 103 of FIG. 1 ) to autonomously drive by controlling the mechanical tractor.

According to an embodiment, the operation information acquiring unit 501 may store the acquired work pattern data in a category table. The category table may be a data table in which the work pattern data is classified and stored for each work process. For example, the category table may include a plurality of categories. Each of the plurality of categories may be a category for different work processes. A first category among the plurality of categories may be a category in which work pattern data obtained during rainy weather is stored. The second category may be a category in which work pattern data obtained during a heat wave is stored. The third category may be a category in which work pattern data obtained from farmland including a plurality of gravel is stored.

According to an embodiment, the pre-stored algorithm may machine-learn the generated work pattern data. The pre-stored algorithm may be updated to generate different working pattern data for different driving conditions, different working conditions, and different climate conditions by machine learning the generated working pattern data. In more detail, the pre-stored algorithm performs machine learning on the work pattern data stored in the first category and the work pattern data stored in the second category to determine driving conditions, working conditions, and climate conditions including new contents. New work pattern data can be created.

For example, the pre-stored algorithm performs an accelerator operation based on work pattern data obtained in rainy weather stored in a first category and an accelerator operation based on work pattern data obtained in strong winds stored in a fourth category through machine learning. , It is possible to create new work pattern data including an accelerator operation method (driving condition including new contents) in a heavy rain warning or typhoon situation.

6 is another diagram for explaining a work progressing part of a system for assisting autonomous driving of a mechanical tractor through a human machine interface according to an embodiment of the present invention.

Referring to FIG. 6, a system 100 for assisting autonomous driving of a mechanical tractor through a human machine interface implemented as a computing device including one or more processors and one or more memories storing instructions executable by the processors (hereinafter, Referred to as an auxiliary system) may include a work progress unit 601 (eg, the work progress unit 101d of FIG. 1 ).

According to an embodiment, the work progressing unit 601 transmits the generated work pattern data 601a to the HMI display 603 (eg, the HMI display 103 of FIG. 1), and the HMI display 603 may request to autonomously drive the mechanical tractor 605 within the farmland.

According to one embodiment, the HMI display 603 may include a driving start unit (not shown). When the operation start unit receives the work pattern data 601a from the work progress unit 601, the mechanical tractor 605 may be controlled based on the received work pattern data 601a. That is, the HMI display 603 can autonomously drive the mechanical tractor 605 through the work pattern data 601a to perform work within the farmland.

According to an embodiment, the HMI display 603, when the driving start unit controls the mechanical tractor 605, the engine information obtained as the mechanical tractor 605 is operated, the hitch information and the Farmland information may be transmitted to a management server (eg, the management server of FIG. 1) in real time.

7 illustrates an example of an internal configuration of a computing device according to an embodiment of the present invention, and in the following description, descriptions of unnecessary embodiments overlapping with those of FIGS. 1 to 6 will be omitted. do it with

As shown in FIG. 7, a computing device 10000 includes at least one processor 11100, a memory 11200, a peripheral interface 11300, an input/output subsystem ( It may include at least an I/O subsystem (11400), a power circuit (11500), and a communication circuit (11600). In this case, the computing device 10000 may correspond to a user terminal connected to the tactile interface device (A) or the aforementioned computing device (B).

The memory 11200 may include, for example, high-speed random access memory, magnetic disk, SRAM, DRAM, ROM, flash memory, or non-volatile memory. there is. The memory 11200 may include a software module, a command set, or other various data necessary for the operation of the computing device 10000.

In this case, access to the memory 11200 from other components, such as the processor 11100 or the peripheral device interface 11300, may be controlled by the processor 11100.

Peripheral interface 11300 may couple input and/or output peripherals of computing device 10000 to processor 11100 and memory 11200 . The processor 11100 may execute various functions for the computing device 10000 and process data by executing software modules or command sets stored in the memory 11200 .

Input/output subsystem 11400 can couple various input/output peripherals to peripheral interface 11300. For example, the input/output subsystem 11400 may include a controller for coupling a peripheral device such as a monitor, keyboard, mouse, printer, or touch screen or sensor to the peripheral interface 11300 as needed. According to another aspect, input/output peripherals may be coupled to the peripheral interface 11300 without going through the input/output subsystem 11400.

The power circuit 11500 may supply power to all or some of the terminal's components. For example, power circuit 11500 may include a power management system, one or more power sources such as a battery or alternating current (AC), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator or power It may contain any other components for creation, management and distribution.

The communication circuit 11600 may enable communication with another computing device using at least one external port.

Alternatively, as described above, the communication circuit 11600 may include an RF circuit and transmit/receive an RF signal, also known as an electromagnetic signal, to enable communication with another computing device.

The embodiment of FIG. 7 is only an example of the computing device 10000, and the computing device 11000 may omit some of the components shown in FIG. 7, further include additional components not shown in FIG. It may have a configuration or arrangement combining two or more components. For example, a computing device for a communication terminal in a mobile environment may further include a touch screen or a sensor in addition to the components shown in FIG. , Bluetooth, NFC, Zigbee, etc.) may include a circuit for RF communication. Components that may be included in the computing device 10000 may be implemented as hardware including one or more signal processing or application-specific integrated circuits, software, or a combination of both hardware and software.

Methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computing devices and recorded in computer readable media. In particular, the program according to the present embodiment may be configured as a PC-based program or a mobile terminal-only application. An application to which the present invention is applied may be installed in a user terminal through a file provided by a file distribution system. For example, the file distribution system may include a file transmission unit (not shown) that transmits the file according to a request of a user terminal.

The device described above may be implemented as a hardware component, a software component, and/or a combination of hardware components and software components. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. You can command the device. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or to provide instructions or data to a processing device. may be permanently or temporarily embodied in Software may be distributed on networked computing devices and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved. Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (8)

A system for assisting autonomous driving of a mechanical tractor through a human machine interface implemented as a computing device including one or more processors and one or more memories storing instructions executable by the processors,
Receiving engine information and hitch information of the mechanical tractor through the HMI display when the mechanical tractor starts working in a state in which a human machine interface (HMI) display is connected to a connection terminal located in one area of the mechanical tractor a power information receiver to receive;
a farmland information receiver configured to receive farmland information of a farmland where the mechanical tractor is located through the HMI display when the mechanical tractor starts working while the mechanical tractor and the HMI display are connected;
When the reception of the engine information, the hitch information, and the farmland information is completed, based on a pre-stored algorithm, the engine information, the hitch information, and the farmland information are analyzed to generate work pattern data of the mechanical tractor a work data generating unit transmitting to the HMI display; and
When the HMI display receives the work pattern data, it requests the HMI display to control the mechanical tractor through the received work pattern data, so that the HMI display autonomously drives the mechanical tractor. A system for assisting autonomous driving of a mechanical tractor through a human machine interface, comprising: a unit;
According to claim 1,
The engine information,
As the mechanical tractor starts driving, information corresponding to the information output on the instrument panel of the mechanical tractor includes engine speed information, gear condition information, transmission oil temperature information, coolant temperature information, and battery voltage information of the mechanical tractor. Including at least one of
The hitch information,
As the mechanical tractor starts driving, as information measured by the hitch sensor of the mechanical tractor, pressure information on the pressure applied to the hitch of the mechanical tractor, flow rate information on the flow rate of the hitch, the A system for assisting autonomous driving of a mechanical tractor through a human machine interface, characterized in that it includes at least one of hitch location information for the position of the hitch and hitch horizontality information for the horizontality of the hitch.
According to claim 1,
The system,
CAN (controller area network) communication is performed between the mechanical tractor and the HMI display, and cellular communication is performed between a management server that manages the HMI display and the mechanical tractor and the HMI display,
When the engine information, the hitch information, and the farmland information are completely received, the engine information, the hitch information, and the farmland information are stored in a database of the management server. A system that assists autonomous driving.
According to claim 1,
The farmland information receiving unit,
When receiving the farmland information from the HMI display, based on the received farmland information, a positioning unit for confirming the location of the farmland where the mechanical tractor is located; and
When the location confirmation of the farmland where the mechanical tractor is located is completed by the location confirmation unit, climate information corresponding to the confirmed location is received from the Meteorological Administration server to confirm the precipitation value, temperature value, and wind volume value of the farmland. Including; confirmation unit;
The farmland information,
A system for assisting autonomous driving of a mechanical tractor through a human machine interface, characterized in that it includes location information of the farmland where the mechanical tractor is located and crop information about crops being cultivated in the farmland.
According to claim 4,
The work data generating unit,
When the reception of the engine information is completed, the engine speed information, the gear state information, the transmission oil temperature information, the coolant temperature information, and the engine speed information at a first time in a designated time range based on the received engine information a driving state determining unit analyzing battery voltage information and determining a driving state of the mechanical tractor;
When the reception of the hitch information is completed, based on the received hitch information, the pressure information, the flow rate information, the hitch position information, and the hitch horizontality information at a first time in the specified time range are analyzed, a working state determining unit for determining a working state of the mechanical tractor;
When the reception of the farmland information is completed, a climate state determination unit for determining a climate state during operation of the mechanical tractor by analyzing the farmland location information, the climate information, and the crop information at a first time in the designated time range. ; and
When the determination of the driving state, the working state, and the climate state at the first time is completed, an operation of obtaining first operation information on the mechanical tractor at the first time based on the pre-stored algorithm A system for assisting autonomous driving of a mechanical tractor through a human machine interface comprising a; information acquisition unit.
According to claim 5,
The operation information acquisition unit,
When the acquisition of operation information for each of a plurality of times included in the designated time range is completed due to the performance of the driving state determination unit, the work state determination unit, and the climate state determination unit, the operation information for each of the plurality of times Based on the operation information, a system for assisting autonomous driving of a mechanical tractor through a human machine interface, characterized in that for generating work pattern data for the operation of the mechanical tractor.
According to claim 6,
The pre-stored algorithm,
An algorithm for generating the work pattern data, characterized in that the generated work pattern data is machine-learned to generate other work pattern data for different driving conditions, different working conditions, and different climate conditions. A system that assists the autonomous driving of a mechanical tractor through a human machine interface.
According to claim 7,
The HMI display,
When receiving the work pattern data from the work progressing unit, based on the received work pattern data, a driving start unit that controls the mechanical tractor connected through the connection terminal to autonomously drive to harvest crops in the farmland; include,
The driving start unit controls the mechanical tractor to transmit the engine information, the hitch information, and the farmland information obtained as the mechanical tractor is driven to the management server in real time. A system that assists the autonomous driving of

Images