KR102127312B1 - Moving body, particularly agricultural working vehicle and system of controlling the same - Google Patents

Abstract

An agricultural work vehicle according to an embodiment includes a vehicle body for moving along a driving path; A detector installed on the vehicle body to sense a current position and posture of the vehicle body, and a detector installed on the vehicle body to acquire a video image of a surrounding environment of the vehicle body; And a current state generating unit generating a current state of the vehicle body based on a current position and posture of the vehicle body, a driving area generating unit generating information about a travelable area of the vehicle body based on the image image, and of the vehicle body. A processor including a next state determination unit that determines a target position and a posture of the vehicle body based on a current state and information on a vehicle's travelable area, and a control unit that controls the vehicle body based on the target position and posture of the vehicle body It can contain.

Description

A moving vehicle, especially an agricultural work vehicle, and a control system thereof [MOVING BODY, PARTICULARLY AGRICULTURAL WORKING VEHICLE AND SYSTEM OF CONTROLLING THE SAME}

Hereinafter, embodiments relate to a moving object and a control system thereof.

The agricultural work vehicle is a vehicle for managing crops in arable land, and it is required to drive the arable land without damaging the crop. Accordingly, the user's skilled experience and concentration are required for the agricultural work vehicle to accurately drive within the cultivated land. A method and apparatus for automatically controlling an agricultural work vehicle have been developed to meet such needs. For example, after installing a GPS system in an agricultural work vehicle, there is a method of sensing a characteristic of a road on which the agricultural work vehicle travels through the GPS system.

Korean Patent Publication No. 10-2017-0053332 (released on May 16, 2017)

An object according to an embodiment is to provide a moving object and a control system thereof for driving a farmland using 2D image information instead of 3D spatial information without a separate geographic information database.

An agricultural work vehicle according to an embodiment includes a vehicle body for moving along a driving path; A detector installed on the vehicle body to sense a current position and posture of the vehicle body, and a detector installed on the vehicle body to acquire a video image of a surrounding environment of the vehicle body; And a current state generating unit generating a current state of the vehicle body based on a current position and posture of the vehicle body, a driving area generating unit generating information about a travelable area of the vehicle body based on the image image, and of the vehicle body. A processor including a next state determination unit that determines a target position and a posture of the vehicle body based on a current state and information on a vehicle's travelable area, and a control unit that controls the vehicle body based on the target position and posture of the vehicle body It can contain.

The next state determining unit may generate a control vector indicating a target position and attitude of the vehicle body based on the current position and attitude of the vehicle body, and transmit the generated control vector to the control unit.

The camera may acquire a video image of a surrounding environment of the vehicle body using at least one of light in the near-infrared band and light in the visible band.

A control system for a moving object according to an embodiment includes a sensor for detecting a current position and posture of the moving object, and a detector including a camera that acquires a video image of a surrounding environment of the moving object; And a current state generating unit generating a current state of the moving object based on the current position and posture of the moving object, a driving area generating unit generating driving area information of the moving object based on the image image, and of the moving object. A processor including a next state determination unit for determining a target position and a posture of the moving object based on a current state and information on a movable area of the moving object, and a control unit for controlling the moving object based on the target position and posture of the moving object. It can contain.

The next state determining unit may generate a control vector indicating a target position and a posture of the moving object based on the current position and posture of the moving object, and transmit the generated control vector to the control unit.

The camera may acquire a video image of the surrounding environment of the moving object using at least one of light in the near-infrared band and light in the visible band.

A control system for an agricultural work vehicle according to an embodiment is formed in a surrounding environment including at least one of a ground, a plurality of vertical structures perpendicular to the ground, and a plurality of horizontal structures connecting the plurality of vertical structures. Become an identifier; And a detector for moving on the ground, a detector for acquiring a two-dimensional image image of the surrounding environment including the identifier, and detecting inertia information of the vehicle body, and based on the two-dimensional image image and the inertia information of the vehicle body. An agricultural work vehicle including a processor for determining target inertia information of the vehicle body on the ground may be included.

The processor may determine the target inertia information using at least one of figure information, color information, and brightness information of an identifier included in the 2D image.

The processor includes a boundary between the ground on the 2D video image and a plurality of vertical structures perpendicular to the ground; A plurality of horizontal structures connecting the plurality of vertical structures; And it is possible to recognize at least one of the plate-like body on the paper as an identifier.

The processor generates driving area information of the vehicle body using pixels of the 2D video image, and based on the driving area information expressed in pixels and setting inertia information of the detector for the vehicle body, a 2D video image Target inertia information of the body of the jacket can be determined.

The identifier may be formed along the plurality of horizontal structures, formed at the ends of the plurality of horizontal structures, formed at a boundary formed by the ground and the plurality of vertical structures, or formed on the ground in a plate shape. .

The identifier may be formed in a pattern that reflects or absorbs light of a set wavelength, a pattern that includes at least two colors, or may include a light source that emits light of a set wavelength.

A moving object according to an embodiment, particularly an agricultural work vehicle, can accurately drive within a farmland without an expensive GPS system using 3D spatial information.

A moving object according to an embodiment, particularly an agricultural work vehicle, may move along a driving path in a farmland without additional hardware or software for constructing a geographic information database.

The effects of the moving object according to one embodiment, especially the agricultural working vehicle and its control system are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a block diagram of a control system of an agricultural work vehicle according to an embodiment.
FIG. 2 is a block diagram of the processor of FIG. 1.
3 is an example of a video image acquired by the camera of FIG. 1.
4 is a diagram for explaining a method of recognizing an identifier on a video image by a control system of an agricultural work vehicle according to an embodiment.
5 to 7 are diagrams for describing parameters used to control an agricultural work vehicle according to an embodiment.
8 is a diagram for describing a method of determining a target position and posture of an agricultural work vehicle according to an embodiment.

Hereinafter, embodiments will be described in detail through exemplary drawings. It should be noted that in adding reference numerals to the components of each drawing, the same components have the same reference numerals as possible even though they are displayed on different drawings. In addition, in describing the embodiments, when it is determined that detailed descriptions of related well-known structures or functions hinder understanding of the embodiments, detailed descriptions thereof will be omitted.

In addition, in describing the components of the embodiment, terms such as first, second, A, B, (a), (b), and the like can be used. These terms are only for distinguishing the component from other components, and the nature, order, or order of the component is not limited by the term. When a component is described as being "connected", "coupled" or "connected" to another component, the component may be directly connected to or connected to the other component, but another component between each component It should be understood that may be "connected", "coupled" or "connected".

Components included in any one embodiment and components including a common function will be described using the same name in other embodiments. Unless there is an objection to the contrary, the description in any one embodiment may be applied to other embodiments, and a detailed description will be omitted in the overlapping range.

1 is a block diagram of a control system of an agricultural work vehicle according to an embodiment.

Referring to FIG. 1, the control system 1 for an agricultural work vehicle according to an embodiment sets a driving path of the agricultural work vehicle based on the image image of the surrounding environment of the agricultural work vehicle and the inertia information of the agricultural work vehicle , It is possible to control the vehicle so that the agricultural work vehicle can move along the set driving path. Agricultural work vehicles may include, for example, tractors, cultivators, etc., but are not limited thereto, and vehicles related to agricultural work may include all vehicles apparent to other skilled workers. On the other hand, it is revealed that the control system 1 according to an embodiment can be applied not only to agricultural work vehicles, but also to moving objects that are moving objects.

The control system 1 for an agricultural work vehicle may include a detector 110, an input unit 120, a processor 130, a driving unit 140, a communication unit 150, and a display unit 160.

The detector 110 may detect inertia information of the agricultural work vehicle, the surrounding environment of the agricultural work vehicle, and the like. For example, the detector 110 may include a sensor 112 and a camera 114. The sensor 112 may detect the position and posture of an agricultural work vehicle. As an example, the sensor 112 may include an acceleration sensor, an angular velocity sensor, a gyro sensor, and the like. The camera 114 may acquire a video image of the surrounding environment of the agricultural work vehicle. Here, the image image may include a plurality of pixels represented by, for example, a two-dimensional Cartesian coordinate system. For example, the camera 114 may acquire an image image using near infrared ray (NIR), visible ray, or a combination thereof. In addition, the camera 114 may acquire a lidar sensor image image using a near infrared or visible laser light source. The sensor 112 and the camera 114 as described above may be installed in an agricultural work vehicle.

The input unit 120 may receive input from a user of an agricultural work vehicle. For example, the input unit 120 may input on the number of people in the agricultural work vehicle, input on weather information, input on the weight of the load loaded on the agricultural work vehicle, and the status of the driving route through which the agricultural work vehicle moves. Inputs, inputs related to driving control of an agricultural work vehicle, and the like can be received.

The processor 130 may transmit command signals to the driving unit 140, the communication unit 150, and the display unit 160, respectively.

Processor 130 based on at least one of the inertia information of the agricultural work vehicle, a video image of the surrounding environment of the agricultural work vehicle and the user's input, information about the current state of the agricultural work vehicle and the driveable area of the agricultural work vehicle It generates information on each, and based on the information, it is possible to determine the target inertia information of the agricultural work vehicle.

The processor 130 may generate driving area information of the agricultural work vehicle using a plurality of pixels constituting a video image of the surrounding environment of the agricultural work vehicle. Here, the travelable area information may be represented by a plurality of pixels. The processor 130 may determine target inertia information on the video image based on the set inertia information of the camera 114 installed in the agricultural work vehicle and the travelable area information represented by a plurality of pixels. For example, setting inertia information of the camera 114 may include a height of the camera 114 with respect to a vehicle, a tilt of the camera 114, and the like.

The processor 130 may recognize an identifier included in a video image of the surrounding environment of an agricultural work vehicle. The identifier may have any form suitable for the camera 114 to recognize. The identifier may include a vertical structure, a horizontal structure, a ground on which an agricultural work vehicle moves, a boundary between the vertical structure and the ground, a plate-like structure on the ground, and the like, included in the video image. For example, the vertical structure may include a tree, a pillar, etc., and the horizontal structure may include a tree, a line connecting the pillars, a pipe for supplying water, and the like. As another example, the identifier may be formed in a pattern that reflects or absorbs light of a set wavelength. As a further example, the identifier may be formed in a pattern including at least two colors. In the above two cases, the identifier may be formed of a material that performs a retroreflective function. As another additional example, the identifier may include a light source that emits light of a set wavelength.

The processor 130 may recognize an identifier included in a video image through a series of recognition techniques. The processor 130 may recognize the identifier using at least one of figure information, color information, and brightness information of the video image. In one example, the processor 130 may detect a vertical structure or horizontal structure included in the video image as one straight line. Here, the linear detection method may include, for example, Hough transformation after pre-processing a video image. In another example, the processor 130 groups a plurality of pixels each having a unique color value within the set range based on whether the unique color values of the plurality of pixels constituting the video image fall within the set range. A straight line can also be detected. In another example, the processor 130 may randomly group a plurality of pixels each having a unique brightness value within a set range based on whether or not the unique brightness values of a plurality of pixels constituting a video image fall within a set range. You can also detect the form of The processor 130 may determine the target inertia information of the agricultural work vehicle by recognizing the identifier as described above.

When the processor 130 does not recognize the identifier on the video image, for example, when the surrounding environment of the agricultural work vehicle has complex and various environmental change factors to acquire the video image, the attitude of the agricultural work vehicle, The detector 110 may be instructed to acquire information about speed, steering angle, and the like again.

The processor 130 may be installed in an agricultural work vehicle, but is not limited thereto, and may be included in an external system that remotely controls the agricultural work vehicle.

The driving unit 140 may receive a command signal from the processor 130 to control driving of the agricultural work vehicle. For example, the driving unit 140 may include a motor, an engine, and the like.

The communication unit 150 may receive a command signal from the processor 130 to communicate with an external system that remotely controls the agricultural work vehicle. For example, the communication unit 150 may use a wireless communication method.

The display unit 160 may display information related to the state of the agricultural work vehicle so that the user can monitor the state of the agricultural work vehicle. The display unit 160 may be installed in an agricultural work vehicle, but is not limited thereto, and may be installed in an external system that remotely controls the agricultural work vehicle.

FIG. 2 is a block diagram of the processor of FIG. 1.

Referring to FIG. 2, the processor 130 according to an embodiment includes an information input/output unit 131, a driving area generation unit 133, a current state generation unit 135, a next state generation unit 137, and a control unit 139 ).

The information input/output unit 131 includes inertia information of the agricultural work vehicle, a video image of the surrounding environment of the agricultural work vehicle, user input and output to the user, driving information of the agricultural work vehicle according to a command signal, communication information, and the like. Outputs can be relayed.

The driving area generating unit 133 may receive a video image of the surrounding environment of the agricultural working vehicle and generate driving area information of the agricultural working vehicle. The current state generating unit 135 may receive the inertia information of the agricultural work vehicle and generate the current state of the agricultural work vehicle.

Next, the state determining unit 137 may determine target inertia information of the agricultural work vehicle based on the current state of the agricultural work vehicle and information about the travelable area of the agricultural work vehicle. Here, the target inertia information may include the target position and posture of the agricultural work vehicle.

When determining the target inertia information of the agricultural work vehicle, the next state determination unit 137 may generate a control vector indicating the target position and attitude of the agricultural work vehicle based on the current position and attitude of the agricultural work vehicle. The next state determination unit 137 may transmit the generated control vector to the control unit 139.

The control unit 139 may transmit a command signal to the information input/output unit 131 based on the control vector received from the next state determination unit 137. The command signal may include driving information for controlling an agricultural work vehicle. The driving information may include steering, driving torque, and the like of an agricultural work vehicle.

FIG. 3 is an example of a video image acquired by the camera of FIG. 1, and FIG. 4 is a view for explaining a method of recognizing an identifier on the video image by a control system of an agricultural work vehicle according to an embodiment.

Referring to FIGS. 3 and 4, the control system of the agricultural work vehicle according to an embodiment may acquire an image image IM of the surrounding environment of the agricultural work vehicle as illustrated in FIG. 3. The control system of agricultural work vehicles includes relatively bright spots such as leaves, weeds, clouds, and sky in the image image (IM) and tree trunks and branches ( Branches, soils, etc. can be used for points that appear relatively dark.

The control system of the agricultural work vehicle includes a plurality of horizontal structures (H) connecting a plurality of vertical structures (V) perpendicular to the ground (G), the ground (G) in the image image (IM) (H) ), a plurality of boundary lines L extending along the ground G, a boundary point P between the ground G and the plurality of vertical structures V, and a plate-like body R on the ground G Etc. can be recognized as an identifier. For example, the plurality of vertical structures V may include a tree trunk, a post, and the plurality of horizontal structures H may include a pipeline for supplying water to the tree trunk. Further, the plate-like body R may be formed in a pattern that reflects light of a specific wavelength or a pattern that absorbs it. As another example, the plate-shaped body R may include a light source such as a light emitting diode. As a further example, the plate-like body R may be formed of a reflection pattern or an absorption pattern formed of at least two colors having high color contrast or brightness contrast. Further, the plate-shaped body R may be formed in the above-described reflection pattern or absorption pattern even when it includes a light source. In an alternative example, the plate-like body R on the ground G may be installed not only on the ground G, but also on a plurality of vertical structures V or a plurality of horizontal structures H.

The control system of an agricultural work vehicle may recognize an identifier included in a video image IM using a machine learning technique. For example, the control system can apply naive bayesian classification. In this case, the control system detects the boundary points P formed by any one vertical structure V and the ground G on the image image IM using the naive Bayes classification, and connects the boundary points P After extracting a free space, and estimating a center line of free space based on a linear regression analysis, the center line can be set as a driving path of an agricultural work vehicle.

5 to 7 are diagrams for describing parameters used to control an agricultural work vehicle according to an embodiment.

) 및 설정 롤 각도(roll angle)를 이루며 차체(100)에 설치될 수 있다. 예를 들어, 틸트 각도(

) 및 롤 각도는 각각 실질적으로 0도로 설정되어 카메라(114)의 광축(optical axis)이 지면에 실질적으로 평행할 수 있다. 또한, 광축을 기준으로 수직 방향으로의 카메라(114)의 수직 각도(

), 광축을 기준으로 수평 방향으로 카메라(114)의 수평 각도(

) 및 지면에 대한 카메라(114)의 높이, 차체(100)의 속도, 카메라(114)의 이미지 획득 주기 등을 설정하고, 상기와 같은 파라미터들에 기초하여 농업용 작업 차량(10)의 주행 가능한 영역 및 목표 관성 정보가 결정될 수 있다.5 to 7, the agricultural work vehicle 10 according to an embodiment may include a vehicle body 100 for moving along the driving path MP. A detector including a sensor (not shown) and a camera 114 may be installed in the vehicle body 100. The camera 114 has a tilt angle set relative to the vehicle body 100 (

) And a set roll angle, and may be installed in the vehicle body 100. For example, the tilt angle (

) And the roll angle are each set to substantially 0 degrees so that the optical axis of the camera 114 can be substantially parallel to the ground. In addition, the vertical angle of the camera 114 in the vertical direction with respect to the optical axis (

), the horizontal angle of the camera 114 in the horizontal direction with respect to the optical axis (

) And the height of the camera 114 with respect to the ground, the speed of the vehicle body 100, the image acquisition cycle of the camera 114, and the like, based on the above parameters, the driving area of the agricultural work vehicle 10 And target inertia information.

8 is a diagram for describing a method of determining a target position and posture of an agricultural work vehicle according to an embodiment.

)만큼 회전된, 차체(100)의 중심(C)을 기준으로 목표 위치(TP)를 가리키는 목표 주행 방향(TD)을 결정할 수 있다. 예를 들어, 농업용 작업 차량(10)은 머신 비전(machine vision) 방식을 이용하여 목표 주행 방향(TD)을 결정할 수 있다. 이 경우 사용자가 목표 주행 방향(TD)을 설정하는 것보다 더 안정적이고 정확한 주행 경로의 설정이 가능하다.Referring to FIG. 8, the agricultural work vehicle 10 has a target steering angle (with respect to the current driving direction CD based on the center C of the vehicle body 100)

), the target driving direction TD indicating the target position TP based on the center C of the vehicle body 100 may be determined. For example, the agricultural work vehicle 10 may determine a target driving direction TD using a machine vision method. In this case, it is possible to set a more stable and accurate driving path than the user sets the target driving direction TD.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, or the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiments or may be known and usable by those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and magnetic media such as floptical disks. Includes hardware devices specifically configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language code that can be executed by a computer using an interpreter, etc., as well as machine language codes produced by a compiler. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described by a limited embodiment and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques are performed in a different order than the described method, and/or the components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or other components Alternatively, proper results can be achieved even if replaced or substituted by equivalents.

Claims (12)

A vehicle body for moving along a driving path;
A detector installed on the vehicle body to sense a current position and posture of the vehicle body, and a detector installed on the vehicle body to acquire a video image of a surrounding environment of the vehicle body; And
A current state generating unit that generates a current state of the vehicle body based on a current position and posture of the vehicle body, a driving area generation unit that generates driving area information of the vehicle body based on the video image, and a current of the vehicle body A processor including a next state determination unit that determines a target position and a posture of the vehicle body based on a state and information on a travelable area of the vehicle body, and a control unit that controls the vehicle body based on the target position and posture of the vehicle body;
Including,
The video image has a plurality of pixels represented by a two-dimensional coordinate system, the video image includes an identifier formed in the surrounding environment, and the identifier is a tree or a post perpendicular to the ground of the surrounding environment and the ground It includes a plurality of vertical structures including a plurality of horizontal structures including a line or pipe connecting the plurality of vertical structures,
The processor detects the type of the identifier by grouping the plurality of pixels based on the unique characteristic values of the plurality of pixels, and uses the identifier to determine a target position and posture of the vehicle body.
According to claim 1,
The next state determining unit generates a control vector indicating a target position and a posture of the vehicle body based on the current position and posture of the vehicle body, and delivers the generated control vector to the control unit.
According to claim 1,
The camera is an agricultural work vehicle for acquiring a video image of the surrounding environment of the vehicle body using at least one of light in the near-infrared band and light in the visible band.
According to claim 1,
The processor is an agricultural work vehicle for determining a target position and posture of the vehicle body using at least one of figure information, color information, and brightness information of an identifier included in the 2D video image.
According to claim 1,
The processor
A boundary between the ground on the two-dimensional image and a plurality of vertical structures perpendicular to the ground;
A plurality of horizontal structures connecting the plurality of vertical structures; And
A plate-like body on the ground;
Agricultural work vehicle that recognizes at least one of the identifiers.
According to claim 1,
The identifier is formed along the plurality of horizontal structures, formed at the ends of the plurality of horizontal structures, formed on a boundary formed by the ground and the plurality of vertical structures, or formed in a plate shape on the ground for agricultural work. vehicle.
According to claim 1,
The identifier is formed in a pattern that reflects or absorbs light of a set wavelength, or is formed in a pattern that includes at least two colors, or agricultural work vehicles comprising a light source that emits light of a set wavelength. delete delete delete delete delete

Images