KR102644930B1 - Apparatus and method for guiding pruning - Google Patents

Abstract

A pruning guide device and method are disclosed that provide information about tree branches to be pruned by analyzing images of tree branches using an artificial neural network. The vestibular guide device according to one embodiment includes a camera unit for taking images; An interface unit for inputting and outputting data; And a control unit connected to the camera unit and the interface unit, wherein the control unit receives an image containing one or more tree branches through the camera unit, determines whether the one or more tree branches included in the image are subject to pruning, and performs pruning. Guide information can be created.

Description

Vestibular guidance device and method {Apparatus and method for guiding pruning}

A technology for providing a tree branch pruning method, in particular, relates to a pruning guide device and method that provides information about tree branches to be pruned by analyzing images of tree branches taken using an artificial neural network.

In the past, research was mainly conducted on devices used in pruning, such as in registered patent 10-1129771. However, no research has been done on guide methods that tell which tree branches to prune.

In the case of fruit growing, the growth of the tree and the quality of the fruit are determined by whether unnecessary branches are properly pruned. However, novice farmers have difficulty starting fruit farming due to lack of knowledge and know-how. Accordingly, the need for technology to analyze the condition and type of tree branches and inform the tree branches to be pruned is increasing.

The purpose is to provide a pruning guide device and method that provides information about tree branches to be pruned by analyzing images of tree branches using an artificial neural network.

According to one aspect, the vestibular guide device includes a camera unit for taking images; An interface unit for inputting and outputting data; And a control unit connected to the camera unit and the interface unit, wherein the control unit receives an image containing one or more tree branches through the camera unit, determines whether the one or more tree branches included in the image are subject to pruning, and performs pruning. Guide information can be created.

The control unit may control the interface unit to output at least one of an image including one or more tree branches and pruning guide information.

The control unit includes an artificial neural network trained with learning data generated based on one or more learning images labeled for the type of tree branch, and uses the artificial neural network to determine the type of tree included in the image containing one or more tree branches. More than one type of tree branch can be analyzed.

One or more learning images are classified and given a direction label based on the direction in which the thickest tree branch among the one or more tree branches included in each learning image extends, and one or more tree branches included in each learning image are classified according to a predetermined standard. After masking, a label for the branch type may be assigned to each one or more masked branches.

The vestibular guide device may further include a sensor unit for measuring the shooting direction, and the control unit inputs the shooting direction information obtained through the sensor unit and an image containing one or more tree branches into an artificial neural network to determine the location of one or more trees included in the image. The types of branches can be analyzed.

The control unit generates shooting guide information to guide the user in the direction in which to shoot an image based on the shooting direction information, and controls the interface unit to output the shooting guide information.

The pruning guide information includes color information for each type of tree branch, and the control unit controls the interface unit to output one or more tree branches included in the image by color based on the pruning guide information.

The pruning guide information includes location information of the branch on which pruning is to be performed, and the control unit controls the interface to create a predetermined image at a specific location of the branch on which pruning is to be performed among one or more tree branches included in the image based on the pruning guide information. Can be printed.

According to one aspect, the pruning guide method includes receiving an image including one or more tree branches through a camera, and determining whether or not the one or more tree branches included in the image are pruning targets to generate pruning guide information. May include steps.

The pruning guide device provides guidance on fruit tree pruning technology based on a lot of basic knowledge and personal know-how, lowering the barrier for novice farmers to enter fruit farming such as citrus cultivation.

1 is a configuration diagram of a vestibular guide device according to an embodiment.
Figure 2 is an example diagram for explaining shooting guide information in one embodiment.
Figure 3 is an example diagram for explaining the operation of an artificial neural network according to an embodiment.
Figures 4 and 5 are exemplary diagrams for explaining a method of generating learning data according to an embodiment.
Figure 6 is an example diagram for explaining vestibular guide information according to an embodiment.
Figure 7 is a flow chart illustrating a vestibular guide method according to one embodiment.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the attached drawings. In describing the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. In addition, the terms described below are terms defined in consideration of functions in the present invention, and may vary depending on the intention or custom of the user or operator. Therefore, the definition should be made based on the content throughout this specification.

Hereinafter, embodiments of the vestibular guide device and method will be described in detail with reference to the drawings.

Figure 1 is a configuration diagram of a vestibular guide device according to an embodiment.

According to one embodiment, the vestibular guide device 100 includes a camera unit 110 for taking images, an interface unit 120 for inputting and outputting data, and a control unit connected to the camera unit 110 and the interface unit 120. It may include (130).

According to one example, the camera unit 110 may include an image sensor capable of generating image information by capturing an image. According to one example, the interface unit 120 may be a touch screen capable of outputting images and receiving touch input.

According to one embodiment, the vestibular guide device may further include a sensor unit 140 for measuring the shooting direction. As an example, the sensor unit 140 may include at least one of a global navigation system (GPS), an inertial navigation system (INS), and a geomagnetic sensor.

According to one embodiment, the control unit 130 may receive an image containing one or more tree branches through the camera unit 110. At this time, the image may be an image taken in a predetermined direction.

In the case of citrus trees, pruning should be performed to suit growth and fruiting. At this time, the main direction of the citrus tree should be set to face east, west, and north so that sunlight can shine into the inside of the tree. Accordingly, it is appropriate for pruning to be done in the direction looking from south to north. To this end, the user can create an image by photographing the tree to be pruned in the direction looking from south to north.

According to one embodiment, the control unit 130 may receive shooting direction information from the sensor unit 140 and generate shooting guide information to guide the user in the direction in which to shoot an image.

According to one example, in order to perform appropriate pruning, the direction in which sunlight shines must be taken into consideration, so it is necessary to perform pruning taking the direction into consideration. Accordingly, the control unit 130 can generate shooting guide information to guide the user to perform shooting in a predetermined direction and to perform shooting in the correct direction. For example, the shooting guide information may be a symbol or character for indicating a direction.

According to one embodiment, the control unit 130 may control the interface unit 140 to output shooting guide information. Referring to FIG. 2, the vestibular guide device 100 may output at least one of a predetermined character 141 and a predetermined symbol 142 to guide the user in the shooting direction through the interface unit 140. For example, the predetermined text may include content that guides the user to move in a predetermined direction or guides the user to take pictures at the current location. For example, a predetermined symbol may be a direction indicator indicating a predetermined direction.

According to one embodiment, the control unit 130 may determine whether one or more tree branches included in the image are pruning targets according to the type and generate pruning guide information. To this end, the control unit 130 may include an artificial neural network trained with training data generated based on one or more training images labeled with tree branch types.

Referring to FIG. 3, the control unit 130 may include an artificial neural network 131 that receives a predetermined image and analyzes the type of one or more tree branches included in the image. For example, the artificial neural network 131 may calculate the probability of the type of tree branch included in the input image.

According to one embodiment, the controller 130 may use the artificial neural network 131 to analyze the type of one or more tree branches included in an image including one or more tree branches.

According to one example, an image input to the artificial neural network 131 may include an image of one or more tree branches, and the artificial neural network 131 may perform analysis on each tree branch.

According to one example, the image input to the artificial neural network 131 may be one or more images created by editing each tree branch region from an image including one or more tree branches. For example, if the image captured by the camera unit 110 includes two tree branches, the control unit 130 may divide the area containing each tree branch to generate two images, and the divided An image of a dog can be input into the artificial neural network 131.

According to one example, the control unit 130 may determine the type with the highest probability as the type of the corresponding tree branch, based on the probability value estimated by the artificial neural network 131.

According to one embodiment, the artificial neural network may be trained with training data generated based on one or more training images labeled with tree branch types. As an example, one or more learning images may be classified and given a direction label based on the direction in which the thickest tree branch among one or more tree branches included in each learning image extends.

Referring to Figure 4(a), the learning image may include one or more tree branches, and if the branch extends upward to the right based on the thickest branch among them, it can be classified as a right branch, and if it extends upward to the left, it can be classified as a left branch. You can. At this time, if the shooting direction is from south to north, the left branch may be analyzed as a branch facing west, and the right branch may be analyzed as facing east.

According to one embodiment, in the learning images, one or more tree branches included in each learning image may be masked according to a predetermined standard, and then a label for the tree branch type may be assigned to each masked one or more tree branches. As shown in FIG. 4(b), one or more tree branches included in the learning image may be masked for each region of each tree branch, and a label for the type of tree branch may be assigned to each masked region.

Learning data can be generated through modeling using the instance segmentation algorithm, which performs localization and class discrimination step by step among the segmentation methods that predict classes in pixel units of learning images. As shown in Figure 5, the location of the desired class can be localized with respect to the learning image, and the class can be determined by determining whether each pixel contains a class in the localized bounding box. Afterwards, pixels determined as classes can be masked for each class. For example, in a training image, the tree branch class may be determined based on the pixel corresponding to the tree branch, and then the classes may be classified and determined according to the type of the tree branch. As an example, a class may be classified as a general branch or a pruning target branch.

According to one embodiment, the control unit 130 inputs the shooting direction information and the image containing one or more tree branches acquired through the sensor unit 140 into an artificial neural network to analyze the type of one or more tree branches included in the image. can do.

For example, in the case of an inward branch, it is a branch that faces in the opposite direction to the direction of the main branch in the tree, and the control unit 130 determines whether the main branch is the left or right branch based on the direction information in which the image was captured. This can be done, and branches extending in the direction opposite to the main branch can be judged as inward-facing branches.

According to one embodiment, the control unit 130 may control the interface unit 120 to output at least one of an image including one or more tree branches and pruning guide information.

Referring to FIG. 6, the control unit 130 may control the interface unit 120 to output an image acquired through the camera unit 110. In addition, the control unit 130 can control the interface unit 120 to output information about the type of tree branch included in the image acquired through the camera unit 110, whether or not it is a pruning target, and the pruning location. there is.

According to one embodiment, the pruning guide information includes color information for each type of tree branch, and the control unit 130 controls the interface unit 120 to color one or more tree branches included in the image based on the pruning guide information. It can be printed separately.

For example, as shown in FIG. 6(a), the control unit 130 outputs information about the type of tree branch using different colors for each type of tree branch along with the image acquired through the camera unit 110. Through this, the user can determine the type of the tree branch in question and whether or not to prune the tree branch.

According to one embodiment, the pruning guide information includes location information of tree branches to be pruned, and the control unit 130 controls the interface unit 120 to select one or more tree branches included in the image based on the pruning guide information. A predetermined image can be output at a specific location of a tree branch to be pruned.

For example, as shown in FIG. 6(b), the control unit 130 displays location information to perform pruning on the image acquired through the camera unit 110. Through this, the user can know at which location of which tree branch to perform pruning, and can perform pruning by referring to the information.

Figure 7 is a flow chart illustrating a vestibular guide method according to an embodiment.

According to one embodiment, the pruning guide method may include receiving an image including one or more tree branches through a camera (710).

According to one embodiment, the vestibular guide device may include a camera for taking images, an interface for inputting and outputting data, and a control unit for controlling the camera and the interface.

According to one embodiment, the vestibular guide device may further include a sensor for measuring the shooting direction. For example, the sensor may be at least one of a global navigation system (GPS), an inertial navigation system (INS), and a geomagnetic sensor.

According to one embodiment, the vestibular guide device may receive an image including one or more tree branches through a camera. At this time, the image may be an image taken in a predetermined direction.

According to one embodiment, the vestibular guide device may receive shooting direction information from a sensor and generate shooting guide information to guide the user in the direction in which to shoot an image.

According to one embodiment, the pruning guide method may include a step 720 of generating pruning guide information by determining whether one or more tree branches included in the image are pruning targets according to the type.

According to one embodiment, the pruning guide device may generate pruning guide information by determining whether one or more tree branches included in the image are pruning targets according to the type. To this end, the vestibular guide device may include an artificial neural network trained with learning data generated based on one or more learning images labeled with tree branch types.

According to one embodiment, the vestibular guide device may output an image acquired through a camera through an interface. In addition, the pruning guide device can control the interface to output information about the type of tree branch included in the image acquired through the camera, whether it is a pruning target, and the pruning location.

According to one embodiment, the pruning guide information includes color information for each type of tree branch, and the pruning guide device can output one or more tree branches included in the image by color based on the pruning guide information through the interface. .

According to one embodiment, the pruning guide information includes location information of a tree branch on which pruning is to be performed, and the pruning guide device controls the interface to select a tree branch on which pruning is to be performed among one or more tree branches included in the image based on the pruning guide information. A predetermined image can be output at a specific location of the branch.

Among the embodiments of the vestibular guide method, parts that overlap with the embodiments described with reference to FIGS. 1 to 6 are omitted.

An aspect of the present invention may be implemented as computer-readable code on a computer-readable recording medium. Codes and code segments implementing the above program can be easily deduced by a computer programmer in the art. Computer-readable recording media may include all types of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media may include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, etc. Additionally, the computer-readable recording medium may be distributed over network-connected computer systems and written and executed as computer-readable code in a distributed manner.

So far, the present invention has been examined focusing on its preferred embodiments. A person skilled in the art to which the present invention pertains will understand that the present invention may be implemented in a modified form without departing from the essential characteristics of the present invention. Accordingly, the scope of the present invention is not limited to the above-described embodiments, but should be construed to include various embodiments within the scope equivalent to the content described in the patent claims.

100: Vestibular guide device
110: Camera unit
120: Interface unit
130: control unit
140: sensor unit

Claims (9)

As a vestibular guide device,
Camera unit for taking images;
An interface unit for inputting and outputting data; and
It includes a control unit connected to the camera unit and the interface unit,
The control unit,
Receives an image containing one or more tree branches through the camera unit,
Generate pruning guide information by determining whether one or more tree branches included in the image are pruning targets according to the type,
The control unit,
It includes an artificial neural network trained with training data generated based on one or more training images labeled for tree branch types,
Analyzing the type of one or more tree branches included in the image including the one or more tree branches using the artificial neural network,
The one or more learning images are,
Among one or more tree branches included in each learning image, the thickest tree branch is classified based on the direction in which it extends and a direction label is given.
A pruning guide device that masks one or more tree branches included in each learning image based on a predetermined standard and then labels each masked tree branch for the type of tree branch. According to claim 1,
The control unit
A vestibular guide device that controls the interface unit to output at least one of an image including the one or more tree branches and the vestibular guide information. delete delete According to claim 1,
It further includes a sensor unit for measuring the shooting direction,
The control unit
A vestibular guide device that inputs the shooting direction information acquired through the sensor unit and the image including the one or more tree branches into the artificial neural network to analyze the type of the one or more tree branches included in the image. According to claim 1,
It further includes a sensor unit for measuring the shooting direction,
The control unit
Based on the shooting direction information, shooting guide information is generated to guide the user in the direction in which to shoot an image,
A vestibular guide device that controls the interface unit to output the shooting guide information. According to claim 1,
The pruning guide information includes color information for each type of tree branch,
The control unit
A pruning guide device that controls the interface unit to output one or more tree branches included in the image by color based on the pruning guide information. According to claim 1,
The pruning guide information includes tree branch location information to perform pruning,
The control unit
A pruning guide device that controls the interface to output a predetermined image at a specific location of a tree branch to be pruned among one or more tree branches included in the image based on the pruning guide information. As a vestibular guide method performed by a vestibular guide device,
Receiving an image including one or more tree branches through a camera; and
By analyzing the types of one or more tree branches included in the image using an artificial neural network, determining whether the one or more tree branches included in the image is a pruning target or not and generating pruning guide information; ,
The artificial neural network is trained with training data generated based on one or more training images labeled with tree branch types,
The one or more learning images are,
Among one or more tree branches included in each learning image, the thickest tree branch is classified based on the direction in which it extends and a direction label is given.
A pruning guide method in which one or more tree branches included in each learning image are masked based on a predetermined standard, and then a label for the tree branch type is assigned to each masked one or more branches.