KR102248654B1 - Cart-path recognition apparatus for autonomous driving of cart in golf cource and method thereof - Google Patents

Abstract

The present invention relates to an apparatus and method for recognizing a cart road for autonomous driving of a cart in a golf course. According to the present invention, in an apparatus for recognizing a cart road for autonomous driving of a cart in a golf course, a learning image captured including a cart road is used as an input value, and a correct answer image with labeling along the boundary of the cart road in the learning image is provided. As an output value, a learning unit that learns a neural network for recognizing the boundary of the cart road, an image acquisition unit that acquires a photographed image from a camera mounted on the cart, and the photographed image are input to the learned neural network Thus, it provides a cart road recognition apparatus including a control unit for recognizing the boundary of the cart road from the photographed image.
According to the present invention, it is possible to automatically recognize the boundary of the cart road by analyzing an image photographed from a cart in a golf course in real time, and to assist the autonomous driving of the cart in the golf course based on the recognition result.

Description

A cart-path recognition apparatus and method for autonomous driving of a cart in a golf course TECHNICAL FIELD OF THE INVENTION

The present invention relates to an apparatus and method for recognizing a cart road for autonomous driving of a cart in a golf course that enables autonomous driving of a cart by recognizing a boundary of a cart road from a photographed image of a cart traveling on a golf course.

As autonomous driving research has recently been activated, a large number of autonomous driving technologies using deep learning have emerged, and autonomous driving studies applied to low-speed mobile vehicles are also actively progressing as an application for this.

Cart autonomous driving technology in a golf course environment can be composed of various sensors and control modules in a cart moving at a low speed. The first step toward safe autonomous driving is to recognize and understand the environment around the subject. However, understanding the environment around the vehicle is a very difficult problem.

General vehicle roads have specified design rules and thus have the same standards or conditions in any road, whereas cart roads in the golf course environment do not have established rules or standards, and the width and curvature of the cart roads are very diverse for each golf course or course. do.

In addition, in the case of a general vehicle road, lanes are recognized through vanishing points using the thickness, color, and direction of lanes in the image. Pixels corresponding to the lanes in the image are detected, and the pixels are grouped and expressed as a single line to determine the angle of the line and If the thickness and color are similar, it is detected as a lane.

However, unlike such a vehicle road, in the case of a cart road existing in a golf course, there is no lane itself, so it is difficult to apply a lane recognition method applied to a general road to a cart road detection algorithm.

Moreover, since the curvature of the curved road of the cart road tends to be much larger than the curvature of the general vehicle road, the information on the vanishing point rather becomes an element that hinders learning, so it is not appropriate.

The technology behind the present invention is disclosed in Korean Patent Publication No. 2019-0016717 (published on February 19, 2019).

An object of the present invention is to provide an apparatus and method for recognizing a cart road for autonomous driving of a cart in a golf course capable of assisting the autonomous driving of a cart within a golf course by recognizing the boundary of a cart road from an image captured by a cart.

In the present invention, in a cart road recognition apparatus for autonomous driving of a cart in a golf course, taking a learning image captured including the cart road as an input value, and outputting a correct answer image with labeling along the boundary of the cart road in the learning image As a value, a learning unit for learning a neural network for recognizing the boundary of the cart road, an image acquisition unit for obtaining a photographed image taken from a camera mounted on the cart, and the photographed image are input to the learned neural network. , It provides a cart road recognition apparatus including a control unit for recognizing the boundary of the cart road from the photographed image.

In addition, the control unit may provide a result of recognizing the boundary of the cart road to a driving unit to allow the cart to be autonomously driven.

In addition, the cart road recognition apparatus further includes a driving information generation unit configured to generate driving information including a driving direction and speed of the cart by measuring curvature information of the cart road using the boundary of the cart road recognized in the photographed image. can do.

In addition, the learning unit includes a first correct answer image labeled with a 1×1 size pixel located at the border of the cart road in the training image, and a pixel located at the border of the cart road in the training image, and a pixel at the periphery thereof. The neural network may be trained by setting a grid of size N×N (where N is an integer equal to or greater than 8) and using a second correct answer image to which labels are given to pixels in the grid.

In addition, the neural network includes a shared layer into which one training image is input, and first and second correct answer images branched from the shared layer and labeled in different ways for the one training image, respectively, as target values. And first and second task layers, and the learning unit may train the neural network while performing multi-tasks through the first and second task layers in response to the same input image of the shared layer.

In addition, the shared layer is learned to extract a common feature for the multi-task from the same input image, and the first and second task layers are branched from the output of the shared layer, and the output of the shared layer is input. It may be learned to extract features independent from each other while following the first and second correct answer images as target values, respectively.

In addition, the learning unit, based on an error obtained by comparing the output of the first task layer derived in response to the input of the first training image and the first correct answer image, the weight applied in the shared layer and the first task After each adjustment of the weight applied in the layer, based on an error obtained by comparing the output of the second task layer derived in response to the input of the first training image and the second correct answer image, the applied in the shared layer The weight and the weight applied in the second task layer may be adjusted, and the neural network may be repeatedly trained by repeating each weight adjustment using N training images.

In addition, the control unit may select the output of the second task layer from among the outputs of the first and second task layers derived by inputting the captured image captured from the camera into the neural network and provide the result of recognizing the cartgil. .

And, the present invention, in the cart road recognition method using a cart road recognition device for autonomous driving of a cart in a golf course, taking a learning image photographed including the cart road as an input value, and determining the boundary of the cart road in the learning image. Learning a neural network for recognizing the boundary of the cart road by using the correct answer image labeled accordingly as an output value, acquiring a photographed image taken from a camera mounted on the cart, and learning the photographed image Provided is a method for recognizing a cart road, including the step of recognizing a boundary of a cart road from the photographed image by inputting it into a neural network.

In addition, the method for recognizing the cart road may further include providing a result of recognizing the boundary of the cart road to a driving unit to allow the cart to run autonomously.

In addition, the cart road recognition method may further include generating driving information including the driving direction and speed of the cart by measuring curvature information of the cart road using the boundary of the cart road recognized in the photographed image. have.

According to the present invention, an image captured from a cart in a golf course is analyzed in real time to automatically recognize the boundary of a cart road, and an advantage of assisting the autonomous driving of a cart in a golf course is provided based on the recognition result.

1 is a diagram showing a configuration of an apparatus for recognizing a cart road for autonomous driving of a cart according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a method of recognizing a cart road using FIG. 1.
3 is a diagram illustrating a method of generating two different types of correct answer images generated corresponding to one learning image in an embodiment of the present invention.
4 is a diagram illustrating first and second correct answer images generated according to an embodiment of the present invention
5 is a diagram schematically showing the structure of a neural network used in an embodiment of the present invention.
6 is a diagram showing the neural network structure of FIG. 5 in more detail.
7 is a diagram illustrating an image of a cartgil recognition result according to an embodiment of the present invention.

Then, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those of ordinary skill in the art can easily implement the present invention.

1 is a diagram showing the configuration of an apparatus for recognizing a cart road for autonomous driving of a cart according to an embodiment of the present invention.

As shown in FIG. 1, the cart road recognition apparatus 100 according to an embodiment of the present invention is a device for autonomous driving of a cart in a golf course, and includes a learning unit 110, an image acquisition unit 120, and a control unit. (130), it includes a driving information generation unit 140.

The learning unit 110 is a neural network for recognizing the boundaries of the cart road by taking the training image captured including the cart road as an input value and the correct answer image labeled along the border of the cart road in the training image as an output value. To learn.

The learning unit 110 trains the neural network using a set of a plurality of learning images and a plurality of correct answer images corresponding thereto. Also, during the learning process, model optimization is performed by adjusting the weights within the neural network. After the learning is completed, a result of recognizing the boundary of the cart road within the image may be provided in response to an input of a predetermined image.

The image acquisition unit 120 acquires a photographed image captured from a camera mounted on a cart. The image acquisition unit 120 may provide an image captured in real time while the cart is driving in a frame unit.

The controller 130 inputs the acquired photographed image to the learned neural network, recognizes the boundary of the cart road from the photographed image, and provides a recognition result.

The driving information generation unit 140 may generate driving information including a driving direction and speed of the cart by measuring curvature information of the cart road in real time using the boundary of the cart road recognized in the photographed image.

The controller 130 may support autonomous driving of the cart by transmitting the generated driving information to a driving unit (not shown) of the cart and controlling the driving unit. Here, the driving unit of the cart means driving means necessary for driving, like a general vehicle driving unit.

Here, the control unit 130 may provide a result of recognizing the boundary of the cart road in real time to the driving unit, so that the driving unit directly analyzes the curvature of the cart road and generates driving information so that the cart can be autonomously driven.

When using the technology of the present invention, it is possible to support autonomous driving not only in a state in which a person is in the cart, but also in a state in which there is no person. For example, a cart may move ahead of a person to prepare, or a person moves to the destination first on a path (grass, fairway, etc.) where it is difficult to enter the cart, and the game continues without interruption. Can move autonomously along the cart road and move to the destination where the person is located.

Next, a method for recognizing a cart road according to an embodiment of the present invention will be described in detail.

FIG. 2 is a diagram illustrating a method of recognizing a cart road using FIG. 1.

First, the learning unit 110 recognizes the boundary of the cart road by taking the training image taken including the cart road as an input value and the correct answer image labeled along the border of the cart road in the training image as an output value. The neural network for training is trained (S210).

Here, the boundary of the cart road means the boundary part (boundary surface) between the cart road (cart road) and the surrounding background (eg, grass, fairway, soil).

However, unlike general roads, since there is no separate lane on the cart road, the existing lane recognition technology of general roads based on the color and thickness of the lane is difficult to be applied to the cart road recognition. They do not recognize the contours of the road at all. This is because the cart road environment in the golf course is only divided into the cart road and the surrounding background (fairway, grass, soil, etc.).

According to an embodiment of the present invention, after collecting images obtained by photographing various types of cartgil as training data, the characteristics of pixels existing on the boundary of the cartgil within the training image are learned through a neural network, and the learned neural network is used. Based on this, the cartgil boundary is recognized from the photographed image. Here, the various forms may include not only the shape or curvature of the cart road, but also background properties (fairways, grass, soil) around the cart road, and color change of the cart road surface and background part by season or time.

According to an embodiment of the present invention, in step S210, the learning unit 110 trains a neural network using a first correct answer image and a second correct answer image created by labeling one learning image in different ways.

At this time, the first correct answer image is each labeled in 1×1 pixel units along the cartgil boundary in the training image, and the second correct answer image is N×N along the cartgil boundary within the training image. (N is an integer equal to or greater than 8) that corresponds to the labeling in grid units. When N=8, one grid includes 64 pixels.

Here, labeling may mean that '1' is allocated only to a corresponding pixel portion or pixels within a corresponding grid among all pixels constituting an image. Accordingly, each correct answer image may be configured in a form having a value of '1' only for a separately labeled portion of the image, and all of the remaining pixels having a value of '0'.

3 is a diagram illustrating a method of generating two different types of correct answer images generated corresponding to one learning image in an embodiment of the present invention.

Referring to FIG. 3, a method of generating a first correct answer image and a second correct answer image that are labeled in different ways for one and the same training image can be identified.

3(a) shows a first correct answer image created by labeling each 1×1 sized pixel located at the boundary of the cart road in the training image. The first correct answer image corresponds to an image in which '1' is assigned to a pixel located along the border of the cartgil among all pixels constituting the training image, and '0' is assigned to all other pixels.

As an example of the generation method, when a user draws a line (line) through a mouse or the like on a part corresponding to the boundary between the cart road and the background in the training image, the learning unit 110 displays' By assigning a value of 1', a first correct answer image can be created and used as an output value of a neural network. Here, depending on the thickness of the drawn line, the number of pixels overlapping the line may increase, but the line thickness is preferably set to a thickness smaller than 8 pixels, which is the width of the grid (e.g., 1 pixel, 2 pixels, 4 pixels length, etc.). Do.

Fig. 3(b) shows 64 pixels in the grid by setting a 8×8 grid including pixels (reference pixels) located at the border of the cart road and surrounding pixels in the training image. A second correct answer image created by labeling each is shown. This second correct answer image corresponds to an image in which '1' is assigned to pixels on each grid set along the border of Cartgill among all pixels in the training image, and '0' is assigned to all pixels outside the grid. .

For example, when the user draws a line corresponding to the boundary of the cart road with a mouse, etc., the learning unit 110 sets and places an 8×8 grid along the line and places the breast body in the grid. A value of '1' can be assigned to pixels, and this can be used as an output value of a neural network.

Here, as the reference pixel in the 8×8 grid, one of the four pixels located in the center may be selected. An example of selecting a pixel as a reference pixel. Of course, if the grid has an odd horizontal/vertical size of 9×9, a pixel located in the center 5 rows and 5 columns can be set as the reference pixel.

In an embodiment of the present invention, if the first correct answer image is a labeling feature of the boundary line of the cart road, the second correct answer image may correspond to labeling the boundary line and even its periphery.

In a golf course environment, there are no lanes, so the thickness, color, and direction of the lanes cannot be utilized at all, so the features of the cart road boundary are very important. Through this, the surrounding features adjacent to the boundary surface can be clearly separated from other background parts in the learning process.

In the embodiment of the present invention, the size of the grid is exemplified as an 8×8 size composed of 64 pixels, but is not limited thereto. For example, the size of the grid for generating the second correct answer image may vary depending on the resolution of the learning image used for learning, the following learning rate, and the like.

4 is a diagram illustrating first and second correct answer images generated according to an embodiment of the present invention. 4 shows image data acquired in the road environment of the Ciel Golf Club golf course in Yeongcheon, Gyeongsangbuk-do using a camera directly installed on the cart.

4(a) shows a first correct answer image. In the case of FIG. 4(a), a label is given to each pixel along the boundary of the cart road, and the labeled portion is shown in the form of a continuous line. Here, for convenience of explanation, the labeled part is indicated by a thicker line than the actual one.

In the case of Fig. 4(b), labeling is given in grid units along the boundary of the cart road, and each grid box is located along the boundary, and the outer edge of the labeled part is in the form of an irregular staircase by grid boxes. Will have.

From FIG. 4, first and second correct answer images labeled in different ways for the same training image can be confirmed. Since the two correct answer images have different labeling methods, the feature regions to be learned are also different.

In other words, in the case of the first correct answer image, the characteristics of the pixels present on the boundary surface are mainly learned by labeling the pixels of the boundary surface, and in the case of the second correct answer image, not only the pixels on the boundary surface but also the characteristics of the surrounding pixels are also included. Make it possible to learn.

According to an embodiment of the present invention, a neural network having a plurality of output values for one input value is constructed, and the above two types of correct answer images are learned together. That is, multi-tasks are performed by configuring multiple output layers in the neural network so that two types of correct answer images are learned together in one neural network.

5 is a diagram schematically showing the structure of a neural network used in an embodiment of the present invention, and FIG. 6 is a diagram showing the neural network structure of FIG. 5 in more detail.

Referring to FIG. 5, the neural network includes a shared layer 11 and first and second task layers 12 and 13 branched from an output of the shared layer 11 to a branch.

One learning image is input to the input (in) of the shared layer 11. In addition, the first and second task layers 12 and 13 perform learning by using the first and second correct answer images labeled in different ways for one learning image, respectively, as target values.

Accordingly, the learning unit 110 performs multi-tasks (first and second tasks) through the first and second task layers 12 and 13 in response to the same input image of the shared layer 11 while performing a neural network. To learn.

Here, of course, the first task refers to a task of learning the first task layer 12 so that the input image follows the first correct answer image, and the second task refers to the second task layer so that the input image follows the second correct answer image. It means the work of learning (13).

Here, the shared layer 11 corresponds to a layer shared by each other at the front end of the two task layers 12 and 13.

Accordingly, in the learning process, a plurality of layers included in the shared layer 11 are learned to extract common features for two tasks. In addition, a plurality of layers in the first task layer 12 are learned to receive the output of the shared layer 11 and extract independent features for the first task, and a plurality of layers in the second task layer 13 are shared. It is trained to receive the output of the layer 11 and extract independent features for the second task.

In this way, the shared layer 11 is learned to extract common features for multi-tasks (two tasks) from the same input image, and the first and second task layers 12 and 13 are output of the shared layer 11 It is learned to extract features independent from each other while following the first and second correct answer images as target values, respectively.

The structure of the neural network will be described in more detail with reference to FIG. 6 as follows.

Referring to FIG. 6, the shared layer 11 includes five convolution layers (conv1 to conv5) and three polling layers (pool), and is configured to learn common features.

The two numbers displayed on the front of each layer indicate the size of the image input to the layer (480 × 640 for the first layer), and the number on the side indicates the number of dimensions (for the first layer, due to RGB characteristics). 3D).

The 480×640 training image input to the shared layer 11 is learned while passing through a plurality of layers, and the number of dimensions gradually increases as the multiplication operation is performed while passing through the convolution layer in the learning process, and the image size decreases through the polling layer. do. If features are extracted while reducing the image size, the learning and processing speed of the neural network can be increased.

The output of the sixth layer in the shared layer 11 becomes an input value of the first and second task layers 12 and 13. Each of the first and second task layers 12 and 13 includes three convolutional layers (Conv6 to 8), and extracts features independent from each other in response to the first and second tasks assigned to them. .

Here, the learning process of the neural network compares the output data (out1, out2) output from each task in response to one input image with tracking data (first correct answer image, second correct answer image), and an error according to the comparison is preset. It is achieved through the process of adjusting (optimizing) each weight in the neural network so that it is below the threshold. This will be described in detail as follows.

First, the learning unit 110 performs a first task using a first learning image. That is, based on the error of comparing the output (out1) of the first task layer 12 derived in response to the input (in) of the first training image and the first correct answer image for the first training image, the shared layer ( 11) Each of the weights applied within and the weights applied within the first task layer 12 is adjusted.

Here, of course, the weight in the shared layer 11 means a weight applied between each layer of the six layers constituting the shared layer 11, and the weight in the first task layer 12 means the first task layer 12 It means the weight applied between each layer for the three layers constituting.

Thereafter, the second task is performed using the same first training image. That is, the learning unit 110 compares the output (out2) of the second task layer 13 derived in response to the input (in) of the first training image and the second correct answer image for the first training image. As a basis, the weight applied in the shared layer 11 and the weight applied in the second task layer 13 are adjusted.

Here, the shared layer 10 is optimized through weight adjustment in both cases, and the first task layer 12 and the second task layer 13 branched from the output of the shared layer 10 Weight adjustment is made intermittently.

The learning unit 110 performs learning while repeatedly performing the above-described two tasks for several learning images. That is, the neural network is repeatedly trained by repeating each weight adjustment through N training images.

Here, learning images photographed in various directions for various types of cart roads in a plurality of golf courses may be used for learning. In addition, even in the same place, the reliability of the system can be increased by using images taken by time (day, night), season, and weather, respectively, for learning.

The neural network used in the embodiment of the present invention is composed of a total of two tasks as described above with reference to FIGS. 5 and 6 and is learned at the same time. Here, according to the exemplary embodiment of the present invention, the importance λ ₂ of the second task layer 13 using the grid box may be set higher than _{the importance λ 1} of the first task layer 12.

If this is described as a concept of loss, it is expressed as in Equation 1 below.

Here, L _reg is the comparison result (difference) between the output value from the second task layer 13 using the grid box and the second correct answer image in the learning process, and L _od is the output value from the first task layer 12 And the first correct answer image.

λ _i is the importance of the i-th task and has a value between 0 and 1, respectively. In this embodiment, since the importance of the second task is high, λ ₁ <λ ₂ is set. Accordingly, when the neural network is trained, the learning result of the second task has more influence than the first task.

As such, the loss value (Loss) represents the error between the actual output value of the neural network and the target estimated value in the learning process of the neural network. In this case, in the case of the embodiment of the present invention, the loss value is composed of the sum of the losses by the first and second tasks, and learning may be performed by differently setting the importance applied to each loss.

In fact, in the embodiment of the present invention, λ ₁ =λ ₂ was initially set equally, and while learning was performed, an appropriate value was designated and finally changed in the learning process. As a result, learning ends when the verification accuracy converges to the initially set target value.

Although the above-described embodiment of the present invention illustrates two tasks, a third task layer (not shown) for detecting objects (obstacles) such as rocks, people, carts, and trees existing in the golf course is shared layer 11 It can be configured by additionally branching from, and through this, object detection and avoidance driving can be performed during autonomous driving.

After the learning is completed in this way, an image taken from an actual cart can be input into a neural network to automatically recognize the cart road.

That is, the image acquisition unit 120 acquires a photographed image from a camera mounted on a cart driving a golf course (S220). Further, the controller 130 inputs a photographed image to the previously learned neural network, recognizes the boundary of the cart road from the photographed image, and provides a recognition result (S230).

Here, the controller 130 inputs the captured image captured from the camera to the neural network, and among the outputs (out1, out2) derived, that is, the outputs of the first and second task layers 12, 13, the second task layer ( By selecting the output (out2) of 13), it can be provided as a recognition result of the cart road.

The reason is that, as a result of performing multi-task-based learning through the embodiment of the present invention, it was found that the output result of the second task layer 13 is more reliable than the output result of the first task layer 12. to be. However, the present invention is not necessarily limited thereto, and it goes without saying that the cartgil recognition result may be provided by using the output of the first task layer 12.

However, when learning is performed by excluding the first task layer 12 in the neural network structure of FIGS. 5 and 6, that is, when learning is performed using only the second correct answer image using only the second task layer 13 alone In the case of learning the first and second tasks together as a multi-task, since the performance reliability of the recognition result is lower, both tasks are necessarily used in the learning process of the actual neural network, and in the step S230 later, the second task layer ( Only the output of 13) can be used.

Thereafter, the control unit 130 may control or assist the autonomous driving of the cart by transmitting the result of recognizing the boundary of the cart road to the driving unit of the cart (S240).

Next, a result of testing the recognition performance of the cartgil recognition apparatus according to the present embodiment will be described.

Table 1 below shows the number of data acquired for the test.

Number of data Training data 1,383 Verification data 345 Test data 562

Here, the training data corresponds to image data used for training the actual neural network. Of course, since there are 1383 pieces of training data input to the neural network, 1383 pieces of first and second correct answer images required in the learning process are also used. In addition, the verification data represents image data used for verification of the neural network after learning, and the test data means data used for an actual test of the neural network for which verification has been completed.

7 is a diagram illustrating an image of a cartgil recognition result derived according to an embodiment of the present invention.

7 shows the results of Cartgill recognition derived by inputting four kinds of test data (test images) into a neural network. The performance detection method defined that when the IoU (Intersection over Union) with the ground-truth for each frame was 50% or more, the frame was detected. When the test data was applied to the neural network by using the weight of the neural network learned through the training data, the performance reached almost 97.3%.

According to the present invention as described above, there is an advantage of automatically recognizing the boundary of a cart road by analyzing an image photographed from a cart in a golf course in real time, and assisting the autonomous driving of a cart traveling in the golf course based on the recognition result.

The present invention has been described with reference to the embodiments shown in the drawings, but these are merely exemplary, and those of ordinary skill in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

100: cartgil recognition device 110: learning unit
120: image acquisition unit 130: control unit
140: driving information generation unit

Claims (16)

In the cart road recognition device for autonomous driving of a cart in a golf course,
Recognize the boundary of the cart road by taking the training image captured including the cart road as an input value, and using the first and second correct answer images that are labeled in different ways along the border of the cart road in the training image as an output value. A learning unit that trains a neural network to perform;
An image acquisition unit that acquires a photographed image taken from a camera mounted on the cart; And
And a control unit for inputting the photographed image to the learned neural network and recognizing a boundary of the cart road from the photographed image,
The first correct answer image is a labeling for each 1×1 sized pixel located at the border of the cart road in the training image, and the second correct answer image is a pixel located at the border of the cart road in the training image and Labeling is given to pixels in a grid of size N×N (N is an integer greater than or equal to 8) including peripheral pixels,
The neural network,
The training image is input and a shared layer that is learned to extract a common feature for a multi-task at a later stage from one input image, and an output of the shared layer, respectively, are input, and the first and second correct answer images are respectively set to target values. It includes first and second task layers that are learned to extract features that are independent from each other while following with,
The control unit,
Cartgil recognition apparatus for providing the output of the second task layer among the outputs of the first and second task layers derived by inputting the captured image to a neural network as a result of recognition of the cartgil. The method according to claim 1,
The control unit,
A cart road recognition device for autonomously driving a cart by providing the result of recognizing the boundary of the cart road to a driving unit. The method according to claim 1,
Cart road recognition apparatus further comprising a driving information generating unit for generating driving information including the driving direction and speed of the cart by measuring the curvature information of the cart road using the boundary of the cart road recognized in the photographed image. delete delete delete The method according to claim 1,
The learning unit,
A weight applied in the shared layer and a weight applied in the first task layer are determined based on an error obtained by comparing the output of the first task layer derived in response to the input of the first training image and the first correct answer image. After adjusting each,
A weight applied in the shared layer and a weight applied in the second task layer based on an error obtained by comparing the output of the second task layer derived in response to the input of the first training image and the second correct answer image But adjust
Cartgill recognition apparatus for repeatedly learning the neural network by repeating each weight adjustment using N training images. delete In the cart way recognition method using a cart way recognition device for autonomous driving of a cart in a golf course,
Recognize the boundary of the cart road by taking the training image taken including the cart road as an input value, and using the first and second correct answer images that are labeled in different ways along the border of the cart road in the training image as an output value. Training a neural network to perform;
Obtaining a photographed image taken from a camera mounted on the cart; And
And inputting the photographed image to the learned neural network, and recognizing a boundary of the cart road from the photographed image,
The first correct answer image is a labeling for each 1×1 sized pixel located at the border of the cart road in the training image, and the second correct answer image is a pixel located at the border of the cart road in the training image and Labeling is given to pixels in a grid of size N×N (N is an integer greater than or equal to 8) including peripheral pixels,
The neural network,
The learning image is input and a shared layer that is learned to extract a common feature for a subsequent multi-task from one input image, and an output of the shared layer, respectively, are received, and the first and second correct answer images are respectively set to target values. It includes first and second task layers that are learned to extract features that are independent from each other while following with,
Recognizing the boundary of the cart road,
A cartgil recognition method for providing the output of the second task layer among outputs of the first and second task layers derived by inputting the captured image to a neural network as a recognition result of the cartgil. The method of claim 9,
And providing a result of recognizing the boundary of the cart road to a driving unit to allow the cart to run autonomously. The method of claim 9,
And generating driving information including a driving direction and speed of the cart by measuring curvature information of the cart road using the recognized cart road boundary in the photographed image. delete delete delete The method of claim 9,
The step of training the neural network,
Adjust the weight of each of the shared layers and the weight of the first task layer based on an error obtained by comparing the output of the first task layer derived in response to the input of the first training image and the first correct answer image. next,
Based on an error obtained by comparing the output of the second task layer derived in response to the input of the first training image and the second correct answer image, the weights of each of the shared layers and the weight of the second task layer are calculated. But adjust,
Cartgill recognition method for repeatedly learning the neural network by repeating each weight adjustment using N training images. delete

Images