KR20180104213A - Autonomous navigation ship controlling apparatus using ensemble artificial neural network combination and method therefor - Google Patents

Abstract

An autonomous navigation ship controlling apparatus using an ensemble artificial neural network combination and a method thereof are disclosed. According to an embodiment of the present invention, the autonomous navigation ship controlling apparatus includes: a sensor part having one or more sensors; an estimating part which estimates an operable region for each of the one or more sensors based on sensing information outputted from each of the at least one sensor and a neural network model for each of the one or more sensors; and a control part for controlling a self-operated ship based on the estimated operable region for each of the one or more sensors.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an autonomous navigation vessel control apparatus using an ensemble artificial neural network combination and an autonomous navigation vessel control apparatus using the same,

The present invention relates to an autonomous navigation ship control technology, and more particularly, to an apparatus and method for controlling an autonomous navigation ship using an ensemble artificial neural network combination.

The existing autonomous navigation ship has a complicated system including an obstacle recognition module for recognizing an obstacle, a module for position recognition, and a controller for controlling the ship in order to avoid an obstacle by itself and follow up to a target point or a way point.

Among the existing systems, the most important module for high level autonomy is the obstacle recognition module, which estimates the obstacle by the conventional judgment algorithm using the camera which outputs the shape information, the expensive sensor such as LiDAR, RADAR and the like.

Here, since the conventional judgment algorithm is determined by using the characteristic specified by the human, the estimation performance becomes different when the sensor operating environment or the driving state deviates from the measurement range assumed by the person. This increases the cost of the overall system, such as reducing the reliability of the entire autonomous navigation system and requiring additional assistance recognition systems to prevent accidents such as obstacle avoidance failures, depending on the environment or situation. Especially in poor marine environments.

In the present invention, an artificial neural network for estimating a current state and a next control state is constructed by inputting sensor information, and an autonomous navigation vessel control apparatus and method for outputting a highly reliable control signal by ensemble combination of an artificial neural network designed for each sensor .

Embodiments of the present invention provide an apparatus and method for controlling autonomous vessels using an ensemble artificial neural network combination.

An autonomous ship control apparatus according to an embodiment of the present invention includes: a sensor unit having at least one sensor; An estimating unit for estimating an operable region for each of the at least one sensor based on sensing information output from each of the at least one sensor and a neural network model for each of the at least one sensor; And a control unit for controlling the self-operated vessel based on the estimated operable area for each of the at least one or more sensors.

The estimator may estimate the operable region for each of the at least one or more sensors based on sensing information output from each of the at least one sensor and a neural network model based on a Convolutional Neural Network (CNN).

The control unit may control the autonomous navigation vessel using the estimated operational area for each of the at least one sensor and the recurrent neural network (RNN).

The control unit may determine the operating area in the operable area based on the operable area and the recurrent neural network, and may control the rudder or speed of the self-operated ship to the determined operating area.

The estimating unit may perform predetermined preprocessing on sensing information output from each of the at least one sensor, and estimate the operable region using the sensed information and the neural network model.

An autonomous ship control method according to an embodiment of the present invention includes: receiving sensing information output from each of at least one sensor; Estimating an operable region for each of the at least one sensor based on sensed information output from each of the at least one sensor and the neural network model for each of the at least one sensor; And controlling the autonomously-operated vessel based on the operable region for each of the at least one or more estimated sensors.

The estimating step may estimate the operable area for each of the at least one or more sensors based on sensing information output from each of the at least one sensor and a neural network model based on a convolutional neural network (CNN) .

The controlling step may control the autonomous navigation vessel using the estimated operational area for each of the at least one sensor and the recurrent neural network (RNN).

The controlling step may determine the operating area in the operable area based on the operable area and the recirculating neural network, and control the rudder or speed of the autonomous operating vessel to the determined operating area.

The estimating step may perform predetermined preprocessing on sensing information output from each of the at least one or more sensors, and estimate the operable area using the sensed information and the neural network model.

According to the embodiments of the present invention, by controlling an autonomous navigation ship using an ensemble artificial neural network combination, it is possible to reduce the number of sensors for controlling an autonomous navigation ship, thereby reducing system construction cost, It can provide robust performance.

1 is a block diagram illustrating a configuration of an autonomous ship control apparatus according to an embodiment of the present invention.
FIG. 2 is a view for explaining an autonomous ship control apparatus shown in FIG. 1. FIG.
FIG. 3 shows an example for explaining the estimation unit shown in FIG. 1. FIG.
FIG. 4 is a diagram illustrating an example of the control unit shown in FIG. 1. Referring to FIG.
FIG. 5 shows another example for explaining the autonomous ship control apparatus shown in FIG. 1. FIG.
6 is a flowchart illustrating an operation of an autonomously operating ship control method according to an embodiment of the present invention.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to or limited by the embodiments. In addition, the same reference numerals shown in the drawings denote the same members.

Embodiments of the present invention can reduce the number of sensors for controlling an autonomous ship by controlling an autonomous navigation ship using an ensemble artificial neural network combination to reduce system construction cost, To provide a robust performance of the system.

Here, the ensemble artificial neural network combination may mean a combination of information output from a neural network for each sensor, for example, a neural network such as a Convolutional Neural Network (CNN) or an RNN. That is, assuming that two sensors are used in the autonomous navigation ship control apparatus according to the present invention, by combining the first information and the second information output from the neural network for the first sensor and the neural network for the second sensor, It can mean to control the ship. Of course, the meaning of the ensemble artificial neural network combination in the present invention is not limited to the above description, but may include all neural network combinations usable in the apparatus and method according to the present invention.

In addition, the artificial neural network may mean a neural network such as CNN and RNN. The artificial neural network concept is obvious to those skilled in the art, and a detailed description thereof will be omitted.

1 is a block diagram illustrating a configuration of an autonomous ship control apparatus according to an embodiment of the present invention.

Referring to FIG. 1, an autonomous ship control apparatus 100 according to an embodiment of the present invention includes a sensor unit 110, an estimating unit 120, and a controller 130. Of course, the autonomous ship control apparatus according to the present invention may include all the basic arrangements for controlling the autonomous ship, though not shown in FIG.

The sensor unit 110 includes at least one sensor for sensing an area for autonomous navigation of the autonomous navigation ship.

Here, the sensor unit 110 may include an image sensor, a radar, a lidar, and the like for photographing a sea in which an autonomous navigation ship is operating, and the number and types of sensors constituting the sensor unit are applied to the present invention And can be divided into an essential sensor and an optional sensor, and the optional sensor can be added or selected in consideration of the performance improvement and the cost aspect according to the present invention.

That is, the sensor unit 110 includes sensing information for controlling the autonomous navigation vessel while configuring the autonomous navigation vessel control apparatus, for example, information for estimating or detecting the sea region for autonomous navigation of the self- For example, it senses ocean information and provides sensed sensing information.

At this time, the sensor unit 110 can independently provide the sensing information sensed by each sensor, and the sensing information thus provided can be input to each neural network model for each sensor defined or learned.

The estimating unit 120 receives sensing information output from each of at least one or more sensors provided in the sensor unit 110 and generates sensing information output from each of the at least one or more sensors and a neural network And estimates the operable area for each of the at least one or more sensors based on the model.

Here, the neural network model for each of the sensors may be a neural network model based on a Convolutional Neural Network (CNN), and the sensor data collected in various environments such as time, weather condition, The neural network model. That is, the estimating unit 120 receives the sensing information of the corresponding sensor as an input to the neural network model previously learned by each sensor, and outputs the operational region in the sea, which is the output information of the neural network model for the sensing information, .

In this way, the estimating unit 120 estimates the operable area free from obstacles in the route from the current position of the sea (or river) to the destination or the transit point using the neural network model of the sensor and the sensing information of the sensor Therefore, only the sensing information about the sea is required regardless of the types of obstacles existing on the sea, and the learning is performed through the deep learning only for the sea area, so that the amount of data for learning can be reduced, And the number of sensors can be reduced. That is, even if an obstacle appears on the sea, the apparatus according to the present invention estimates an area capable of autonomous navigation from the current location to the next location on the route through learning about the sea, There is no need to learn about.

Further, the estimator 120 may perform a preprocessing process on the sensing information to accurately estimate the operational area (or area) from the sensing information sensed by the sensor, thereby making it possible to clarify the sensed sea information. For example, when the sensed information is a sea image, the estimating unit 120 prepares a part of the sea image that is changed according to light reflection, time, weather, etc. to obtain a clearer sea image, And the neural network model of the sensor can be used to estimate the autonomous operationable area. Of course, the estimating unit is not limited to the above-described method of pre-processing the sensing information. The estimating unit may be configured to preprocess the main data portion for determining whether the region is the operable region, for example, It is also possible to exclude data parts which are difficult to judge.

That is, the estimating unit 120 may estimate the operable region for each sensor by pre-processing the sensing information and then using the pre-processed sensing information and the neural network model for each sensor.

The control unit 130 controls the autonomously operated vessel based on the operable region for each of the at least one sensor estimated by the estimating unit 120.

At this time, the control unit 130 can control the self-operated vessel using the operational area for each of the estimated at least one sensor and the recurrent neural network (RNN).

For example, the control unit 130 determines the operation area to be operated by the self-operated vessel in the operable area based on the operable area and the recurrent neural network for each of the estimated at least one sensor, You can control the rudder or speed of the ship in operation.

The autonomous ship control apparatus according to the present invention will be described in detail with reference to FIGS. 2 to 5. FIG.

FIG. 2 is a view for explaining an autonomous ship control apparatus shown in FIG. 1. FIG.

As shown in FIG. 2, the autonomous ship control device senses information about the sea from at least one sensor provided in the sensor unit 110 and outputs sensing information (T, T-1, T-2) , The estimating unit 120 searches the available area in each of the sensors using the CNN-based neural network model and the sensing information T, T-1, and T-2, Estimate the operational area.

At this time, the estimating unit 120 estimates the operable area for each sensor by searching for the obstacle-free operable area through the CNN neural network model based on the sensing information for each sensor, as shown in FIG. 3A controllable space).

The neural network model shown in FIG. 3A is a CNN-based neural network model for classifying the operable region for each sensor, and may be a neural network model in which sensing information is input and the operable region is output.

FIG. 3B illustrates a CNN-based neural network model used in the present invention. The neural network model is a learned neural network model in which sensing data collected in various environments are input to each sensor. CNN Learning (neural network model) can be designed (or defined).

The control unit 130 generates a control signal for autonomous navigation of the self-operated vessel to the final operation region through the RNN for the operational region for each of the sensors estimated from the CNN-based neural network model, Operate the flight area autonomously.

Herein, the controller 130 combines the current location information and the information on the operable area with the RNN, and combines the information on the operable area through the RNN. Thus, And to control the rudder or speed (e.g., motor speed) for autonomous navigation to the determined final operating area.

In this case, as shown in FIG. 4A, the control unit 130 determines whether or not the current position information of the self-operated vessel (for example, GPS information, IMU The control signal for controlling the autonomous vessel is generated by using the neural network model with input of measured information (pose).

The neural network model used in the control part may be the neural network model shown in FIG. 4B. As shown in FIG. 4B, the neural network model of the control part inputs the operational areas collected by each sensor in various environments, It may be a neural network model learned by the RNN. That is, the neural network model of the control unit may be a neural network model for outputting an optimal control value for autonomous navigation to a target point (for example, a final operating area) using the CNN output change and the ship control state according to time. Of course, if necessary, the neural network model of the controller may be a neural network model based on RNN.

In this case, the neural network model of the control unit can be composed of the learning data and the neural network structure for outputting the optimum control value regardless of the autonomous navigation ship platform.

FIG. 5 shows another example for explaining the autonomous ship control apparatus shown in FIG. 1. FIG.

Referring to FIG. 5, the autonomous ship control apparatus may be composed of two parts, an end-to-end artificial neural network model and an ensemble control module 530.

The end-to-end artificial neural network model can be divided into a sensor module 510 composed of sensors and a part 520 composed of neural network models of each sensor. The neural network model that estimates the control state based on information (or sensing information) is learned for each sensor.

For example, the end-to-end artificial neural network model can be learned by sensor data collected in various environments based on CNN. It can independently estimate the rudder of autonomous vessel based on sensor information, can do.

The ensemble control module 530 is a constituent means for outputting the final rudder and speed by combining the control state values estimated for each sensor so as to autonomously operate robustly according to various environments and situations. Can be used.

Here, the ensemble control module 530 deduces the most reliable result based on the estimated result for each of the plurality of sensors (or the single sensor), and outputs the control signal robustly against malfunction or mis-estimation of some sensors .

The neural network model used in the present invention may have various structures, but the output part of the neural network uses a feature vector value output from one inner product layer. This can be achieved by constructing a fully connected layer with multiple layers at the time of neural network learning, and then using the output values of the updated layer at the beginning or at the middle level. This allows robust control The feature value can be output.

In addition, a coordinate conversion operation is performed to fit the dimension of the feature vector output from each sensor before the support vector regression. Assuming Gaussian or normal distribution using various environmental data not used for learning, normalization Can be performed. This can control the scale of the feature vectors output from the heterogeneous sensor so as not to be biased by one sensor.

In the present invention, since only the sea region is required to learn, expensive sensors are not necessarily required and can be implemented as a simple device or system. Since learning is performed based on deep learning based on the sea region, learning complexity and computational complexity Can be reduced.

Further, since only the sea area is learned, the present invention does not affect the self-driving regardless of any obstacles appearing in the sea, and the amount of learning data can be reduced because only the sea is learned.

FIG. 6 is a flowchart illustrating an operation of the autonomous navigation ship control method according to an embodiment of the present invention, and shows an operation flow chart in the apparatuses of FIGS. 1 to 5 described above.

Referring to FIG. 6, an autonomous navigation ship control method according to an embodiment of the present invention includes receiving sensed sensing information from at least one sensor, sensing information of at least one sensor received, (S610, S620) based on the model.

Here, the step S620 may use a CNN-based neural network model for each of the sensors, and the CNN-based neural network model may include sensor data collected in various environments such as time, weather conditions, Lt; / RTI > neural network model. That is, in step S620, sensing information of the corresponding sensor is received as an input to the neural network model previously learned by each sensor, and an output of the neural network model corresponding to the sensing information is provided as an output .

Further, in step S620, in order to accurately estimate the operable area (or area) from the sensing information sensed by the sensor, a preprocessing process of the sensing information is performed and then the preprocessed sensing information is provided as an input of the neural network model, It is possible to estimate the operable area.

If it is estimated in step S620 that the operable area for each of the sensors is estimated, a control signal for controlling the self-operated vessel is generated based on the estimated operable area, and the controlled vessel is controlled using the generated control signal S630).

Here, in step S630, the self-operated vessel can autonomously operate the self-operated vessel to the final operation region by generating a control signal for autonomous navigation to the final operation region through the RNN for the operational region for each of the sensors.

Specifically, in step S630, information on the current position of the autonomous vessel and information on the operable area are input to the RNN-based neural network model, and the information on the operable area is combined through the RNN-based neural network model. It is possible to determine the final operating area to be autonomously operated at the present position of the area and to control the rudder or speed (for example, motor speed) for autonomous operation to the final operating area determined.

In addition, the neural network model in step S630 may be a neural network model for outputting an optimal control value for autonomous navigation to a target point (for example, a final operation area) using CNN output change and vessel control state according to time have.

Although a detailed description is omitted in the method of FIG. 6, it is obvious to those skilled in the art that the above-described contents of FIGS. 1 to 5 may be all included.

The system or apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the systems, devices, and components described in the embodiments may be implemented in various forms such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array ), A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to embodiments may be implemented in the form of a program instruction that may be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (10)

A sensor unit having at least one sensor;
An estimating unit for estimating an operable region for each of the at least one sensor based on sensing information output from each of the at least one sensor and a neural network model for each of the at least one sensor; And
A control unit for controlling the self-operated vessel based on the estimated operable area for each of the at least one sensor,
And a control unit for controlling the self-propelled vessel.
The method according to claim 1,
The estimation unit
Wherein the estimating means estimates the operable region for each of the at least one sensor based on sensed information output from each of the at least one sensor and a neural network model based on a convolutional neural network (CNN) controller.
The method according to claim 1,
The control unit
Wherein the autonomous navigation vessel is controlled using an operational area for each of the estimated at least one sensor and a recurrent neural network (RNN).
The method of claim 3,
The control unit
Wherein the control unit determines the operating area in the operable area based on the operable area and the recirculating neural network and controls the rudder or speed of the self-operated ship to the determined operating area.
The method according to claim 1,
The estimation unit
Wherein the pre-processing unit performs predetermined preprocessing on sensing information output from each of the at least one sensor, and estimates the operable area using the sensed information and the neural network model.
Receiving sensing information output from each of at least one sensor;
Estimating an operable region for each of the at least one sensor based on sensed information output from each of the at least one sensor and the neural network model for each of the at least one sensor; And
Controlling the self-operated vessel based on the estimated operable region for each of the at least one sensor
And a control unit for controlling the autonomous vessel.
The method according to claim 6,
The estimating step
Wherein the estimating means estimates the operable region for each of the at least one sensor based on sensed information output from each of the at least one sensor and a neural network model based on a convolutional neural network (CNN) Control method.
The method according to claim 6,
The step of controlling
Wherein the autonomous navigation vessel is controlled using the estimated operational area for each of the at least one sensor and the recurrent neural network (RNN).
9. The method of claim 8,
The step of controlling
Wherein the control unit determines the operating area in the operable area based on the operable area and the recirculating neural network and controls the rudder or speed of the autonomous operating vessel to the determined operating area.
The method according to claim 6,
The estimating step
A pre-set pre-processing of sensing information output from each of the at least one sensor, and estimating an operable area using the sensed information and the neural network model.

Images