KR102640791B1 - System for digitalizing onboard voice - Google Patents

Abstract

The present invention provides an on-board voice digitization system and method capable of converting conversations within a bridge, wireless communications received through a VHF device, etc. into text, and a computer program for executing the method on a computer readable recording medium. It's about. The on-board voice digitization system according to an embodiment of the present invention forms a first audio signal using the contents of a conversation inside the on-board bridge, and forms a second audio signal using the contents of wireless communication when communicating with the outside of the ship, The first and second voice signals are converted into text data, and the text data is used to form navigation information necessary for ship operation.

Description

Onboard voice digitization system{SYSTEM FOR DIGITALIZING ONBOARD VOICE}

The present invention relates to a voice digitization system and method on board a ship, and a computer-readable recording medium on which a computer program for executing the method is recorded on a computer. More specifically, it relates to a conversation inside a bridge and a wireless signal received through a VHF device. It relates to a shipboard voice digitization system and method that can convert communications, etc. into text, and a computer-readable recording medium on which a computer program for executing the method on a computer is recorded.

An artificial intelligence (AI) system is a computer system that implements human-level intelligence, and unlike existing rule-based smart systems, it is a system in which machines learn and make decisions on their own. As artificial intelligence systems are used, their recognition rates improve and they can more accurately understand user preferences, and existing rule-based smart systems are gradually being replaced by deep learning-based artificial intelligence systems.

Artificial intelligence technology consists of machine learning and element technologies using machine learning. Machine learning is an algorithmic technology that classifies/learns the characteristics of input data on its own, and elemental technology is a technology that mimics the functions of the human brain such as cognition and judgment by utilizing machine learning algorithms such as deep learning, including linguistic understanding and visual It consists of technical areas such as understanding, reasoning/prediction, knowledge expression, and motion control.

The various fields where artificial intelligence technology is applied are as follows. Linguistic understanding is a technology that recognizes and applies/processes human language/characters and includes natural language processing, machine translation, conversation systems, question and answer, and voice recognition/synthesis. Visual understanding is a technology that recognizes and processes objects like human vision, and includes object recognition, object tracking, image search, person recognition, scene understanding, spatial understanding, and image improvement. Inferential prediction is a technology that judges information to make logical inferences and predictions, and includes knowledge/probability-based reasoning, optimization prediction, preference-based planning, and recommendations. Knowledge expression is a technology that automatically processes human experience information into knowledge data, and includes knowledge construction (data creation/classification) and knowledge management (data utilization). Motion control is a technology that controls the autonomous driving of vehicles and the movement of robots, and includes motion control (navigation, collision, driving), operation control (behavior control), etc.

Generally, in order to apply machine learning algorithms to real life, learning is performed using a trial and error method due to the nature of the basic methodology of machine learning. In particular, deep learning requires hundreds of thousands of iterations. It is impossible to execute this in an actual physical external environment, so instead, the actual physical external environment is virtually implemented on a computer and learning is performed through simulation.

On the other hand, if one of the key information related to the operation of a ship, such as conversations inside the bridge and contents of wireless communications received through VHF (Very High Frequency) devices, is stored in the form of an audio file, it cannot be stored for a long time due to the enormous capacity, and it cannot be stored for a long time, and it cannot be stored for a long time due to the enormous capacity, There is a problem that land transmission using a communication network is impossible.

Therefore, there is a need to convert the contents of conversations within the bridge or wireless communications received through VHF devices into relatively small text files, and there is a technical requirement for automatically converting audio files into text through an artificial intelligence system. is increasing.

Korean Patent Publication No. 10-0568621 (Ship voyage record automatic reporting system and method, announced on April 7, 2006)

The present invention provides an on-board voice digitization system and method that can convert conversations within a bridge, wireless communications received through a VHF device, etc. into text, and a computer-readable record in which a computer program for executing the method on a computer is recorded. Provides media.

An onboard voice digitization system according to an embodiment of the present invention includes one or more microphones that form a first voice signal using the contents of a conversation inside the onboard bridge; One or more VHF devices that form a second voice signal using wireless communication contents when communicating with the outside of the ship; a text conversion device that converts the first voice signal and the second voice signal into text data; and an autonomous navigation platform that uses the text data to form navigation information necessary for operation of the vessel.

In addition, it may further include a communication unit that transmits the text data to the outside of the ship.

Additionally, the text conversion device may be formed integrally with a Voyage Data Recorder (VDR).

Additionally, the text conversion device may be provided separately from a Voyage Data Recorder (VDR).

In addition, the text conversion device can learn by applying artificial intelligence technology and an audio input unit that receives the first voice signal and the second voice signal, and can learn the input of the first voice signal and the second voice signal. It may include a voice recognition AI that forms text information, and a text converter that forms the text data with the formed text information.

In addition, the voice recognition AI acquires training voice signals, extracts training text objects from the training voice signals, acquires first labels that are word information corresponding to the training text objects, and acquires the training text objects. can be applied to a neural network to generate training outputs corresponding to the training text objects, and to learn the neural network based on the training outputs and the first labels.

In addition, it may further include an autonomous navigation control device that controls autonomous navigation of the ship using the navigation information generated by the autonomous navigation platform.

In addition, it may further include a land control center that receives the text data through the communication unit.

Meanwhile, an onboard voice digitization method according to another embodiment of the present invention includes forming a first voice signal using the contents of a conversation inside the onboard bridge; Forming a second voice signal using wireless communication contents when communicating with the outside of the ship; Converting the first voice signal and the second voice signal into text data; and forming navigation information necessary for operation of the vessel using the text data.

Additionally, after converting the text data into text data, the method may further include transmitting the text data to the outside of the ship.

In addition, the step of converting the first voice signal and the second voice signal into text data includes receiving the first voice signal and the second voice signal, and artificial intelligence technology can be applied to learn the input. It may include forming text information of the received first voice signal and the second voice signal, and forming the text data using the formed text information.

In addition, learning to which the artificial intelligence technology is applied includes obtaining training voice signals, extracting training text objects from the training voice signals, and obtaining first labels that are word information corresponding to the training text objects. A step of applying the training text objects to a neural network to generate training outputs corresponding to the training text objects, and training the neural network based on the training outputs and the first labels. It can be performed including.

In addition, after the step of forming navigation information necessary for operation of the vessel, the step of controlling autonomous navigation of the vessel using the navigation information may be further included.

Additionally, transmitting the text data to the outside of the ship may include transmitting the text data to a land control center.

Meanwhile, a computer-readable recording medium according to another embodiment of the present invention records a computer program for executing the above-described onboard voice digitization method on a computer.

According to embodiments of the present invention, the voice can be stored for a long period of time by converting it into a relatively small text file, and can be transmitted to a control center on land in real time or periodically through a satellite, etc., and in the case of autonomous ships, Communication contents can be provided to a server related to the operation of a ship to assist in the operation of the ship.

In addition, according to embodiments of the present invention, the contents of conversations and communications can be easily checked at any time anywhere on or off the ship through the Smart Ship platform, which can be useful for ship maintenance.

Figure 1 is an exemplary diagram showing the configuration of an onboard voice digitization system according to an embodiment of the present invention.
Figure 2 is an exemplary diagram showing the configuration of an onboard voice digitization system according to another embodiment of the present invention.
Figure 3 is an exemplary diagram showing the configuration of a text conversion device according to an embodiment of the present invention.
Figure 4 is a diagram for explaining machine learning of a neural network according to an embodiment of the present invention.
Figure 5 is a flowchart showing the procedure of the onboard voice digitization method according to an embodiment of the present invention.

Embodiments of the present invention are illustrated for the purpose of explaining the technical idea of the present invention. The scope of rights according to the present invention is not limited to the embodiments presented below or the specific description of these embodiments.

Hereinafter, embodiments of the present invention will be described with reference to the attached drawings. In the accompanying drawings, identical or corresponding components are given the same reference numerals. Additionally, in the description of the following embodiments, overlapping descriptions of identical or corresponding components may be omitted. However, even if descriptions of components are omitted, it is not intended that such components are not included in any embodiment.

Figure 1 is an exemplary diagram showing the configuration of an onboard voice digitization system according to an embodiment of the present invention.

As shown in Figure 1, the on-board voice digitization system 100 includes one or more microphones (110-1, 110-2, 110-3,..., 110-n) and one or more Very High Frequency (VHF) devices ( 120-1, 120-2), text conversion device (130), autonomous navigation platform (140), autonomous navigation control device (150), land control center (160), cable (170), communication department and database. there is. According to one embodiment, the text conversion device 130 may include an audio input unit 132, a voice recognition artificial intelligence (AI) 134, and a text conversion unit 136. For example, one or more microphones (110-1, 110-2, 110-3,…, 110-n), one or more Very High Frequency (VHF) devices (120-1, 120-2), a text conversion device ( 130), autonomous navigation platform 140, autonomous navigation control device 150, land control center 160, cable 170, communication unit, and database may be connected through an on-board network to enable mutual communication.

One or more microphones (110-1, 110-2, 110-3,..., 110-n) may form a first voice signal using the content of the conversation inside the bridge within the ship. A bridge or bridge is a space within a single ship that can control the entire ship. According to one embodiment, one or more microphones 110-1, 110-2, 110-3,..., 110-n may be installed in various locations to record conversations inside the bridge within the ship.

One or more VHF (Very High Frequency) devices (120-1, 120-2) can form a second voice signal using wireless communication contents when communicating with the outside of the ship. According to one embodiment, the VHF devices 120-1 and 120-2 may perform wireless communication with the outside of the ship using microwave signals. The VHF devices (120-1, 120-2) can record wireless communication content when communicating with the outside of the ship and form a second voice signal.

The text conversion device 130 can convert the first and second voice signals into text data. According to one embodiment, the text conversion device 130 may be included in an existing Voyage Data Recorder (VDR). In other words, it can be a next-generation navigation recording device in which the function of the text conversion device 130 is added to the existing navigation recording device. The navigation recorder stores related data such as ship position, speed, course, bridge worker's voice, communicator voice, radar data, water depth, rudder operation history, engine use history, wind direction, wind speed, and automatic vessel identification system (AIS). In an emergency, it operates with the ship's emergency power, and when the ship's emergency power is cut off, the voyage recording device can continuously record bridge communications using spare power (storage batteries) for a certain period of time. These navigation recorders could be designed to automatically rise to sea level if a ship sinks.

The audio input unit 132 includes a first voice signal formed by one or more microphones (110-1, 110-2, 110-3,..., 110-n) and one or more Very High Frequency (VHF) devices (120-1, The second voice signal formed in 120-2) can be input through the cable 170.

Voice recognition AI (Artificial Intelligence) 134 can form text information in real time by applying artificial intelligence technology to the first and second voice signals. According to one embodiment, the voice recognition AI 134 acquires training voice signals, extracts training text objects from the training voice signals, acquires first labels that are word information corresponding to the training text objects, and trains. Text objects can be applied to a neural network to generate training outputs corresponding to the training text objects, and the neural network can be learned based on the training outputs and first labels.

The text conversion unit 136 can form text data using text information generated by the voice recognition AI 134.

The autonomous navigation platform 140 can use text data generated by the text conversion device 130 to form navigation information necessary for the operation of the ship.

The autonomous navigation control device 150 can control the autonomous navigation of the ship using navigation information generated by the autonomous navigation platform 140.

The land control center 160 can receive text data periodically or aperiodically through the ship's communication unit and store it.

The communication unit may transmit text data generated in the text conversion device 130 to the outside of the ship.

A database can store a variety of data. Data stored in the database is data acquired, processed, or used by at least one component of the onboard voice digitization system 100 and may include software (eg, a program). A database may include volatile and/or non-volatile memory. As an example, the database may store first and second voice signals, text data, navigation information, etc.

In the present invention, the program is software stored in a database, and includes one or more microphones (110-1, 110-2, 110-3,..., 110-n), one or more VHF (Very High Frequency) devices (120-1, 120-2), text conversion device 130, autonomous navigation platform 140, autonomous navigation control device 150, land control center 160, cable 170, operating system for controlling resources of communication department and database, Application and/or one or more microphones (110-1, 110-2, 110-3,…, 110-n), one or more Very High Frequency (VHF) devices (120-1, 120-2), text conversion devices ( 130), autonomous navigation platform (140), autonomous navigation control device (150), land control center (160), cable (170), middleware that provides various functions to the application to utilize the resources of the communication department and database, etc. It can be included.

In the present invention, artificial intelligence (AI) refers to technology that imitates human learning ability, reasoning ability, perception ability, etc. and implements this with a computer, and may include concepts such as machine learning and symbolic logic. there is. Machine Learning (ML) is an algorithmic technology that classifies or learns the characteristics of input data on its own. Artificial intelligence technology is a machine learning algorithm that analyzes input data, learns the results of the analysis, and makes judgments or predictions based on the results of the learning. Additionally, technologies that mimic the functions of the human brain, such as cognition and judgment, using machine learning algorithms can also be understood as the category of artificial intelligence. For example, technical fields such as verbal understanding, visual understanding, reasoning/prediction, knowledge representation, and motion control may be included.

Machine learning can refer to the process of training a neural network model using experience processing data. Machine learning can mean that computer software improves its own data processing capabilities. A neural network model is built by modeling the correlation between data, and the correlation can be expressed by a plurality of parameters. A neural network model extracts and analyzes features from given data to derive correlations between data. Repeating this process to optimize the parameters of the neural network model can be called machine learning. For example, a neural network model can learn the mapping (correlation) between input and output for data given as input-output pairs. Alternatively, a neural network model may learn the relationships by deriving regularities between given data even when only input data is given.

An artificial intelligence learning model or neural network model may be designed to implement the human brain structure on a computer, and may include a plurality of network nodes with weights that simulate neurons of a human neural network. A plurality of network nodes may have a connection relationship with each other by simulating the synaptic activity of neurons in which neurons exchange signals through synapses. In an artificial intelligence learning model, multiple network nodes are located in layers of different depths and can exchange data according to convolutional connection relationships. The artificial intelligence learning model may be, for example, an artificial neural network model (Artificial Neural Network), a convolution neural network (CNN) model, etc. As an example, an artificial intelligence learning model may be machine-learned according to methods such as supervised learning, unsupervised learning, and reinforcement learning. Machine learning algorithms for performing machine learning include Decision Tree, Bayesian Network, Support Vector Machine, Artificial Neural Network, Adaboost, and Perceptron. Perceptron, genetic programming, clustering, etc. can be used.

Among them, CNN is a type of multilayer perceptrons designed to use minimal preprocessing. CNN consists of one or several convolution layers and general artificial neural network layers on top of them, and additionally utilizes weights and pooling layers. Thanks to this structure, CNN can fully utilize input data with a two-dimensional structure. Compared to other deep learning structures, CNN shows good performance in both video and audio fields. CNNs can also be trained via standard back propagation. CNNs have the advantage of being easier to train and using fewer parameters than other feedforward artificial neural network techniques.

Convolutional networks are neural networks that contain sets of nodes with bound parameters. The increasing size of available training data and the availability of computational power, combined with algorithmic advances such as piecewise linear unit and dropout training, have led to significant improvements in many computer vision tasks. For extremely large data sets, such as those available for many tasks today, overfitting is not critical, and increasing the size of the network improves test accuracy. Optimal use of computing resources becomes a limiting factor. For this purpose, distributed, scalable implementations of deep neural networks can be used.

Figure 2 is an exemplary diagram showing the configuration of an onboard voice digitization system according to another embodiment of the present invention.

As shown in FIG. 2, the on-board voice digitization system 200 includes one or more microphones (210-1, 210-2, 210-3,..., 210-n) and one or more VHF devices (220-1, 220). -2), including text conversion device (230), navigation recorder (240), autonomous navigation platform (250), land control center (260), autonomous navigation control device (270), cable (280), communication department and database can do. According to one embodiment, the text conversion device 230 may include an audio input unit 232, a voice recognition AI 234, and a text conversion unit 236. For example, one or more microphones (210-1, 210-2, 210-3,…, 210-n), one or more VHF devices (220-1, 220-2), text converter (230), navigation log The device 240, autonomous navigation platform 250, land control center 260, autonomous navigation control device 270, cable 280, communication unit, and database may be connected through an onboard network to enable mutual communication.

One or more microphones (210-1, 210-2, 210-3,..., 210-n) may form a first voice signal using the content of the conversation inside the bridge within the ship.

One or more VHF devices (220-1, 220-2) may form a second voice signal using wireless communication contents when communicating with the outside of the ship.

The text conversion device 230 can convert the first and second voice signals into text data. According to this embodiment, the text conversion device 230 may be provided separately from the navigation recording device 240.

The audio input unit 232 receives a first voice signal formed from one or more microphones (210-1, 210-2, 210-3,..., 210-n) and one or more VHF devices (220-1, 220-2). The formed second voice signal can be input through the cable 280.

The voice recognition AI 234 can form text information in real time by applying artificial intelligence technology to the first and second voice signals.

The text conversion unit 236 can form text data using text information generated by the voice recognition AI 234.

The navigation recording device 240 records a first voice signal formed from one or more microphones (210-1, 210-2, 210-3,..., 210-n) and one or more VHF devices (220-1, 220-2). The formed second voice signal can be input through the cable 280 and stored.

The autonomous navigation platform 250 can use text data generated by the text conversion device 230 to form navigation information necessary for the operation of the ship.

The land control center 260 can receive text data periodically or aperiodically through the ship's communication unit and store it.

The autonomous navigation control device 270 can control the autonomous navigation of the ship using navigation information generated by the autonomous navigation platform 250.

The communication unit may transmit text data generated in the text conversion device 230 to the outside of the ship.

A database can store a variety of data. Data stored in the database is data acquired, processed, or used by at least one component of the onboard voice digitization system 200 and may include software (eg, a program). A database may include volatile and/or non-volatile memory. As an example, the database may store first and second voice signals, text data, navigation information, etc.

In the present invention, the program is software stored in a database, and includes one or more microphones (210-1, 210-2, 210-3,..., 210-n), one or more Very High Frequency (VHF) devices (220-1, 220-2), text conversion device (230), navigation recorder (240), autonomous navigation platform (250), land control center (260), autonomous navigation control device (270), cable (280), communication department and database. An operating system, an application, and/or one or more microphones (210-1, 210-2, 210-3,..., 210-n), one or more Very High Frequency (VHF) devices (220-1, 220-) for controlling resources. 2), text conversion device 230, navigation recorder 240, autonomous navigation platform 250, land control center 260, autonomous navigation control device 270, cable 280, communication department and database resources It may include middleware that provides various functions to the application for use.

Figure 3 is an exemplary diagram showing the configuration of a text conversion device according to an embodiment of the present invention.

As shown in FIG. 3, the text conversion devices 130 and 230 may include one or more processors 131, one or more memories 133, and/or transceivers 135. In one embodiment, at least one of these components of the text conversion device 130, 230 may be omitted, or another component may be added to the text conversion device 130. Additionally or alternatively, some components may be integrated and implemented, or may be implemented as a single or plural entity. At least some of the components inside and outside the text conversion devices 130 and 230 are configured through a bus, general purpose input/output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI). They are connected to each other and can exchange data and/or signals.

One or more processors 131 may control at least one component of the text conversion devices 130 and 230 connected to the processor 131 by running software (eg, commands, programs, etc.). Here, at least one component of the text conversion devices 130 and 230 may be an audio input unit 132, a voice recognition AI 134, and a text conversion unit 136. Additionally, the processor 131 can perform various operations related to the present invention, such as calculation, processing, data generation, and processing. Additionally, the processor 131 may load data, etc. from one or more memories 133 or store them in one or more memories 133 .

As described above, one or more processors 131, one or more microphones 110-1, 110-2, 110-3,..., 110-n) 210-1, 210-2, 210-3, ..., 210-n), receives the first voice signal in the form of a digital packet in real time or non-real time, and one or more VHF (Very High Frequency) devices (120-1, 120-2) (220- 1, 220-2), the second voice signal can be received in real time or non-real time in the form of a digital packet. According to one embodiment, the received first and second voice signals may be stored in the memory 133 in the form of digital packets.

One or more processors 131 may convert the first and second voice signals received through the transceiver 135 into text data. According to one embodiment, the processor 131 acquires training voice signals, extracts training text objects from the training voice signals, acquires first labels that are word information corresponding to the training text objects, and creates training text objects. can be applied to a neural network to generate training outputs corresponding to training text objects, and to learn the neural network based on the training outputs and first labels.

One or more processors 131 may control the formed text data to be transmitted to the autonomous navigation platforms 140 and 250 through the transceiver 135.

One or more memories 133 may store various data. Data stored in the memory 133 is data acquired, processed, or used by at least one component of the text conversion device 130, 230, and may include software (e.g., commands, programs, etc.). there is. Memory 133 may include volatile and/or non-volatile memory. In the present invention, a command or program is software stored in the memory 133, and an operating system, application, and/or application for controlling the resources of the text conversion devices 130 and 230 are used to control the resources of the text conversion devices 130 and 230. It may include middleware that provides various functions to the application so that it can be utilized.

One or more memories 133 may store the above-described first and second voice signals, text data, navigation information, etc. Additionally, one or more memories 133 may store instructions that allow one or more processors 131 to perform operations when executed by one or more processors 131 .

As an example, the text conversion devices 130 and 230 may further include a transceiver 135. The transceiver 135 includes one or more microphones (110-1, 110-2, 110-3,..., 110-n) (210-1, 210-2, 210-3,..., 210-n), one or more Very High Frequency (VHF) devices (120-1, 120-2) (220-1, 220-2), text conversion devices (130, 230), autonomous navigation platforms (140, 250), autonomous navigation control devices (150) , 270), wireless or wired communication may be performed between the land control centers 160 and 260, a communication unit, a database, and/or other devices. For example, the transceiver 135 may support enhanced Mobile Broadband (eMBB), Ultra Reliable Low-Latency Communications (URLLC), Massive Machine Type Communications (MMTC), long-term evolution (LTE), LTE Advance (LTE-A), UMTS (Universal Mobile Telecommunications System), GSM (Global System for Mobile communications), CDMA (code division multiple access), WCDMA (wideband CDMA), WiBro (Wireless Broadband), WiFi (wireless fidelity), Bluetooth, NFC ( Wireless communication can be performed using methods such as near field communication, GPS (Global Positioning System), or GNSS (global navigation satellite system). For example, the transceiver 135 can perform wired communication according to a method such as universal serial bus (USB), high definition multimedia interface (HDMI), recommended standard232 (RS-232), or plain old telephone service (POTS). there is.

As an embodiment, one or more processors 131 control the transceiver 135 to transmit one or more microphones 110-1, 110-2, 110-3,..., 110-n) 210-1, 210-2, 210-3, …, 210-n), one or more Very High Frequency (VHF) devices (120-1, 120-2) (220-1, 220-2), text conversion devices (130, 230), autonomous navigation Information can be obtained from the platform (140, 250), autonomous navigation control device (150, 270), land control center (160, 260), and communication department. One or more microphones (110-1, 110-2, 110-3,…, 110-n) (210-1, 210-2, 210-3, …, 210-n), one or more Very High Frequency (VHF) Devices (120-1, 120-2) (220-1, 220-2), text conversion devices (130, 230), autonomous navigation platforms (140, 250), autonomous navigation control devices (150, 270), land control Information obtained from the centers 160 and 260 and the communication unit may be stored in one or more memories 133.

As an example, the text conversion devices 130 and 230 may be of various types. For example, the text conversion devices 130 and 230 may be portable communication devices, computer devices, or a combination of one or more of the foregoing devices. The text conversion devices 130 and 230 of the present invention are not limited to the devices described above.

Various embodiments of the text conversion devices 130 and 230 according to the present invention may be combined with each other. Each embodiment can be combined depending on the number of cases, and embodiments of the text conversion devices 130 and 230 created by combining them also fall within the scope of the present invention. Additionally, the internal and external components of the text conversion devices 130 and 230 according to the present invention described above may be added, changed, replaced, or deleted depending on the embodiment. Additionally, the internal and external components of the text conversion devices 130 and 230 described above may be implemented as hardware components.

Figure 4 is a diagram for explaining machine learning of a neural network according to an embodiment of the present invention.

As shown in FIG. 4, the learning device can train the neural network 138 to form text data from voice signals collected within the ship. According to one embodiment, the learning device may be configured integrally with the voice recognition AIs 134 and 234 of the text conversion devices 130 and 230, but the learning device is not limited thereto and may be configured separately.

The neural network 138 includes an input layer 137 through which training samples are input and an output layer 139 through which training outputs are output, and can be learned based on the difference between training outputs and labels. Here, labels may be defined based on body part information corresponding to the feature point object. The neural network 138 is connected to a group of a plurality of nodes, and is defined by weights between connected nodes and an activation function that activates the nodes.

The learning device may learn the neural network 138 using a gradient descent (GD) technique or a stochastic gradient descent (SGD) technique. The learning device can use a loss function designed by the outputs and labels of the neural network.

The learning device can calculate the training error using a predefined loss function. The loss function may be predefined with a label, output, and parameter as input variables, where the parameter may be set by weights in the neural network 138. For example, the loss function may be designed in the form of MSE (Mean Square Error), entropy, etc., and various techniques or methods may be employed in embodiments in which the loss function is designed.

The learning device can use the backpropagation technique to find weights that affect the training error. Here, the weights are relationships between nodes in the neural network 138. The learning device can use the SGD technique using labels and outputs to optimize the weights found through the backpropagation technique. For example, the learning device can update the weights of the loss function defined based on the labels, outputs, and weights using the SGD technique.

According to one embodiment, the learning device may acquire training voice signals and extract training text objects from the training voice signals. The learning device may obtain pre-labeled information (first labels) for each training text object, and may obtain first labels indicating word information predefined in the training text objects.

According to one embodiment, the learning device may generate first training feature vectors based on appearance features, pattern features, and color features of training voice signals. Various methods can be employed to extract features of training voice signals.

According to one embodiment, the learning device may obtain training outputs by applying the first training feature vectors to the neural network 138. The learning device may train the neural network 138 based on the training outputs and the first labels. The learning device may learn the neural network 138 by calculating training errors corresponding to the training outputs and optimizing the connection relationships of nodes within the neural network 138 to minimize the training errors. The text conversion devices 130 and 230 may form text data from the first and second voice signals using the learned neural network 138.

Figure 5 is a flowchart showing the procedure of the onboard voice digitization method according to an embodiment of the present invention. Although the process steps, method steps, algorithms, etc. are described in the flow chart of FIG. 5 in a sequential order, such processes, methods, and algorithms may be configured to operate in any suitable order. In other words, the steps of the processes, methods and algorithms described in various embodiments of the invention do not need to be performed in the order described herein. Additionally, although some steps are described as being performed asynchronously, in other embodiments, some such steps may be performed concurrently. Additionally, the illustration of a process by depiction in the drawings does not mean that the illustrated process excludes other variations and modifications thereto, and that any of the illustrated process or steps thereof may be incorporated into any of the various embodiments of the invention. It does not imply that more than one is required, nor does it imply that the illustrated process is preferred.

As shown in Figure 5, in step S510, for example, referring to Figures 1 to 4, one or more microphones 110-1, 110-2, 110-3,..., 110-n) ( 210-1, 210-2, 210-3, ..., 210-n) can form the first voice signal using the content of the conversation inside the bridge within the ship.

In step S520, a second voice signal is formed. For example, referring to Figures 1 to 4, one or more VHF (Very High Frequency) devices (120-1, 120-2) (220-1, 220-2) transmit wireless communication contents when communicating with the outside of the ship. A second voice signal can be formed using .

In step S530, it is converted into text data. For example, referring to FIGS. 1 to 4 , the text conversion devices 130 and 230 may convert the first and second voice signals input through the cables 170 and 280 into text data.

In step S540, navigation information is formed. For example, referring to FIGS. 1 to 4 , the autonomous navigation platform 140 and 250 may use text data generated by the text conversion devices 130 and 230 to form navigation information necessary for the operation of the ship.

Although the method has been described through specific embodiments, the method can also be implemented as computer-readable code on a computer-readable recording medium. Computer-readable recording media include all types of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. Additionally, computer-readable recording media can be distributed across computer systems connected to a network, so that computer-readable code can be stored and executed in a distributed manner. And, functional programs, codes, and code segments for implementing the above embodiments can be easily deduced by programmers in the technical field to which the present invention pertains.

Above, the present invention has been described with reference to the embodiments shown in the drawings. However, the present invention is not limited thereto, and various modifications or other embodiments within the scope equivalent to the present invention can be made by those skilled in the art. Therefore, the true scope of protection of the present invention will be determined by the following claims.

100,200: Onboard voice digitization system 110,210: Microphone
120,220: VHF device 130,230: Text conversion device
140,250: Autonomous navigation platform 150,270: Autonomous navigation control device
160,260: Land control center 170,280: Cable
131: Processor 132,232: Audio input unit
133: Memory 134,234: Voice recognition AI
135: Transceiver 136,236: Text conversion unit
137: input layer 138: neural network 139: output layer

Claims (15)

One or more microphones that form a first voice signal using conversation content inside the bridge within the ship;
One or more VHF devices that form a second voice signal using wireless communication contents when communicating with the outside of the ship;
a text conversion device that converts the first voice signal and the second voice signal into text data and stores it; and
It includes an autonomous navigation platform that uses the text data to form navigation information necessary for operation of the vessel,
It further includes a communication unit that transmits the text data to the outside of the ship,
The text conversion device,
It is separately equipped with a Voyage Data Recorder (VDR) that stores the first voice signal and the second voice signal in the form of an audio file,
The text conversion device,
An audio input unit that receives the first voice signal and the second voice signal,
A voice recognition AI that can be learned by applying artificial intelligence technology and forms text information of the received first voice signal and the second voice signal, and
A text conversion unit that forms the text data using the formed text information,
Further comprising a land control center that receives the text data through the communication unit,
The text conversion device that converts and stores the first voice signal and the second voice signal into the text data, and the navigation recording device that stores the first voice signal and the second voice signal in the form of an audio file are separate. Characterized in that only the relatively low-capacity text data is transmitted to the land control center in real time or periodically through the communication unit,
Onboard voice digitization system. delete delete delete delete According to claim 1,
The voice recognition AI is,
Obtain training voice signals, extract training text objects from the training voice signals, obtain first labels that are word information corresponding to the training text objects, and apply the training text objects to a neural network, generating training outputs corresponding to training text objects, and training the neural network based on the training outputs and the first labels,
Onboard voice digitization system. According to claim 1,
Further comprising an autonomous navigation control device that controls autonomous navigation of the ship using the navigation information generated in the autonomous navigation platform,
Onboard voice digitization system. delete delete delete delete delete delete delete delete