KR20240053154A - Speech recognition media playback device and method - Google Patents

Abstract

The present invention recognizes the environment in a work environment and voices from which work noise has been removed, performs control operations such as media playback, and controls media by voice without manipulating electronic devices by hand.

Description

Voice recognition media playback device and method {SPEECH RECOGNITION MEDIA PLAYBACK DEVICE AND METHOD}

The present invention relates to a voice recognition media playback device and method. More specifically, a voice recognition media playback device that allows a user to enjoy media even in a noisy environment and allows the user to watch media while working while holding a tool with his or her hand. It's about method.

Conventional technologies related to the present invention include surveillance systems, spoken voice recognition, and voice interaction. Patent Document 1: A surveillance system and surveillance method equipped with an automatic detection/photography function transmits images in 3D normal state, 3D zoom-in state, and 3D zoom-out state. Additionally, Patent Document 2: System and method for accessing multimedia content connects to one or more portions of multimedia content. In addition, Patent Document 3, an artificial intelligence device and method for recognizing a user's spoken voice, recognizes a user's spoken voice. In addition, Patent Document 4's voice interaction method and vehicle using the same change whether or not there is voice interaction between the vehicle and passengers and the voice interaction settings.

However, the prior art has the problem of not being able to recognize the environment in the work environment and voices from which work noise has been removed, and perform control operations such as media playback.

Registered Patent Publication No. 10-0997497 Surveillance system and surveillance method with automatic detection/photography function Public Patent Publication No. 10-2014-0108180 System and method for accessing multimedia content Registered Patent Publication No. 10-2281602 Artificial intelligence device and method for recognizing user's spoken voice Registered Patent Publication No. 10-2330978 Voice interaction method and vehicle using the same

The purpose of the present invention is to provide a voice recognition media playback device and method that recognizes voices from which environmental and work noises have been removed in a work environment and performs control operations such as media playback.

Another object of the present invention is to provide a voice recognition media playback device and method for controlling media by voice without manipulating electronic devices by hand.

A voice recognition media playback device and method according to a preferred embodiment of the present invention includes a sound and voice input unit 100 that receives user voice data including ambient noise through a voice recognition system included in a terminal 6; a speaker-independent voice recognition unit 200 that removes environmental and operational noise from user voice data and acquires the speaker's voice; A speaker-dependent voice recognition unit 300 that identifies the speaker's voice from the original data based on the speaker's voice data learned in advance; a speaker voice-to-text conversion unit 400 that converts speaker voice data extracted by the speaker-independent voice recognition unit and the speaker-dependent voice recognition unit 300 into text data; a user intention inference unit 500 that extracts user speech intention and components through an artificial intelligence model based on text data converted by the speaker voice-to-text conversion unit 400; An intention command connection unit 600 that connects the extracted speech intention with a computer command capable of controlling media; And a media playback unit 700 that plays the video and controls the video through a media control command.

In addition, the speaker-independent speech recognition unit 200 listens to environmental noise, learns characteristic elements of the heard sound based on the environmental noise storage 212, and a deep neural network learns characteristic elements of the environmental noise, An environmental noise removal unit 210 that removes environmental noise from user voice data based on the learning results and stores the learning results in the environmental noise storage 212; And listen to the work noise, learn the characteristic elements of the heard sound based on the work noise storage 222, the deep neural network learns the feature elements of the work noise, and based on the learning results, the environmental noise is primarily removed. It is characterized in that it includes a work noise removal unit 220 that removes work noise from voice data and stores the learning results in the work noise storage 222.

In addition, the speaker-dependent voice recognition unit 300 extracts the voiceprint, which is an individual's unique phonetic characteristic, from the original data, analyzes it through statistical methods, and separates the voices of each speaker in a situation where multiple speakers speak. (310); When the voices of multiple speakers are detected at the same time, a first speaker recognition unit 320 recognizes and extracts only the voice of the first speaker based on the data analyzed by the voiceprint recognition unit 310; and a speaker voice storage 330 that stores characteristic elements of an existing registered user voice.

In addition, the user intention inference unit 500, if the data has previously been extracted based on text data converted from voice data, immediately stores the utterance intention in the user intention storage 520 without going through an artificial intelligence model, A user intention management unit (510) that imports components to reduce unnecessary artificial intelligence model calculations, and stores the utterance intent and components extracted through the artificial intelligence model in the user intent storage (520) in the case of data that has no previously extracted history. ); and a user intent storage 520 that stores speech intent and components extracted through an artificial intelligence model.

In addition, the intent command connection unit 600 is characterized by including a media control command storage 610 that stores pairs of media control commands corresponding to each utterance intention.

The present invention recognizes the work environment and voices with work noise removed, and performs control operations such as media playback, allowing users to enjoy media even in noisy environments.

Additionally, the present invention controls media by voice without manipulating electronic devices with hands, allowing users to enjoy media while working while holding tools with their hands.

Figure 1 is an example diagram showing a working environment.
Figure 2 is a block diagram showing the configuration of the voice recognition media playback device of the present invention.
Figure 3 is a flowchart showing the operation of the voice recognition media playback method of the present invention.
Figure 4 is an example diagram illustrating a configuration for verifying data errors to explain the present invention.
Figure 5 is an example diagram illustrating the hardware resources, operating system, operation of the core control unit, and system authentication configuration that grants authority to execute the control unit operation to explain the present invention.

Hereinafter, a voice recognition media playback device and method according to an embodiment of the present invention will be described in detail with reference to the drawings. Below, descriptions of previously known matters are omitted or simplified to clarify the gist of the present invention. The components included in the description of the present invention operate individually or in combination.

Referring to FIG. 1 as an exemplary diagram showing a working environment, a user 21 holds a tool 22 with his hand and performs woodworking or steel work in the working environment 20. The voice recognition media playback device and method of the present invention, which allows the user 21 to control media by voice in this work environment 20, will be described.

Figure 2 is a block diagram showing the configuration of the voice recognition media playback device of the present invention. Referring to Figure 2, the voice recognition media playback device includes a sound and voice input unit 100, a speaker independent voice recognition unit 200, and an environmental noise suppressor. Rejection (210), environmental noise collection unit (211), environmental noise storage (212), work noise removal unit (220), work noise collection unit (221), work noise storage (222), speaker-dependent voice recognition unit (300) ), voiceprint recognition unit 310, initial speaker recognition unit 320, speaker voice storage 330, speaker voice-to-text conversion (400), user intention inference unit 500, user intention management unit 510, user intention storage It includes (520), an intent command connection unit (600), a media control command storage (610), and a media playback unit (700).

The terminal 6 operates as a voice recognition media playback device.

The sound and voice input unit 100 receives user voice data including ambient noise through a voice recognition system included in the terminal 6.

The speaker-independent voice recognition unit 200 removes environmental noise and work noise from user voice data and acquires the speaker's voice. The environmental noise removal unit 210 listens to environmental noise and learns characteristic elements of the heard sound based on the environmental noise storage 212. A deep neural network learns the characteristic elements of environmental noise, removes environmental noise from user voice data based on the learning results, and stores the learning results in the environmental noise storage 212. The work noise removal unit 220 listens to work noise and learns characteristic elements of the heard sound based on the work noise storage 222. The deep neural network learns the characteristic elements of task noise, removes task noise from voice data from which environmental noise has been primarily removed based on the learning results, and stores the learning results in the task noise storage 222.

The speaker-dependent voice recognition unit 300 identifies the speaker's voice from the original data based on pre-learned speaker voice data, and uses the voiceprint recognition unit 310, the initial speaker recognition unit 320, and the speaker voice storage 330. Includes. The voiceprint recognition unit 310 extracts the voiceprint, which is an individual's unique vocal characteristic, from the original data, analyzes it through statistical methods, and separates the voices of each speaker in a situation where multiple speakers speak.

When the voices of multiple speakers are detected simultaneously, the first speaker recognition unit 320 recognizes and extracts only the voice of the first speaker based on the data analyzed by the voiceprint recognition unit 310. The speaker voice storage 330 stores characteristic elements of existing registered user voices.

The speaker voice-to-text conversion unit 400 converts the speaker voice data extracted by the speaker-independent voice recognition unit and the speaker-dependent voice recognition unit 300 into text data.

The user intention inference unit 500 extracts the user speech intention and components through an artificial intelligence model based on the text data converted by the speaker voice-to-text conversion unit 400, and the user intention management unit 510 and the user intention storage ( 520).

Based on the text data converted from voice data, the user intention management unit 510 directly retrieves the utterance intention and components stored in the user intention storage 520 without going through the artificial intelligence model if the data has a history of being previously extracted. Reduce unnecessary artificial intelligence model calculations. Additionally, in the case of data that has no previous extraction history, the user intention management unit 510 stores the utterance intention and components extracted through the artificial intelligence model in the user intention storage 520.

The user intention storage 520 stores speech intention and components extracted through an artificial intelligence model.

The intent command connection unit 600 connects the extracted speech intention with a computer command capable of controlling media, and includes a media control command storage 610. The media control command storage 610 stores pairs of media control commands corresponding to each utterance intention. For example, the play utterance intent is paired with a play command.

The media playback unit 700 plays the video and controls the video through media control commands.

Figure 3 is a flowchart showing the operation of the voice recognition media playback method of the present invention. Referring to Figure 3, the control unit 5 performs a voice input step (S201), a noise removal step (S202), a voice recognition step (S203), and a recognition step. The result storage step (S204) and the intention inference step (S205) are performed.

In the voice input step (S201), user voice data including noise is input, in the noise removal step (S202), noise is removed from the user voice data, and in the voice recognition step (S203), the voice data with the noise removed is converted into text data. Recognized, the recognition result storage step (S204) stores the recognized text data, and the intention inference step (S205) converts the intent of the text data into a media control command and controls the media according to the media control command.

Figure 4 is an exemplary diagram illustrating a configuration for verifying data errors for illustrating the present invention.

Referring to FIG. 4, the control unit 5 stores sampling data, calculates a probability distribution by accumulating the number of occurrences for each size of the sampling data for a certain time, calculates a probability distribution for another certain time, and calculates the two probabilities. By calculating the distribution difference, area difference, and difference distance accumulation (S101), sampling circuit abnormalities, data errors, and data changes can be predicted and responded to (S102). The control unit 5 notifies the user of the prediction result so that the user can respond, or the control unit 5 can respond to hardware failure, data error, or data change.

Sampling data includes user voice data, environmental noise, work noise, and text data, and the control unit 5 responds to hardware failure, data error, and data change based on the sampling data.

The control unit 5 monitors the trend of each probability distribution for a certain period of time, predicts abnormalities in the probability distribution, responds to abnormal accidents, and reports normal operation to the outside if the data change in the probability distribution is constant. Additionally, the control unit 5 feeds back the data change rate to adjust a certain time interval. For example, if the data change rate is large, the certain time interval is increased, and if the data change rate is small, the certain time interval is shortened.

FIG. 5 is an exemplary diagram illustrating hardware resources and an operating system for explaining the present invention, operations of a control unit as a core, and a system authentication configuration that grants authority to execute control unit operations. Referring to FIG. 5, the present invention is a processor (1). ), memory (2), input/output device (3), operating system (4), and control unit (5).

The processor (1) is a CPU (Central Processing Unit), GPU (Graphic Processing Unit), FPGA (Field Programmable Gate Array), and NPU (Neural Processing Unit), and the operating system (4) and control unit (5) mounted on the memory (2) ) executes the execution code.

Memory (2) includes permanent mass storage devices such as random access memory (RAM), read only memory (ROM), disk drives, solid state drives (SSD), and flash memory. can do.

The input/output device 3 is an input device, such as a camera, keyboard, microphone, mouse, etc. including an audio sensor and/or image sensor, and an output device such as a display, speaker, haptic feedback device, etc. May include devices.

The operating system 4 may include Windows, Linux, IOS, virtual machines, web browsers, and interpreters, and supports tasks, threads, timer execution, scheduling, resource management, graphics, font processing, communication, etc.

The control unit 5 determines the state based on sensor, key, touch, and mouse input of the input/output device 3 with the support of the operating system 4 and performs operations according to the determined state. The control unit 5 performs job scheduling by timers and threads using parallel execution routines.

The control unit 5 determines the state using the sensor value of the input/output device 3 and performs an algorithm according to the determined state.

Referring to Figure 5, the system authentication configuration includes a terminal 6 including a control unit 5, and an authentication server 7.

The terminal 6 duplicates the data channel, receives the key value and biometric information of the terminal 6, and requests user authentication through the first data channel to the authentication server 7, and the terminal 6 receives the generated kit value. It is displayed on the display and transmitted to the authentication server (7).

The terminal 6 inputs the kit value displayed on the display of the terminal 6 and transmits it along with the user information to the authentication server 7 through the second data channel. The terminal 6 requests the authentication server 7 to authenticate the system mounted on the terminal 6 using the kit value and user information. The kit value of the terminal 6 can be generated from computer-specific information such as the CPU manufacturing number and the Ethernet chip number. The terminal 6 can obtain user information through face recognition using a camera, voice recognition using a microphone, and handwriting recognition using a display, and use it for authentication.

The authentication server 7 receives the kit value from the terminal 6, receives the kit value and user information from the terminal 6 through a duplicated data channel, compares the kit value and the user information of the terminal 6, and By matching, authentication for use of the system of the terminal 6 is processed. The authentication server 7 transmits the authentication result to the terminal 6 to authorize the user's use of the system. Due to the dual data channels of the terminal 6, kit value loss can be minimized.

The authentication server 7 performs history analysis of user information and compares and determines consistency and changes in user information over time. In history analysis, if user information shows consistency, the user's use is permitted; if it shows changes, the user's use is not permitted. By allowing users to use the system when user information shows consistency, security is strengthened to prevent users with altered user information from accessing the system.

The authentication server 7 assigns weights to consistency, change, frequency, frequency trend, and high frequency, and blocks access by untrusted users using a combination of weights. For example, if the frequency threshold is exceeded, access by untrusted users can be blocked in proportion to the accumulated number of excesses, and users who attempt access over a long period of time can be authenticated. At this time, additional authentication is requested for untrusted users.

The terminal 6, which is a means of authenticating the use of the system, does not connect directly to the system, but forms a bypass route through the authentication server 7, so that the network that makes up the Internet network is composed of an internal network and an external network, and the IP address setting process There is an advantage that the authentication process using the terminal 6 is performed smoothly in this cumbersome time. At this time, the system is mounted on the terminal 6, the terminal 6 becomes an authentication terminal means, and the authentication server 7 becomes an authentication server means.

The cloud (12) is a modularization of the container (7) with the support of the operating system (4) that manages the processor (1), memory (2), input/output device (3), and communication unit (6), and the web (8) and DB ( 9), provides the services of the protocol 10 and library 11, and the control unit 5 executes a cloud application using the services of the container 7. A standard software package, called a container (7), bundles an application's code with associated configuration files, libraries (11), and dependencies needed to run the app.

The cloud 12 integrates control of multiple terminals 6, stores sensor values received from the terminal 6, monitors them over time, processes operation errors of the terminal 6, and sends error messages to other terminals. Notifies the terminal 6, and performs switching control on the terminal 6 that is the control target.

Neural network learning selects features from time series data collected from input devices such as temperature, altitude, fingerprints, various sensors, images, infrared cameras, and lidar, selects a model through algorithm selection, and repeats through the learning and performance verification process. Model selection is repeated through trial and error. After performance verification is completed, an artificial intelligence model is selected.

The control unit 5 performs a deep learning algorithm using a neural network to determine sensor values, uses training data to learn the neural network, and verifies the neural network performance with test data.

The present invention is not limited to the specific preferred embodiments described above, and various modifications can be made by anyone skilled in the art without departing from the gist of the invention as claimed in the claims. Of course, such changes are within the scope of the claims.

1: processor
2: memory
3: Input/output device
4: Operating system
5: Control unit
6: Terminal
7: Authentication server
8: web
9: DB
11: Library
12: Cloud
14: Container
16: Department of Communications
100: Sound and voice input unit
200: Speaker independent voice recognition unit
210: Environmental noise removal unit
211: Environmental noise collection department
212: Environmental noise storage
220: Work noise removal unit
221: Work noise collection unit
222: Work noise storage
300: Speaker-dependent voice recognition unit
310: Voiceprint recognition unit
320: First speaker recognition unit
330: Talker voice storage
400: Speaker speech-to-text conversion
500: User intent inference unit
510: User intent management unit
520: User intent store
600: Intent command connection part
610: Media control command storage
700: Media playback unit

Claims (5)

a sound and voice input unit 100 that receives user voice data including ambient noise through a voice recognition system included in the terminal 6;
a speaker-independent voice recognition unit 200 that removes environmental and operational noise from user voice data and acquires the speaker's voice;
A speaker-dependent voice recognition unit 300 that identifies the speaker's voice from the original data based on the speaker's voice data learned in advance;
A speaker voice-to-text conversion unit 400 that converts the speaker voice data extracted by the speaker-independent voice recognition unit 200 and the speaker-dependent voice recognition unit 300 into text data;
a user intention inference unit 500 that extracts user speech intention and components through an artificial intelligence model based on text data converted by the speaker voice-to-text conversion unit 400;
An intention command connection unit 600 that connects the extracted speech intention with a computer command capable of controlling media; and
A voice recognition media playback device and method comprising a media playback unit 700 that plays video and controls the video through media control commands. According to paragraph 1,
The speaker independent voice recognition unit 200,
Listen to environmental noise, learn feature elements of the heard sound based on the environmental noise storage 212, a deep neural network learns feature elements of the environmental noise, remove environmental noise from user voice data based on the learning results, and , an environmental noise removal unit 210 that stores learning results in the environmental noise storage 212; and
Listen to work noise, learn feature elements of the heard sound based on the work noise storage 222, deep neural network learn feature elements of work noise, and produce a voice with environmental noise primarily removed based on the learning results. A voice recognition media playback device and method comprising a task noise removal unit (220) that removes task noise from data and stores learning results in the task noise storage (222). According to paragraph 1,
The speaker-dependent voice recognition unit 300,
a voiceprint recognition unit 310 that extracts voiceprints, which are individual unique vocal characteristics, from the original data, analyzes them through statistical methods, and separates the voices of each speaker in a situation where multiple speakers speak;
When the voices of multiple speakers are detected at the same time, a first speaker recognition unit 320 that recognizes and extracts only the voice of the first speaker based on the data analyzed by the voiceprint recognition unit 310; and
A voice recognition media playback device and method comprising a speaker voice storage 330 that stores characteristic elements of an existing registered user voice. According to paragraph 1,
The user intention inference unit 500,
Based on text data converted from voice data, if the data has a history of being previously extracted, the utterance intention and components stored in the user intention storage 520 are immediately imported without going through the artificial intelligence model to reduce unnecessary artificial intelligence model calculations. , a user intent management unit 510 that stores utterance intent and components extracted through an artificial intelligence model in the user intent storage 520 in the case of data that has no previous extraction history; and
A voice recognition media playback device and method, comprising a user intention storage 520 that stores speech intent and components extracted through an artificial intelligence model. According to paragraph 1,
The intent command connection unit 600,
A media control command storage 610 that stores pairs of media control commands corresponding to each speech intention. A voice recognition media playback device and method comprising a.