KR20190084460A - Method and system for noise-robust sound-based respiratory disease detection - Google Patents

Abstract

Disclosed are a detection method and system for noise-robust sound-based respiratory disease. The detection method includes a step of converting the sound signal of an object into a two-dimensional image; a step of generating a texture image including texture information from the two-dimensional image; and a step of detecting a respiratory disease corresponding to the texture information through an image classification learning model.

Description

TECHNICAL FIELD [0001] The present invention relates to a noise-based sound-based respiratory disease detection method and system,

The following description relates to a technique for detecting respiratory disease based on the sound of an object.

In an environment where a large number of weaned pigs are closely packed in a small space, the infected diseases such as foot-and-mouth disease and swine respiratory diseases are very vulnerable to the rapid spread of the damage.

Among the environmental factors that cause pig respiratory disease, the causative agents of major bacterial and viral respiratory diseases in pig farms are MH (Mycoplasma Hyopneumonia), PCV2 (Porcine Circo Virus 2) and Porcine Reproductive Respiratory Syndrome (PRRS) It causes systemic wasting syndrome (PMWS), which leads to enormous economic losses, especially for older pigs. In order to prevent massive loss of pig respiratory disease, it is necessary to prevent and prevent disease before the disease develops.

However, due to the current technical and economic situation, the early disaster prevention system for swine respiratory diseases is very limited and the economic loss due to this is increasing day by day.

One of the solutions to the problems of livestock farmers is researching new technologies such as IT and computer and electronics in agriculture. For example, various spectrum studies using video and sound sensors are underway to monitor livestock behavior more deeply, starting with basic monitoring for pig farm management such as feed consumption of pork, humidity and temperature.

A method and system for detecting respiratory disease based on the sound of an object of interest.

A method and system for detecting respiratory disease based on noise-robust sound information in a variety of noisy environments.

A computer-implemented respiratory disease detection method, comprising: converting a sound signal of an object into a two-dimensional image; Generating a texture image including texture information from the two-dimensional image; And detecting respiratory disease corresponding to the texture information through an image classification learning model.

According to an aspect of the present invention, the generating step may include generating the texture image by applying a dominant neighborhood structure (DNS) algorithm.

According to another aspect, the generating step comprises: a first step of covering an nxn-size search window with respect to an arbitrary edge of the two-dimensional image; A second step of generating a first vector by surrounding a neighboring window having a size of m x m around a center pixel in the search window; A third step of generating a second vector by covering a neighboring window having a size of m x m around one pixel in the search window; A fourth step of replacing the distance value between the first vector and the second vector with a pixel value of a new image; And generating a texture image that is a DNS map image by repeating the third step and the fourth step for all pixels in the search window.

According to another aspect, the converting step may include generating a grayscale image by normalizing the value of the sound signal and mapping the normalized value to a two-dimensional matrix.

According to another aspect, the converting step may include: normalizing a length of the sound signal through a linear transformation; Normalizing the value of the normalized sound signal through the linear transformation to a value between 0 and 255; And mapping the normalized values from 0 to 255 into a two-dimensional matrix to generate a grayscale image.

According to another aspect of the present invention, the image classification learning model is a CNN (Convolutional Neural Network) model that learns a normal sound signal and a respiratory disease-specific sound signal, and the detecting step comprises: And detecting respiratory disease of the object.

A computer-implemented respiratory disease detection system, comprising: a preprocessor for converting a sound signal of an object into a two-dimensional image; A texture extracting unit for extracting texture information from the two-dimensional image; And a disease detection unit for detecting a respiratory disease corresponding to the texture information through an image classification learning model.

According to embodiments of the present invention, a respiratory disease detection system can be constructed at low cost by detecting respiratory disease based on the sound of an object of interest, and a robust and excellent performance system can be configured even in a noisy environment.

1 is a block diagram for explaining an example of the internal configuration of a computer system according to an embodiment of the present invention.
2 is a diagram illustrating an example of components that a processor of a computer system according to an embodiment of the present invention may include.
3 is a flowchart illustrating an example of a respiratory disease detection method that can be performed by a computer system according to an embodiment of the present invention.
4 is an exemplary diagram for explaining a process of converting a sound signal into a two-dimensional image according to an embodiment of the present invention.
5 is an exemplary diagram for explaining a DNS map generation process according to an embodiment of the present invention.
FIG. 6 illustrates an example of a CNN structure that receives a DNS map as an input according to an exemplary embodiment of the present invention.
FIG. 7 shows an example of a normal sound and a sound signal type according to respiratory disease in an embodiment of the present invention.
FIG. 8 is a graph showing an experiment result of converting a sound signal pattern of a normal sound and a respiratory disease into a texture image according to an embodiment of the present invention.
FIG. 9 shows an example of applying DNS to a sound created by synthesizing white noise and environmental noise in normal sound in an embodiment of the present invention.
10 shows a graph of SSIM under various noise conditions in an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Embodiments of the present invention are directed to techniques for detecting respiratory disease based on the sound of an object.

Embodiments, including those specifically disclosed herein, can detect respiratory disease using noise-robust sound signals and achieve significant advantages in terms of accuracy, efficiency, cost savings, and the like.

In this specification, an object refers to an object for disease detection, and may include an animal (livestock) such as a pig, a cattle, a sheep, etc. Hereinafter, a pig is described as a specific example, but the present invention is not limited thereto.

Recently, a technique for detecting respiratory diseases based on the vocal sounds of pigs has been studied. MFCC features a technique for identifying respiratory diseases of pigs using DTW (Dynamic Time Warping) technique after generating feature vectors by applying filtering techniques and amplitude modulation techniques in the sound frequency band, and cough sounds caused by respiratory diseases Extracts information, detects respiratory diseases with Support Vector Data Description (SVDD), classifies disease types through SRC (Sparse Representation Classifier), and generates respiratory disease sounds in various time and frequency domains And algorithms for selecting meaningful features for detection of respiratory disease have been proposed. Most of the studies for detecting respiratory diseases are based on sound signals.

However, since the sound signal is data that is very vulnerable to ambient noise, it is necessary to construct a noise-robust system considering various environments.

1 is a block diagram for explaining an example of the internal configuration of a computer system according to an embodiment of the present invention.

A respiratory disease detection system in accordance with embodiments of the present invention may be implemented through the computer system 100 of FIG. 1, the computer system 100 includes a processor 110, a memory 120, a persistent storage 130, a bus 140, an input / output interface (I / O) 150 and a network interface 160.

Processor 110 may include or be part of any device capable of processing a sequence of instructions. The processor 110 may comprise, for example, a processor and / or a digital processor within a computer processor, a mobile device, or other electronic device. The processor 110 may be, for example, a server computing device, a server computer, a series of server computers, a server farm, a cloud computer, a content platform, a mobile computing device, a smart phone, a tablet, The processor 110 may be connected to the memory 120 via a bus 140.

The memory 120 may include volatile memory, permanent, virtual or other memory for storing information used by or output by the computer system 100. For example, memory 120 may include random access memory (RAM) and / or dynamic RAM (DRAM). The memory 120 may be used to store any information, such as the state information of the computer system 100. Memory 120 may also be used to store instructions of computer system 100, including, for example, instructions for disease detection. Computer system 100 may include one or more processors 110 as needed or where appropriate.

The bus 140 may comprise a communication infrastructure that enables interaction between the various components of the computer system 100. The bus 140 may carry data between components of the computer system 100, for example, between the processor 110 and the memory 120. The bus 140 may comprise a wireless and / or wired communication medium between the components of the computer system 100 and may include parallel, serial, or other topology arrangements.

The persistent storage 130 may store components such as memory or other persistent storage as used by the computer system 100 to store data for a predetermined extended period of time (e.g., as compared to the memory 120) . The persistent storage device 130 may include non-volatile main memory as used by the processor 110 in the computer system 100. For example, the persistent storage device 130 may comprise flash memory, hard disk, optical disk, or other computer readable medium.

The input / output interface 150 may include interfaces to a keyboard, a mouse, a microphone, a camera, a display, or other input or output device. Inputs related to configuration commands and / or disease detection functions may be received via input / output interface 150.

The network interface 160 may include one or more interfaces to networks such as a local area network or the Internet. The network interface 160 may include interfaces for wired or wireless connections. Configuration commands may be received via the network interface 160. Information related to the disease detection function may be received or transmitted via the network interface 160. [

Also, in other embodiments, the computer system 100 may include more components than the components of FIG. However, there is no need to clearly illustrate most prior art components. For example, the computer system 100 may be implemented to include at least some of the input / output devices connected to the input / output interface 150 described above, or may include a transceiver, a Global Positioning System (GPS) module, Databases, and the like. More specifically, when the computer system 100 is implemented in the form of a mobile device such as a smart phone, it is generally used by a camera included in a mobile device, an acceleration sensor, a gyro sensor, a camera, various physical buttons, Buttons, an input / output port, a vibrator for vibration, and the like may be further included in the computer system 100. [

FIG. 2 is a diagram illustrating an example of components that a processor of a computer system according to an embodiment of the present invention may include; FIG. 3 is a block diagram of a respiratory disease A flowchart showing an example of a detection method.

2, the processor 110 may include a preprocessing module 210, a texture extraction module 220, and a disease detection module 230 . The components of such a processor 110 may be representations of different functions performed by the processor 110 in accordance with control commands provided by at least one program code. For example, the preprocessor 210 may be used as a functional representation that the processor 110 operates to control the computer system 100 to perform preprocessing on the audio signal. The components of the processor 110 and the processor 110 may perform the steps S310 through S340 included in the respiratory disease detection method of FIG. For example, the components of processor 110 and processor 110 may be implemented to execute instructions in accordance with the at least one program code described above with the code of the operating system that memory 120 contains. Here, the at least one program code may correspond to a code of a program implemented to process a respiratory disease detection method.

The respiratory disease detection method may not occur in the order shown in FIG. 3, and some of the steps may be omitted or an additional process may be further included.

In step S310, the processor 110 may load the program code stored in the program file for the respiratory disease detection method into the memory 120. [ For example, a program file for a respiratory disease detection method may be stored in the persistent storage 130 described with reference to FIG. 1, and the processor 110 may retrieve the program files stored in the persistent storage 130, The computer system 110 may be controlled such that the program code is loaded into the memory 120. [ At this time, the preprocessing unit 210, the texture extracting unit 220, and the disease detecting unit 230 included in the processor 110 and the processor 110 respectively receive the instruction of the corresponding part of the program code loaded in the memory 120 And may be different functional representations of processor 110 for executing subsequent steps S320 through S340. For the execution of steps S320 through S340, the processor 110 and the components of the processor 110 may process an operation according to a direct control command or control the computer system 100. [

In step S320, the preprocessing unit 210 receives the one-dimensional sound signal of the pig to be detected and converts it into a two-dimensional still image. First, the preprocessing unit 210 normalizes the length of a signal while maintaining a characteristic of a sound signal through a linear transformation. Then, the preprocessing unit 210 generates a two-dimensional grayscale image by mapping a normalized sound signal having the same length to an image having a value ranging from 0 to 255 through linear conversion. 4 is an exemplary diagram for explaining a process of converting a sound signal into a two-dimensional image. 4, the preprocessor 210 normalizes the length of the sound signal through the normalization (linear transformation) of the X-axis of the sound signal and normalizes the value of the sound signal through the Y-axis normalization of the sound signal to a value between 0 and 255 . ≪ / RTI > The pre-processing unit 210 may generate a grayscale image by mapping the normalized values to a two-dimensional matrix after the normalization of the above-described process is completed.

Referring again to FIG. 3, in step S330, the texture extracting unit 220 may extract texture information from the grayscale image using a dominant neighborhood structure (DNS) algorithm. In order to solve the problem of a sound signal having a structural weak point that is vulnerable to noise, the present invention applies a DNS technique for a noise problem to an image to a sound signal. The DNS algorithm is a proven methodology with robust performance in noisy images, such as <F. Kelllah, &quot; Texture classification using dominant neighborhood structure, &quot; IEEE Transaction on Image Processing , Vol. 21, No. 11, pp. 3270-3279, In the present invention, an application domain of the DNS is expanded by newly applying the DNS algorithm to the sound signal. 5 is an exemplary diagram for explaining a DNS map generation process. Referring to FIG. 5, in order to generate a DNS map, a searching window and a neighborhood window must be defined. An n × n size search window is written based on an arbitrary left edge of the image, The center pixel of the search window is called the center pixel. In order to extract the texture feature, a vector V _s is generated by surrounding a neighboring window having a size of m × m around the center pixel in the search window. In addition, a vector V _n is generated by covering a neighboring window centered on the upper left pixel in the search window, and V _s and V _n The Euclidean distance is calculated and the pixel value is sequentially replaced from the upper left corner of the new blank image. This process is repeated until all the pixels in the search window are processed to generate the final DNS map. In other words, the texture extracting unit 220 can extract the texture information from the grayscale image using the DNS algorithm, thereby generating the DNS map, i.e., the texture image, converted from the sound signal into the texture information.

Referring again to FIG. 3, in step S340, the disease detection unit 230 may detect a disease corresponding to the texture information through the image classification learning model. In the disease detection unit 230, a CNN (Convolutional Neural Network), which is one of representative models of deep learning which shows good performance in image classification, is designed and applied for detecting diseases in order to detect and identify the respiratory diseases of pigs. CNN is generally composed of three layers: a convolution layer, a pooling layer, and a fully-connected layer. FIG. 6 shows a DNS map converted from a sound signal to texture information 1 shows an example of a CNN structure received as an input. The CNN is learned by using the sound signal of the pig with the disease by the normal sound signal of the diseased pig and the respiratory disease, and the DNS map image of the sound signal of the pig is input to the CNN structure which has completed the learning, The presence of respiratory diseases and the type of disease can be detected. Although the CNN structure is exemplarily described as the image classification learning model, the present invention is not limited thereto, and any learning model capable of image classification can be applied.

An example of the normal sound of the pig and the sound signal form of the respiratory disease is shown in FIG. 7, and the sound signal due to PMWS, PRRS and MH disease, which is one of the normal sounds, grunt and respiratory diseases, It is difficult to distinguish easily.

The normalized sound signal is transformed into a 158 × 158 2D grayscale image by linear transformation and the texture information is extracted by DNS. In the DNS experiment, the size of the search window is fixed to 64, and the size of the neighboring window is fixed to 32. As a result, a texture image of 64 × 64 size can be generated. FIG. 8 shows experimental results obtained by converting the signal types of the normal sound of the pig and the sound of the respiratory disease type into the texture image. In FIG. 8, it is confirmed that the pigs respiratory texture sound waveforms hardly distinguishable in FIG. 7 have their own texture information . 8 (a) shows a texture feature extending horizontally and long in reference to a center coordinate of an image, (b) is a texture feature arranged horizontally and closely arranged, (c) is horizontal texture information based on a center coordinate, And (d) shows unique characteristic information of the diagonal line.

FIG. 9 shows an example of a result obtained by applying DNS to a sound produced by synthesizing white noise and environmental noise in a grunt sound, which is one of normal sounds. From the texture image results of FIG. 9, which are sequentially compared according to the SNR (signal to noise ratio) intensity of the noise, most of the white noise and environmental noise are removed through the DNS, . This can be confirmed by quantitative values using Structural Similarity (SSIM) indicators (see FIG. 10). SSIM is a method of measuring the similarity of the original image to the distortion caused by the compression and transformation of the image, on the assumption that the loss of image quality is caused by the structural distortion of the image signal itself. SSIM is expressed as a value between 0 and 1, and the smaller the difference between the original image and the image quality, the closer the value is. In FIG. 10, the dotted line represents the SSIM value before applying the DNS, and the solid line represents the SSIM similarity value between the original image and the noise image after applying the DNS. As can be seen from FIGS. 9 to 10, it can be seen that the unique texture of each respiratory disease is maintained even in a noisy environment, so that it is robust to noise.

According to embodiments of the present invention, a respiratory disease detection system can be constructed at low cost by detecting respiratory disease based on the sound of an object of interest, and a robust and excellent performance system can be configured even in a noisy environment.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit, a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be embodied in any type of machine, component, physical device, computer storage media, or device for interpretation by a processing device or to provide instructions or data to the processing device have. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. At this time, the medium may be a program that continuously stores a computer executable program, or temporarily stores the program for execution or downloading. Further, the medium may be a variety of recording means or storage means in the form of a combination of a single hardware or a plurality of hardware, but is not limited to a medium directly connected to any computer system, but may be dispersed on a network. Examples of the medium include a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, an optical recording medium such as CD-ROM and DVD, a magneto-optical medium such as a floptical disk, And program instructions including ROM, RAM, flash memory, and the like. As another example of the medium, a recording medium or a storage medium that is managed by a site or a server that supplies or distributes an application store or various other software is also enumerated.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI &gt; or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (10)

A computer-implemented respiratory disease detection method,
Converting a sound signal of the object into a two-dimensional image;
Generating a texture image including texture information from the two-dimensional image; And
Detecting a respiratory disease corresponding to the texture information through an image classification learning model
/ RTI &gt; The method according to claim 1,
Wherein the generating comprises:
A step of generating the texture image by applying a dominant neighborhood structure (DNS) algorithm
/ RTI &gt; 3. The method of claim 2,
Wherein the generating comprises:
Comprising: a first step of covering an nxn-size search window based on an arbitrary edge of the two-dimensional image;
A second step of generating a first vector by surrounding a neighboring window having a size of m x m around a center pixel in the search window;
A third step of generating a second vector by covering a neighboring window having a size of m x m around one pixel in the search window;
A fourth step of replacing the distance value between the first vector and the second vector with a pixel value of a new image; And
Generating a texture image that is a DNS map image by repeating the third step and the fourth step for all the pixels in the search window
/ RTI &gt; The method according to claim 1,
Wherein the converting comprises:
Generating a grayscale image by normalizing the value of the sound signal and mapping the normalized value to a two-dimensional matrix
/ RTI &gt; The method according to claim 1,
Wherein the converting comprises:
Normalizing a length of the sound signal by linear transformation;
Normalizing the value of the normalized sound signal through the linear transformation to a value between 0 and 255; And
A step of generating a grayscale image by mapping a normalized value from 0 to 255 into a two-dimensional matrix
/ RTI &gt; The method according to claim 1,
The image classification learning model is a CNN (Convolutional Neural Network) model that learns normal sound signals and sound signals of respiratory diseases,
Wherein the detecting comprises:
Detecting the respiratory disease of the object using the texture image as an input to the CNN model
/ RTI &gt; A computer-implemented respiratory disease detection system,
A preprocessing unit for converting a sound signal of the object into a two-dimensional image;
A texture extracting unit for extracting texture information from the two-dimensional image; And
A disease detection unit for detecting a respiratory disease corresponding to the texture information through an image classification learning model,
A respiratory disease detection system. 8. The method of claim 7,
The pre-
Normalizing the length of the sound signal by linear transformation,
Normalizing the value of the normalized sound signal through the linear conversion to a value between 0 and 255,
Generating a grayscale image by mapping normalized values from 0 to 255 to a two-dimensional matrix
A respiratory disease detection system. 8. The method of claim 7,
The texture extracting unit extracts,
Generating a texture image including the texture information by applying a dominant neighborhood structure (DNS) algorithm
A respiratory disease detection system. 10. The method of claim 9,
The image classification learning model is a CNN (Convolutional Neural Network) model that learns normal sound signals and sound signals of respiratory diseases,
The disease-
Detecting the respiratory disease of the object using the texture image as an input to the CNN model
A respiratory disease detection system.

Images