WO2016005403A1

WO2016005403A1 - Real time audification of neonatal electroencephalogram (eeg) signals

Info

Publication number: WO2016005403A1
Application number: PCT/EP2015/065515
Authority: WO
Inventors: Andriy Temko; Gordon Lightbody; Liam MARNANE; Geraldine BOYLAN
Original assignee: University College Cork
Current assignee: University College Cork
Priority date: 2014-07-07
Filing date: 2015-07-07
Publication date: 2016-01-14
Anticipated expiration: 2017-01-07
Also published as: US20170172523A1; US10667761B2; EP3166484A1; GB201412071D0

Abstract

The present invention discloses a method and system of providing a real time audification of neonatal EEG signals. The method comprises the steps of: receiving preprocessed neonatal EEG signals; changing a characteristic of the preprocessed signals in a phase vocoder; resampling the output signals from the vocoder to a predetermined audio frequency range; converting the resampled signals into stereo signals; and selecting a plurality of channels from the stereo signals as the output audio signals.

Description

Title

Real time audification of neonatal electroencephalogram (EEG) signals Field

The present invention is concerned with neonatal electroencephalogram (EEG) signals.

Background

Neonatal seizures are the most common neurological emergency in the neonate and are a serious concern for clinicians and parents worldwide. Only about one third of all neonatal seizures are clinically visible and many remain undetected in the busy Neonatal Intensive Care Unit (NICU) environment. The only method available to detect all neonatal seizures accurately is continuous multi-channel EEG monitoring. Interpretation of neonatal EEG requires a neurophysiologist or paediatric neurologist with specific expertise in neonatal EEG. This expertise is not available on a 24h basis, 7 days a week. To fill the gap in the availability of appropriate expertise, clinical staff in the NICU often use a simpler form of EEG monitoring, known as amplitude integrated EEG (aEEG). Amplitude integrated EEG is an logarithmically-scaled, temporally-smoothed and compressed display of EEG which is usually computed from two EEG channels, one from each hemisphere. Despite the fact that many short and focal neonatal seizures are undetectable with aEEG and inter-observer agreement is poor, aEEG currently serves as a trade-off between very inaccurate clinical detection of seizures and very accurate but scarcely available neurophysiologic expertise, and thus is widely adopted worldwide in the NICU.

In view of the above mentioned problems associated with the use of aEEG, it will be appreciated that an automated decision support system that could detect and annotate seizures on the neonatal EEG would be extremely useful for clinicians in the NICU.

In this regard, it is believed that human hearing input is better than the visual input when it comes to assessing both the spatial and temporal evolution of the frequency characteristics of a signal. Hearing is flexible and low-cost. Hearing also allows for faster processing than visual presentation, has better temporal resolution, and represents an additional information channel, releasing visual sense for other tasks. Therefore, an automated decision support system that could detect and annotate seizures on the neonatal EEG through the use of audio would be desirable.

It is known to audify the EEG signals of adults, for purposes such as to detect epilepsy. One such audification method involves lifting an adult's recorded brain frequencies into the human audible range, by saving the original waveform with a higher sampling rate. In this signal resampling process, time and pitch manipulations are always linked so that time compression scales pitches upwards, time stretching scales pitches downwards. However, with this direct EEG audification, the resultant audio sounds noisy. Furthermore, real-time EEG playback is also not possible.

Another such audification method of adult EEG signals is to map EEG spectral frequencies to the audible range by sonification. In this process, the dominant EEG frequencies are extracted using the fast Fourier transform, and then the tones of the mapped frequencies are created with pre-specified parameters such as duration and pitch. However, it should be appreciated that when sonification is used, what is heard by a listener is not the real EEG, but rather 'synthetic' artificially generated audio waveforms which have been extracted from the original EEG content.

Accordingly, an object of the present invention is to provide a method and system for the audification of neonatal EEG signals which overcomes at least one of above mentioned problems associated with existing adult EEG signal audification methods. Summary

The present invention provides a method of providing a real time audification of neonatal EEG signals, comprising the steps of:

receiving preprocessed neonatal EEG signals;

changing a characteristic of the preprocessed signals in a phase vocoder;

resampling the output signals from the phase vocoder to a predetermined audio frequency range;

converting the resampled signals into stereo signals; and

selecting a plurality of channels from the stereo signals to form an output audio signal.

This method provides an audification of a neonate's EEG signals which time stretches without affecting the pitch, and pitch scales without affecting the signal duration. In addition, the method enables direct EEG playback without any artificially created synthetic sounds.

The step of converting the resampled signals into stereo signals may comprise the step of converting the resampled signals into left and right stereo channels corresponding to left and right brain hemispheres.

The step of selecting a plurality of channels from the stereo signals to form the output audio signal may comprise the step of selecting a channel from the left stereo channel and selecting a channel from the right stereo channel to form the output audio signal.

The method may further comprise the steps of:

determining segments of the output audio signal which satisfy a predetermined criterion; and

amplifying the gain of the output audio signal for the determined segments.

The step of selecting a channel from the left stereo channel and selecting a channel from the right stereo channel as the output audio signal may further comprise the steps of: determining a channel from the left brain hemisphere and a channel from the right brain hemisphere which has the highest cumulative probability of exhibiting a seizure; and

selecting the determined channel from the left brain hemisphere and the determined channel from the right brain hemisphere to form the output audio signal.

The step of determining segments of the output audio signal which satisfy a predetermined criterion may correspond to determining those segments which have the highest probability of exhibiting a seizure.

Preferably, the temporal characteristics of the preprocessed signals are changed.

The changing of the temporal characteristics of the preprocessed signals may comprise the steps of:

segmenting the preprocessed signal into overlapping frames;

applying the short-time Fourier transform to each frame;

separating the magnitude and phase of each frame;

estimating the instantaneous frequency of each frame;

determining the phase function associated with the frames; and

performing the inverse short-time Fourier transform on each frame.

The temporal characteristic of the signal spectrogram may be changed by slowing down the preprocessed signals by a factor of 100.

The resampling may correspond to mapping those frequencies known to correspond to the dominant frequencies of a seizure to the audible frequency range. The predetermined audio frequency range may correspond to 3-4 KHz.

The step of preprocessing the neonatal EEG signals may correspond to:

filtering of the EEG signals by low pass and high pass filters; and downsampling the frequency of the signals to 32Hz.

The downsampling may correspond to 32Hz. The output signals from the vocoder may be resampled at a 32kHz sampling rate.

The present invention also provides a system for providing a real time audification of neonatal EEG signals, comprising:

means or module for receiving preprocessed neonatal EEG signals;

means or module for changing a characteristic of the preprocessed signals in a phase vocoder;

means or module for resampling the output signals from the vocoder to a predetermined audio frequency range;

means or module for converting the resampled signals into stereo signals; and means or module for selecting a plurality of channels from the stereo signals as the output audio signals.

In another embodiment there is provided a method of providing a real time audification of neonatal EEG signals, comprising the steps of:

receiving preprocessed neonatal EEG signals;

changing a characteristic of the preprocessed signals in a phase vocoder with application specific parameters in order to increase audio intelligibility of the neonatal EEG signals;

re-sampling the output signals from the phase vocoder to a predetermined audio frequency range to match a human hearing system;

converting the re-sampled signals into stereo audio signals and playing the created stereo audio; and

selecting a plurality of channels from the signals to form an output stereo audio signal.

There is also provided a computer program comprising program instructions for causing a computer program to carry out the above method which may be embodied on a record medium, carrier signal or read-only memory. Brief Description of the Drawings The invention will be more clearly understood from the following description of an embodiment thereof, given by way of example only, with reference to the accompanying drawings, in which :-

Figure 1 details the main steps in the process of the invention which provides a real time audification of neonatal EEG signals.

Figure 2 shows a flowchart for the audification of neonatal EEGs;

Figure 3 shows a probabilistic output for each stereo channel representing the EEG signal; and

Figure 4 shows a flowchart of the main processing steps of the phase vocoder of the present invention.

Detailed Description of the Drawings

The present invention provides a method and system for listening to the EEG of newborn babies or neonates. By listening to the EEG of a neonate, various patterns of the EEG signals can be detected by the listener. In the described embodiment of the invention, such patterns are used to indicate to the listener the presence of a seizure in the brain of a neonate.

Figure 1 details the main steps in the process of the invention which provides a real time audification of neonatal EEG signals. In step 100, preprocessed neonatal EEG signals are received. In step 105, a characteristic of the preprocessed signals is changed in a phase vocoder. The output signals from the vocoder are then resampled to a predetermined audio frequency range (step 1 10). The resampled signals are then converted into stereo signals (step 1 15). Finally, a plurality of channels from the stereo signals are selected as the output audio signals (step 120).

Figure 2 details the main steps in one embodiment of the audification process of the present invention, with an example of 1000s of EEG as an input. In the first step, the output signals of the EEG are pre-processed prior to being input to a vocoder through the filtering of the EEG signals by low pass and high pass filters, to attenuate any unwanted information. This is followed by the downsampling of the frequency of the signals to 32Hz. In the vocoder, the temporal characteristics of the signals are changed. This intuitively corresponds to stretching the time-base of the signal spectrogram while retaining their short-time spectral characteristics. Therefore, the signals are slowed down by a factor of N, which in the preferred embodiment has a value of 100. These signals are then resampled to an audible range.

In the frequency mapper, the output signals from the vocoder are resampled at a 32 kHz sampling rate. This corresponds to pitch shifting the original frequency range of 0.5-16 Hz to a new range of 0.5-16 kHz. Through frequency mapping, the most dominant frequencies of seizure (typically 0.5-6 kHz) can be mapped to signals within the audible range, in particular to the range of human scream, namely 3-4 kHz.

As a result, the EEG real-time playback is sped up by a factor which is inversely proportional to N. It will be appreciated that this provides for a significant time saving for the listener, for example where in the preferred embodiment 1 hour of EEG can be played back in around 6 minutes.

In a further step, the audio signals are converted into stereo signals, such that the left stereo signal corresponds to the left brain hemisphere and the right stereo signal corresponds to the right brain hemisphere. These stereo signals are then input to a seizure detection module. The output of the seizure detection module corresponds to the final audified EEG signal.

In the preferred embodiment of the seizure detection module, the channel from each hemisphere which is determined to have the highest cumulative probability of a seizure event is selected to form the output audio signal. This is determined by computing the cumulative seizure probability from the per-channel probabilistic outputs which are shown in Figure 3, which shows a plot of probability output for each channel versus time (in minutes). The channels with the maximum cumulative probability is taken separately from the left (F3-C3, C3-01 , Cz-C3, C3-T3) and right hemisphere (F4-C4, C4-02, T4-C4, C4-Cz). In this figure, the maximum probabilistic output across channel is shown in bold, while seizure onset and offset are annotated as 16m37s-30m05s and superimposed on top in red. The signal gain is also controlled by the probabilistic output of the system, such that the signal is multiplied by the maximum probabilistic output of the system. As a result, EEG segments which are likely to be seizures are more audible than the background.

Figure 4 shows a flowchart of the main processing steps of the phase vocoder of the present invention. It comprises three main processing stages, namely analysis, processing, and synthesis. During the analysis stage, the signal is segmented into overlapping frames with analysis time-instants t_a=uR_a, for an integer u, and the short-time Fourier transform (STFT) is then applied to each frame:

Where x is the original signal, h(n) is the analysis window, Ωκ = 2π \Ι is the centre frequency of /cth STFT channel, and N is the size of the STFT. X(t,Q) is a function of both time and frequency.

In the synthesis stage, the inverse short-time Fourier transform is performed on each frame spectrum. The overlapping spectrum frames result in the output segments overlapping each other. The overlapping output segments are all summed together, yielding the following output signal (known as the overlap-add method) in the time domain:

In the absence of modifications, t_s=at_a, a=R_a/R_s=1 , and the output signal is identical to the original signal, where R_a and R_s are the hop size of the analysis and synthesis, respectively. It should be noted that all modifications that are done in the spectrum representation need to preserve the appropriate correlation between adjacent frequency bins and time frames. In the processing stage, the magnitude and phase are separated. The time evolution of sine-wave amplitude is modified simply by setting:

In the synthesis stage, the process requires phase unwrapping. The phase, which is measured modulo 2π, is unwraped by keeping track of cumulative phase variation and taking its principal determination between ±ττ:

Where zX(t^u) and zX(t^u_1) are the phases at time instances t^u and t^u_1 , respectively. Phase unwrapping is a process whereby the phase increment between two consecutive frames is used to estimate the instantaneous frequency of the closest sinusoid:

1

w_k{t^) = n_k +— Δ where Wk is the instantaneous frequency.

The phase increment ΔΦ is simply the small phase shift resulting from WK being close but not equal to Ωκ. Once the instantaneous frequency is estimated, the phase of the time-scaled output signal Y is set according to the phase- propagation formula

This phase function can be decimated or interpolated to the new time scale, so that the phase of the output short-time Fourier Transform, STFT can be computed at any given synthesis time-instant: i=l

It will be appreciated that the method of audification of neonatal EEG signals of the present invention provides a number of advantages when compared to existing adult EEG signal audification methods. Firstly, the phase vocoder of the present invention allows for time stretching without affecting the pitch, and pitch scaling without affecting the signal duration. In addition, the method of the present invention enables direct EEG playback without any artificially created synthetic sounds.

While the present invention has been described in an application where the audification of a neonate's EEG signals are used as a unique audio alarm for seizures, the audification of present invention could equally be used in a number of other applications. For example, the audio signals could be used to help classify the different background EEG patterns occurring in term and preterm babies, and in the sleep state analysis of newborns. In addition, the audio signals could allow parents to listen' to their babies brain waves, and could also be played to sick newborns to help reduce stress. The embodiments in the invention described with reference to the drawings comprise a computer apparatus and/or processes performed in a computer apparatus. However, the invention also extends to computer programs, particularly computer programs stored on or in a carrier adapted to bring the invention into practice. In particular computer programs for controlling the system and method as hereinbefore described. The program may be in the form of source code, object code, or a code intermediate source and object code, such as in partially compiled form or in any other form suitable for use in the implementation of the method according to the invention. The carrier may comprise a storage medium such as ROM, e.g. CD ROM, or magnetic recording medium, e.g. a floppy disk or hard disk. The carrier may be an electrical or optical signal which may be transmitted via an electrical or an optical cable or by radio or other means. Additionally, at least a portion of the systems, methodologies and techniques described with respect to the exemplary embodiments of present disclosure can incorporate a machine, such as, but not limited to, computer system, or any other computing device within which a set of instructions, when executed, may cause the machine to perform any one or more of the methodologies or functions discussed above. The machine may be configured to facilitate various operations conducted by the systems disclosed herein. For example, the machine may be configured to, but is not limited to, assist the systems by providing processing power to assist with processing loads experienced in the systems, by providing storage capacity for storing instructions or data traversing the systems, or by assisting with any other operations conducted by or within the systems.

Dedicated hardware implementations including, but not limited to, application specific integrated circuits, programmable logic arrays and other hardware devices can likewise be constructed to implement the methods described herein. Applications that may include the apparatus and systems of various embodiments broadly include a variety of electronic and computer systems. Some embodiments implement functions in two or more specific interconnected hardware modules or devices with related control and data signals communicated between and through the modules, or as portions of an application-specific integrated circuit. Thus, the example system is applicable to software, firmware, and hardware implementations.

In accordance with various embodiments of the present disclosure, the methods described herein are intended for operation as software programs running on a computer processor. Furthermore, software implementations can include, but not limited to, distributed processing or component/object distributed processing, parallel processing, or virtual machine processing can also be constructed to implement the methods described herein.

In the specification the terms "comprise, comprises, comprised and comprising" or any variation thereof and the terms include, includes, included and including" or any variation thereof are considered to be totally interchangeable and they should all be afforded the widest possible interpretation and vice versa.

The invention is not limited to the embodiments hereinbefore described but may be varied in both construction and detail.

Claims

1 . A method of providing a real time audification of neonatal EEG signals, comprising the steps of:

receiving preprocessed neonatal EEG signals;

changing a characteristic of the preprocessed signals in a phase vocoder;

converting the resampled signals into stereo signals; and

selecting a plurality of channels from the stereo signals to form an output stereo audio signal.

2. The method of Claim 1 , wherein the step of converting the resampled signals into stereo signals comprises the step of converting the resampled signals into left and right stereo channels corresponding to left and right brain hemispheres.

3. The method of Claim 2, wherein the step of selecting a plurality of channels from the stereo signals to form the output audio signal comprises the step of selecting a channel from the left stereo channel and selecting a channel from the right stereo channel to form the output audio signal.

4. The method of Claim 3, further comprising the step of:

amplifying the gain of the output audio signal for the determined segments.

5. The method of Claim 4, wherein the step of selecting a channel from the left stereo channel and selecting a channel from the right stereo channel as the output audio signal further comprises the steps of:

determining a channel from the left brain hemisphere and a channel from the right brain hemisphere which has the highest cumulative probability of exhibiting a seizure; and selecting the determined channel from the left brain hemisphere and the determined channel from the right brain hemisphere to form the output audio signal.

6. The method of Claim 5, where the step of determining segments of the output audio signal which satisfy a predetermined criterion corresponds to determining those segments which have the highest probability of exhibiting a seizure.

7. The method of Claim 1 , wherein the temporal characteristics of the preprocessed signals are changed.

8. The method of Claim 7, wherein the changing of the temporal characteristics of the preprocessed signals comprises the steps of:

segmenting the preprocessed signal into overlapping frames;

applying the short-time Fourier transform to each frame;

separating the magnitude and phase of each frame;

estimating the instantaneous frequency of each frame;

determining the phase function associated with the frames; and

performing the inverse short-time Fourier transform on each frame.

9. The method of Claim 7, wherein the temporal characteristic of the signal spectrogram is changed by slowing down the preprocessed signals by a factor of 100.

10. The method of Claim 1 wherein the resampling corresponds to mapping those frequencies known to correspond to the dominant frequencies of a seizure to the audible frequency range.

1 1 . The method of Claim 1 , wherein the predetermined audio frequency range corresponds to 3-4 KHz.

12. The method of any of the preceding claims wherein the step of preprocessing the neonatal EEG signals corresponds to:

filtering of the EEG signals by low pass and high pass filters; and

downsampling the frequency of the signals to 32Hz.

13. The method of Claim 12 wherein the downsampling corresponds to 32Hz.

14. The method of any of the preceding claims, wherein the output signals from the vocoder are resampled at a 32kHz sampling rate.

15. A system for providing a real time audification of neonatal EEG signals, comprising:

means for receiving preprocessed neonatal EEG signals;

means for changing a characteristic of the preprocessed signals in a phase vocoder;

means for resampling the output signals from the vocoder to a predetermined audio frequency range;

means for converting the resampled signals into stereo signals; and

means for selecting a plurality of channels from the stereo signals as the output audio signals.

16. A method of providing a real time audification of neonatal EEG signals, comprising the steps of:

receiving preprocessed neonatal EEG signals;