US20030171936A1 - Method of segmenting an audio stream - Google Patents

Method of segmenting an audio stream Download PDF

Info

Publication number
US20030171936A1
US20030171936A1 US10/370,065 US37006503A US2003171936A1 US 20030171936 A1 US20030171936 A1 US 20030171936A1 US 37006503 A US37006503 A US 37006503A US 2003171936 A1 US2003171936 A1 US 2003171936A1
Authority
US
United States
Prior art keywords
stage
values
sub
modules
segmentation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
US10/370,065
Other versions
US7346516B2 (en
Inventor
Mikhael Sall
Sergei Gramnitskiy
Alexandr Maiboroda
Victor Redkov
Anatoli Tikhotsky
Andrei Viktorov
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
LG Electronics Inc
Original Assignee
LG Electronics Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by LG Electronics Inc filed Critical LG Electronics Inc
Assigned to LG ELECTRONICS INC. reassignment LG ELECTRONICS INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: GRAMNITSKIY, SERGEI N., MAIBORODA, ALEXANDR L., REDKOV, VICTOR V., SALL, MIKHAEL A., TIKHOTSKY, ANATOLI I, VIKTOROV, ANDREI B.
Publication of US20030171936A1 publication Critical patent/US20030171936A1/en
Application granted granted Critical
Publication of US7346516B2 publication Critical patent/US7346516B2/en
Expired - Fee Related legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/04Segmentation; Word boundary detection
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/233Processing of audio elementary streams

Definitions

  • the present invention relates to a segmentation method of an audio data stream, which is broadcasted or recorded using some media, wherein this audio data stream is a sequence of digital samples, or may be transformed to the sequence of digital samples.
  • the goal of such segmentation is the division of audio data stream into segments, which correspond to different physical sources of audio signal.
  • parameters of background source(s) is not changed essentially in the framework of one segment.
  • Audio and video recordings have become commonplace with the advent of consumer grade recording equipment. Unfortunately, both the audio and video streams provide few clues to assist in accessing the desired section of the record.
  • indexing is provided by the table of contents at the front and the index at the end, which readers can browse to locate authors and references to authors.
  • a similar indexing scheme would be useful in an audio stream, to help in location of sections where, for example, specific speakers were talking.
  • the limited amount of data associated with most audio records does not provide enough information for confidently and easily access to desired points of interest. So, user has to peruse the contents of a record in sequential order to retrieve desired information.
  • the indexation process consists of two sequential parts: segmentation and classification processes. Segmentation process implies division of the audio stream into homogeneous (in some sense) segments. Classification process implies the attributing of these segments by appropriate notes. Thus, segmentation process is the first and very important stage in the indexation process. To this problem, the basic notice in the given invention is given.
  • Broadcast news data come to use in long unsegmented speech streams, which not only contain speech with various speakers, backgrounds, and channels, but also contain a lot of non-speech audio information. So it is necessary to chop the long stream into smaller segments. It is also important to make these smaller segments homogeneous (each segment contains the data from one source only), so that the non-speech information can be discarded, and those segments from the same or similar source can be clustered for speaker normalization and adaptation.
  • An amplitude-based detector was used to break the input into chunks that are 20 to 30 seconds long.
  • Balasubramanian et al. patent U.S.5,606,643, enables retrieval based on indexing an audio stream of a recording according to the speaker.
  • the audio stream may be segmented into speaker events, and each segment labeled with the type of event, or speaker identity.
  • the audio stream may be segregated into events according to speaker difference, with segments created by the same speaker identified or marked.
  • Creating an index in an audio stream may enable a user to locate particular segments of the audio data. For example, this may enable a user to browse a recording to select audio segments corresponding to a specific speaker, or “fast-forward” through a recording to the next speaker.
  • knowing the ordering of speakers can also provide content clues about the conversation, or about the context of the conversation.
  • the ultimate goal of the segmentation is to produce a sequence of discrete segments with particular characteristics remaining constant within each one.
  • the characteristics of choice depend on the overall structure of the indexation system.
  • the present invention provides a segmentation procedure to chunk an input audio stream into segments having homogeneous acoustic characteristics.
  • This audio stream is a sequence of digital samples, which are broadcasted or recorded using some media.
  • An object of the invention is to provide a fast segmentation procedure with a relatively low numerical complexity.
  • the segmentation procedure comprises three stages. These are: first-grade characteristic calculation, second-grade characteristic calculation, and decision-making.
  • the stage of first-grade characteristic calculation is aimed for calculation of audio features vectors from the input audio stream. These features vectors define characteristics of audio signals.
  • the stage of second-grade characteristic calculation forms sequence of statistic features vectors from the sequence of audio features vectors.
  • the statistic features vectors define statistic features of the first-grade features.
  • the stage of decision-making analyses variation of the second grade features and performs definition of the segments boundaries basing on that analysis.
  • an essential aim of the invention is to provide the segmentation method, firstly, that can be used for a wide variety of applications, secondary, that the segmentation procedure may be industrial-scaled manufactured, based on the development of one relatively simple integrated circuit.
  • FIG. 1 is a block diagram of a generalized audio processing system within which the present invention may be embodied
  • FIG. 2 is a generalized flow diagram of the audio segmentation procedure
  • FIG. 3 is a flow diagram in detail of the audio segmentation procedure
  • FIG. 4 illustrates a flowchart of the sub-stage of initial segmentation
  • FIG. 5 illustrates a flowchart of the sub-stage of accurate segmentation
  • FIG. 6 shows improvement of the dividing markers positions
  • FIG. 7 shows defining the homogeneous interval inside the segment.
  • FIG. 1 is a block diagram of a generalized audio processing system 1 , within which the present invention may be embodied.
  • an audio stream is provided from a source of audio data 2 , which may be provided by recorded broadcast, a recorded video with accompanying audio track, or other audio source.
  • the audio data is sent to an audio processor 3 , which may be any well-known device such as a general purpose computer, configured according to the present invention.
  • the audio processor outputs segments of the audio data 4 .
  • FIG. 2 is a generalized flow diagram of an audio segmentation procedure 5 .
  • Box 10 is the audio stream input, for example, broadcast news input.
  • the step in box 20 is aimed for calculation of audio features vectors from the audio stream. These features vectors define characteristic features of the audio-stream.
  • Next step 30 forms sequence of statistic features vectors from the sequence of audio features vectors. The statistic features vectors define statistic characteristic of the audio features vectors.
  • variation of the statistic features vectors is analyzed and the definition of the segment boundaries basing on that analysis is performed.
  • the proposed segmentation procedure is based on the analysis of audio signal statistical features variation.
  • the output of the resulting index segmentation of the audio stream is performed at the step 50 .
  • FIG. 3 is a flow diagram in detail of the audio segmentation procedure.
  • an input sequence of the digital samples is divided into the sequence of short (e.g. 10-20 ms) not-overlapped frames 21 .
  • the feature vectors are computed for each frame. This computation is performed using 10th order Linear Predictive Coding (LPC) analysis of the samples in possibly overlapped windows, which contain said frames.
  • LPC Linear Predictive Coding
  • K 1 ,K 2 First and the Second Reflection Coefficients
  • the audio feature vectors (9 characteristics at all). These vectors have definite physical meaning and the dynamical range sufficient for the precise segmentation of the audio stream.
  • the further work of the segmentation procedure is the statistical analysis of the obtained data. The calculation of the statistical characteristics is performed in non-overlapped second-grade windows, each of these windows consists of some predefined number of frames (e.g. 20-100 frames in one window). Thus, some number of vectors of the first-grade characteristics describes such a window.
  • the division of the input sequence of the audio feature vectors is performed at the step 31 .
  • the sequence of those vectors is transformed to the statistic feature vectors.
  • - ⁇ i 1 M ⁇
  • M is a number of frames in one window.
  • FIG. 3 serves to give an overview of the method described by the invention.
  • the sub-stage of initial segmentation 100 is performed in such a way that the dividing markers, which corresponds boundaries of segments, are determined with the accuracy corresponding to one second-grade window.
  • the sub-stage of improvement of the segmentation precision 200 carried out by the previous step implies the correction of the position of each dividing marker with the accuracy corresponding to one frame and eliminating of false segments.
  • the sub-stage of internal markers definition 300 implies the determination of a stationary interval inside each segment. The resulting sequence of the not intersected audio segments with their time boundaries is outputted at the step 50 .
  • FIG. 4 illustrates a flowchart of the sub-stage of initial segmentation 100 of FIG. 3.
  • the algorithm of this sub-stage parses four sequential entry vectors. The result of the analysis is the information, where the dividing marker is placed.
  • [0054] are calculated for the first sub-vectors of the statistical features vectors 137 . If at least one of these values is greater than the corresponding predefined threshold 138 , the dividing marker is installed between the second-range windows 139 . In this case, another steps of this sub-stage does not performed and the next four vectors, first of which is the first vector after the installed dividing marker will be taken from the set of sequential statistical features vectors for analysis 148 .
  • next four vectors first of which is the vector ⁇ right arrow over (V) ⁇ [k+1] will be taken from the set of sequential statistical features vectors for analysis 148 . If the dividing marker is taken at the step in diamond 147 , then the sub-stage of initial segmentation ends and the initial segmentation marker passes to the sub-stage of accurate segmentation.
  • FIG. 5 illustrates a flowchart of the sub-stage of accurate segmentation 200 of FIG. 3.
  • the new dividing marker ⁇ is placed into the position corresponded to this J between shaded rectangles on FIG. 6.
  • the shift of vectors is performed from the position of the new marker ⁇ .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

Disclosed herein is a segmentation method, which divides an input audio stream into segments containing different homogeneous signals. The main objective of this method is localization of segments with stationary properties. This method seeks all no-stationary points or intervals in the audio stream and creates a list of segments. The obtained list of segments can be used as an input data for the following procedures, such as classification, speech/music/noise attribution and so on. The proposed segmentation method is based on the analysis of audio signal statistical features variation and comprises three main stages: stage of first-grade characteristics calculation, stage of second-grade characteristics calculation and stage of decision-making.

Description

    BACKGROUND OF THE INVENTION
  • 1. Field of the Invention [0001]
  • The present invention relates to a segmentation method of an audio data stream, which is broadcasted or recorded using some media, wherein this audio data stream is a sequence of digital samples, or may be transformed to the sequence of digital samples. The goal of such segmentation is the division of audio data stream into segments, which correspond to different physical sources of audio signal. In the case when some source(s) and some background source(s) emit audio signal, parameters of background source(s) is not changed essentially in the framework of one segment. [0002]
  • 2. Description of the Related Art [0003]
  • Audio and video recordings have become commonplace with the advent of consumer grade recording equipment. Unfortunately, both the audio and video streams provide few clues to assist in accessing the desired section of the record. In books, indexing is provided by the table of contents at the front and the index at the end, which readers can browse to locate authors and references to authors. A similar indexing scheme would be useful in an audio stream, to help in location of sections where, for example, specific speakers were talking. The limited amount of data associated with most audio records does not provide enough information for confidently and easily access to desired points of interest. So, user has to peruse the contents of a record in sequential order to retrieve desired information. [0004]
  • As a solution of this problem, it is possible to use the automatic indexing system of audio events in the audio data stream. The indexation process consists of two sequential parts: segmentation and classification processes. Segmentation process implies division of the audio stream into homogeneous (in some sense) segments. Classification process implies the attributing of these segments by appropriate notes. Thus, segmentation process is the first and very important stage in the indexation process. To this problem, the basic notice in the given invention is given. [0005]
  • As the basic audio events in the audio stream, it is accepted to consider speech, music and noise (that is non-speech and non-music). The basic notice in a world is given to the speech detection, segmentation and indexation in audio stream, such as broadcast news. [0006]
  • Broadcast news data come to use in long unsegmented speech streams, which not only contain speech with various speakers, backgrounds, and channels, but also contain a lot of non-speech audio information. So it is necessary to chop the long stream into smaller segments. It is also important to make these smaller segments homogeneous (each segment contains the data from one source only), so that the non-speech information can be discarded, and those segments from the same or similar source can be clustered for speaker normalization and adaptation. [0007]
  • Zhan et al., “Dragon Systems' 1997 Mandarin Broadcast News System”, Proceedings of the Broadcast News transcription and Understanding Workshop, Lansdowne, Va., pp. 25-27, 1998, produced the segments by looking for sufficiently long silence regions in the output of a coarse recognition pass. This method generated considerable multi-speaker segments, and no speaker change information was used in the segmentation. [0008]
  • In the subsequent works, Wegmann et al., “Progress in Broadcast News Transcription at Dragon System”, Proceedings of ICASSP'99, Phoenix, Ariz., March, 1999, used the speaker change detection in the segmentation pass. The following is a procedure of their automatic segmentation: [0009]
  • An amplitude-based detector was used to break the input into chunks that are 20 to 30 seconds long. [0010]
  • These chunks were chopped into 2 to 30 seconds long, based on silences produced from a fast word recognizer. [0011]
  • These segments were further refined using a speaker change detector. [0012]
  • Balasubramanian et al., patent U.S.5,606,643, enables retrieval based on indexing an audio stream of a recording according to the speaker. In particular, the audio stream may be segmented into speaker events, and each segment labeled with the type of event, or speaker identity. When speech from individuals is intermixed, for example in conversational situations, the audio stream may be segregated into events according to speaker difference, with segments created by the same speaker identified or marked. [0013]
  • Creating an index in an audio stream, either in real time or in post-processing, may enable a user to locate particular segments of the audio data. For example, this may enable a user to browse a recording to select audio segments corresponding to a specific speaker, or “fast-forward” through a recording to the next speaker. In addition, knowing the ordering of speakers can also provide content clues about the conversation, or about the context of the conversation. [0014]
  • The ultimate goal of the segmentation is to produce a sequence of discrete segments with particular characteristics remaining constant within each one. The characteristics of choice depend on the overall structure of the indexation system. [0015]
  • Saunders, “Real-Time Discrimination of Broadcast Speech/Music”, Proc. ICASSP 1996, pp. 993-996, has described a speech/music discriminator based on zero-crossings. Its application is for discrimination between advertisements and programs in radio broadcasts. Since it is intended to be incorporated in consumer radios, it is intended to be low cost and simple. It is mainly designed to detect the characteristics of speech, which are described as, limited bandwidth, alternate voiced and unvoiced sections, limited range of pitch, syllabic duration of vowels, energy variations between high and low levels. It is indirectly using the amplitude, pitch and periodicity estimate of the waveform to carry out the detection process since zero-crossings give an estimate of the dominant frequency in the waveform. [0016]
  • Zue and Spina, “Automatic Transcription of General Audio Data: Preliminary Analyses”, Proc. ICSP 1996, pp. 594-597, use an average of the cepstral coefficients over a series of frames. This is shown to work well in distinguishing between speech and music when the speech is band-limited to 4 kHz and music to 16 kHz but less well when both signals occupied a 16 kHz bandwidth. [0017]
  • Scheier and Slaney, “Construction and Evalution of a Robust Multifeature Speech/Music Discriminator”, Proc. ICASSP 1997, pp. 1331-1334, use a variety of features. These are: four hertz modulation energy, low energy, roll off of the spectrum, the variance of the roll off of the spectrum, the spectral centroid, variance of the spectral centroid, the spectral flux, variance of the spectral flux, the zero-crossing rate, variance of the zero-crossing rate, the cepstral residual, variance of the cepstral residual, pulse metric. The first two features are amplitude related. The next six features are derived from the fine spectrum of the input signal and therefore are related to the techniques described in the previous reference. [0018]
  • Carey et al., “A Comparison of Features for Speech, Music Discrimination”, Proc. IEEE 1999, pp. 149-152, use a variety of features. There are: cepstral coefficients, delta cepstral coefficients, amplitude, delta amplitude, pitch, delta pitch, zero-crossing rate, delta zero-crossing rate. The pitch and cepstral coefficients encompass the fine and broad spectral features respectively. The zero-crossing parameters and the amplitude were believed worthy of investigation as a computationally inexpensive alternative to the other features. [0019]
  • SUMMARY OF THE INVENTION
  • The present invention provides a segmentation procedure to chunk an input audio stream into segments having homogeneous acoustic characteristics. This audio stream is a sequence of digital samples, which are broadcasted or recorded using some media. [0020]
  • An object of the invention is to provide a fast segmentation procedure with a relatively low numerical complexity. [0021]
  • The segmentation procedure comprises three stages. These are: first-grade characteristic calculation, second-grade characteristic calculation, and decision-making. The stage of first-grade characteristic calculation is aimed for calculation of audio features vectors from the input audio stream. These features vectors define characteristics of audio signals. The stage of second-grade characteristic calculation forms sequence of statistic features vectors from the sequence of audio features vectors. The statistic features vectors define statistic features of the first-grade features. The stage of decision-making analyses variation of the second grade features and performs definition of the segments boundaries basing on that analysis. [0022]
  • Thus, an essential aim of the invention is to provide the segmentation method, firstly, that can be used for a wide variety of applications, secondary, that the segmentation procedure may be industrial-scaled manufactured, based on the development of one relatively simple integrated circuit. [0023]
  • Other aspects of the present invention can be seen upon review of the figure, the detailed description and the claims, which follow.[0024]
  • BRIEF DESCRIPTION OF THE DRAWING
  • The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate an embodiment of the invention and together with the description serve to explain the principle of the invention. In the drawings: [0025]
  • FIG. 1 is a block diagram of a generalized audio processing system within which the present invention may be embodied; [0026]
  • FIG. 2 is a generalized flow diagram of the audio segmentation procedure; [0027]
  • FIG. 3 is a flow diagram in detail of the audio segmentation procedure; [0028]
  • FIG. 4 illustrates a flowchart of the sub-stage of initial segmentation; [0029]
  • FIG. 5 illustrates a flowchart of the sub-stage of accurate segmentation; [0030]
  • FIG. 6 shows improvement of the dividing markers positions; [0031]
  • FIG. 7 shows defining the homogeneous interval inside the segment.[0032]
  • DETAILED DESCRIPTION OF THE INVENTION
  • FIG. 1 is a block diagram of a generalized [0033] audio processing system 1, within which the present invention may be embodied. Generally, an audio stream is provided from a source of audio data 2, which may be provided by recorded broadcast, a recorded video with accompanying audio track, or other audio source. The audio data is sent to an audio processor 3, which may be any well-known device such as a general purpose computer, configured according to the present invention. The audio processor outputs segments of the audio data 4.
  • FIG. 2 is a generalized flow diagram of an [0034] audio segmentation procedure 5. Box 10 is the audio stream input, for example, broadcast news input. The step in box 20 is aimed for calculation of audio features vectors from the audio stream. These features vectors define characteristic features of the audio-stream. Next step 30 forms sequence of statistic features vectors from the sequence of audio features vectors. The statistic features vectors define statistic characteristic of the audio features vectors. At the step 40, variation of the statistic features vectors is analyzed and the definition of the segment boundaries basing on that analysis is performed. Thus, the proposed segmentation procedure is based on the analysis of audio signal statistical features variation. The output of the resulting index segmentation of the audio stream is performed at the step 50.
  • FIG. 3 is a flow diagram in detail of the audio segmentation procedure. After the input of [0035] audio stream data 10, an input sequence of the digital samples is divided into the sequence of short (e.g. 10-20 ms) not-overlapped frames 21. At the step 25, the feature vectors are computed for each frame. This computation is performed using 10th order Linear Predictive Coding (LPC) analysis of the samples in possibly overlapped windows, which contain said frames.
  • Parameters of the autoregressive linear model, which is the foundation of LPC analysis are reliable and may be defined with relatively small computation complexity. The following parameters form coordinates of audio features vector: [0036]
  • Λ[0037] i,i=1.5—Formants Frequencies;
  • K[0038] 1,K2—First and the Second Reflection Coefficients;
  • E[0039] 0—Energy of the Prediction Error Coefficient;
  • E[0040] 1—Preemphasized Energy Ratio Coefficient.
  • Parameters K[0041] 1,K2, E0 are calculated simultaneously with LPC analysis, according to Marple, Jr. “Digital Spectral Analysis”, Prentice-Hall, Inc., Englewood Cliffs, N.J., 1987. After LPC analysis, 10 coefficients Line Spectral Pairs (LSP) are computed according to the patent U.S. Pat. No. 4,393,272 or ITU-T, Study Group 15 Contribution--Q. 12/15, Draft Recommendation G.729, Jun. 8, 1995, Version 5.0. Λi,i=1.5 Formant Frequencies are calculated as half of sum of the corresponding LSP coefficients. E1 is the ratio of the energies in the 6-dB preemrhasized first-order difference audio signal to the regular audio signal, according to Campbell et al. “Voiced/Unvoiced Classification of Speech with Applications to the U.S.
  • Government LPC-10E Algorithm”, Proceedings ICASSP' 86, April, Tokyo, Japan, V.1, pp 473-476. [0042]
  • As the result, there are the audio feature vectors (9 characteristics at all). These vectors have definite physical meaning and the dynamical range sufficient for the precise segmentation of the audio stream. The further work of the segmentation procedure is the statistical analysis of the obtained data. The calculation of the statistical characteristics is performed in non-overlapped second-grade windows, each of these windows consists of some predefined number of frames (e.g. 20-100 frames in one window). Thus, some number of vectors of the first-grade characteristics describes such a window. The division of the input sequence of the audio feature vectors is performed at the [0043] step 31. At the step 35, the sequence of those vectors is transformed to the statistic feature vectors.
  • The statistical features vector {right arrow over (V)} consists of two sub-vectors, the first of them consists of: [0044] V j = 1 M i = 1 M M Λ i , j = 1 5 V j + 5 = 1 M i = 1 M ( Λ i - V j ) 2 , j = 1 5
    Figure US20030171936A1-20030911-M00001
  • and the second of these sub-vectors consists of: [0045] V 11 = ( max i = 1 M { K t 2 } - min i = 1 M { K t 2 } ) × 1 M i = 1 M K i 2 V 12 = 1 M i = 1 M E i 0 × 1 M i = 1 M ( E i 0 - 1 M i = 1 M E i 0 ) 2 V 13 = i = 2 M | E i 0 - E i - 1 0 | - i = 1 M | E i 0 | V 14 = max i = 1 M { E i 1 } - min i = 1 M { E i 1 } V 15 = i = 1 M B ( i ) , where B ( i ) = { 1 , K i 1 > predefined thresholds 0 , otherwise ,
    Figure US20030171936A1-20030911-M00002
  • where M is a number of frames in one window. [0046]
  • As the result, there are the statistic feature vectors (15 characteristics at all). [0047]
  • The sub-stages of the decision-making [0048] 40 will be discussed in more details below, but FIG. 3 serves to give an overview of the method described by the invention.
  • The sub-stage of [0049] initial segmentation 100 is performed in such a way that the dividing markers, which corresponds boundaries of segments, are determined with the accuracy corresponding to one second-grade window. The sub-stage of improvement of the segmentation precision 200 carried out by the previous step implies the correction of the position of each dividing marker with the accuracy corresponding to one frame and eliminating of false segments. The sub-stage of internal markers definition 300 implies the determination of a stationary interval inside each segment. The resulting sequence of the not intersected audio segments with their time boundaries is outputted at the step 50.
  • Sub-Stage of Initial Segmentation [0050]
  • FIG. 4 illustrates a flowchart of the sub-stage of [0051] initial segmentation 100 of FIG. 3. In this sub-stage, the statistical features vectors {right arrow over (V)}[k],k=1, . . . , K are analyzed. On each step, the algorithm of this sub-stage parses four sequential entry vectors. The result of the analysis is the information, where the dividing marker is placed.
  • Let {right arrow over (V)}[k],{right arrow over (V)}[k+1],{right arrow over (V)}[k+2],{right arrow over (V)}[k+3] four sequential statistical features vectors, which are taken [0052] 136 from the set of sequential statistical features vectors.
  • The differences A[0053] j i=|Vi[k+j]−Vi[k+j+1]|, j=0,1,2,i=1, . . . , 10
  • are calculated for the first sub-vectors of the [0054] statistical features vectors 137. If at least one of these values is greater than the corresponding predefined threshold 138, the dividing marker is installed between the second-range windows 139. In this case, another steps of this sub-stage does not performed and the next four vectors, first of which is the first vector after the installed dividing marker will be taken from the set of sequential statistical features vectors for analysis 148.
  • Otherwise the differences A[0055] i=|(Vi[k]+Vi[k+1])−(Vi[k+2]+Vi[k+3])|, i=11.15 are calculated 140 for the second sub-vectors of the statistical features vectors. These values are matched with the predefined thresholds 141. The case when all of these values are smaller than the corresponding threshold values corresponds to the absence of the dividing marker 142. In this case, the last steps of this sub-stage does not performed and the next four vectors, first of which is the vector {right arrow over (V)}[k+1] will be taken from the set of sequential statistical features vectors for analysis 148. Otherwise the differences Aj i=|Vi[k+j]−Vi[k+j+1]|, i=11.15, j=0,1,2 are calculated 143 for the second sub-vectors of the statistical features vectors. If at least one of these values is greater than the corresponding predefined thresholds 144 then the dividing marker is installed between the second-range windows 145. In this case, another steps of this sub-stage is not performed and the next four vectors, first of which is the first vector after the installed dividing marker will be taken from the set of sequential statistical features vectors 148. Otherwise the next four vectors, first of which is the vector {right arrow over (V)}[k+1] will be taken from the set of sequential statistical features vectors for analysis 148. If the dividing marker is taken at the step in diamond 147, then the sub-stage of initial segmentation ends and the initial segmentation marker passes to the sub-stage of accurate segmentation.
  • Sub-Stage of Accurate Segmentation [0056]
  • FIG. 5 illustrates a flowchart of the sub-stage of [0057] accurate segmentation 200 of FIG. 3. The sense of this given stage operation consists in an improvement of dividing markers positions. It is achieved as a result of a precise statistical analysis of the sequence of LSP coefficients Λi,i=1, . . . , 5 close to each dividing marker (see FIG. 6). Let's consider an arbitrary dividing marker μ with some neighborhood, which consists of n frames, close to Formants Frequencies coefficients. At the step in box 210, the difference is evaluated: S j = | 1 j + 1 p = 0 j i = 1 5 Λ i ( k + p ) - 1 n - j p = j + 1 n i = 1 5 Λ i ( k + p ) | , j = a , , n - a - 1 , a < n 2
    Figure US20030171936A1-20030911-M00003
  • Argument, which correspond to maximum value S[0058] j, is calculated at the step 220: J = arg max j = n , , n - n - 1 ( S j )
    Figure US20030171936A1-20030911-M00004
  • At the [0059] step 230, the new dividing marker μ is placed into the position corresponded to this J between shaded rectangles on FIG. 6. At the step in box 148 in FIG. 4, the shift of vectors is performed from the position of the new marker μ.
  • Sub-Stage of Internal Markers Definition [0060]
  • The sub-stage of internal markers definition of the final segmentation analyses each segment with the purpose of the definition of two internal markers (μ[0061] intint) defining the most homogeneous interval inside the segment. It is made with the following purposes: the placed dividing markers separate two audio events of the different nature. These events, as a rule, smoothly transiting one to another and do not have drastic border. Therefore there is a time interval containing information about both the events. That may hamper their correct classification.
  • As well as at the previous sub-stage, this task is solved by usage of a precise statistical analysis of a sequence of Formants Frequencies coefficients Λ[0062] i,i=1, . . . , 5 close to each dividing marker. Let's consider an arbitrary segment, limited by markers μ and η, (so that η−μ=n+1 frames), and composed from Formants Frequencies coefficients (see FIG. 7).
  • Firstly, two differences are evaluated: [0063] S 1 j = 1 j + 1 p = 0 j i = 1 5 Λ i ( k + p ) - 1 n / 2 - j p = j + 1 n / 2 i = 1 5 Λ l ( k + p ) | , j = a , , n / 2 - a - 1 , a < n 4 S 2 j = 1 j + 1 p = n / 2 j + n / 2 i = 1 5 Λ l ( k + p ) - 1 n / 2 - j p = j + 1 + n / 2 n l = 1 5 Λ i ( k + p ) | , j = a , , n / 2 - a - 1 , a < n 4
    Figure US20030171936A1-20030911-M00005
  • At the second, arguments, which correspond to maximum values S[0064] 1j and S2j, are calculated: J 1 = arg max j = a , , n / 2 - a - 1 ( S 1 j ) , J 2 = arg max j = a , , n / 2 - a - 1 ( S 2 j ) .
    Figure US20030171936A1-20030911-M00006
  • Then, the new markers μ[0065] int and ηint are placed into the positions corresponded to these J1, J2 between shaded rectangles on FIG. 7.
  • Thus, the process of segmentation is ended. As the result, the sequence of not intersected audio intervals with their time boundaries is obtained. [0066]

Claims (22)

What is claimed is:
1. A method of segmentation of an audio stream, wherein the segmentation is the division of the audio stream into segments containing different homogeneous signals.
2. The method according to claim 1, wherein the audio stream is a sequence of digital samples which are broadcasted or recorded using some media.
3. The method according to claim 1, wherein the audio stream segmentation is performed in three stages: the stage of the first-grade characteristic calculation, the stage of the second-grade characteristic calculation, and the stage of the decision-making analysis.
4. The method according to claim 3, wherein the stage of the first-grade characteristic calculation is performed by the division of the audio stream into frames for which of them the audio feature vector is calculated.
5. The method according to claim 4, wherein the audio feature vector consists of five formant frequencies, the first and the second reflection coefficients, the energy of the prediction error coefficient, and the preemphasized energy ratio coefficient.
6. The method according to claim 3, wherein the stage of the second-grade characteristic calculation is performed in the sequence of the predefined and not overlapped windows, each of them consists from definite number of said frames with said audio feature vectors calculated at the stage of the first-grade characteristic calculation.
7. The method according to claim 6, wherein the stage of the second-grade characteristic calculation consists in the calculation of the statistical feature vector for each said window.
8. The method according to claim 7, wherein the statistical feature vector consists from two sub-vectors, firsts of which consists from mean values of the formant frequencies, and dispersions of the formant frequencies, and the second of said sub-vectors consists from difference between maximal and minimal values of the second reflection coefficient multiplied by the mean value of the second reflection coefficient, product of the mean value and the dispersion of the energy of the prediction error coefficient, sum of the modules of differences between said energies of the prediction error coefficients for said neighboring frames divided by the sum of the modules of said energies of the prediction error coefficients, difference between maximal and minimal values of said preemphasized energy ratio coefficients, and number of said frames in the window in which the first reflection coefficients outnumber predefined a predefined threshold value.
9. The method according to claim 3, wherein the stage of the decision-making analysis is performed in three sub-stages: the sub-stage of initial segmentation, the sub-stage of accurate segmentation, and the sub-stage of the internal markers definition.
10. The method according to claim 9, wherein the sub-stage of initial segmentation is performed basing on the analysis of the four sequential statistical feature vectors to define where the dividing marker has to be placed.
11. The method according to claim 10, wherein said analysis of the four sequential statistical feature vectors is performed in three steps, first of which may signalize about installation of the dividing marker, in this case other steps do not performed, the second of these steps may signalize about the absence of the dividing marker, in this case the third of said steps does not performed, and the third of said steps signalize about absence or installation of the dividing marker.
12. The method according to claim 11, wherein the first of said steps includes calculation of the modules of differences between two sequential coordinates of the first said sub-vector of the statistical feature vectors, comparison of the calculated values with the predefined threshold values, and the installation of the dividing marker if at least one said modulo is greater than the corresponding threshold value
13. The method according to claim 11, wherein the second of said steps includes calculation of the modules of differences between sum of two sequential coordinates of the second said sub-vector of the statistical feature vectors and sum of next two sequential coordinates of the second said sub-vector of the statistical feature vectors, comparison of the calculated values with the predefined threshold values, and signalizing about the absence of the dividing marker in the case when all of said modules are smaller than the corresponding threshold values.
14. The method according to claim 11, wherein the third of said steps includes calculation of the modules of differences between two sequential coordinates of the second said sub-vector of the statistical feature vectors, comparison of the calculated values with the predefined threshold values, the installation of the dividing marker if at least one said modulo is greater than the corresponding threshold value, and signalizing about the absence of the dividing marker in the opposite case.
15. The method according to claim 9, wherein the sub-stage of accurate segmentation is performed basing on the results of the initial segmentation and the analysis of sequence of said formant frequencies calculated for frames close to the dividing marker.
16. The method according to claim 15, wherein the analysis is based on calculation of the set of values of modules of differences between sums of mean values of the formant frequencies, wherein each modulo is calculated for the two intervals close to the dividing marker.
17. The method according to claim 16, wherein the intervals are sequential not-overlapped intervals, each of them has one and only one fixed border, and the border between said intervals is varied when the calculation of the set of values of the modules is performed.
18. The method according to claim 16, wherein the result of the accurate segmentation is the new position of the dividing marker, which corresponds to the maximum value of the values of modules of differences between sums of mean values of said formant frequencies.
19. The method according to claim 9, wherein goal of the sub-stage of the internal markers definition is a definition of two internal markers, which determine a most homogeneous interval inside each segment that was received at the sub-stage of accurate segmentation.
20. The method according to claim 19, wherein the two internal markers correspond to left and right halves of the segment and are calculated independently using calculation of the set of values of modules of differences between sums of mean values of the formant frequencies for each half of the segment.
21. The method according to claim 20, wherein the values of modules of differences are calculated for two not-overlapped intervals, each of them covers corresponding half of said segment, and the border between these two intervals is varied when the calculation of the set of values of the modules is performed.
22. The method according to claim 21, wherein each of the internal markers corresponds to the maximum of the values of modules of differences.
US10/370,065 2002-02-21 2003-02-21 Method of segmenting an audio stream Expired - Fee Related US7346516B2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020020009209A KR20030070179A (en) 2002-02-21 2002-02-21 Method of the audio stream segmantation
KR2002/9209 2002-02-21

Publications (2)

Publication Number Publication Date
US20030171936A1 true US20030171936A1 (en) 2003-09-11
US7346516B2 US7346516B2 (en) 2008-03-18

Family

ID=29546250

Family Applications (1)

Application Number Title Priority Date Filing Date
US10/370,065 Expired - Fee Related US7346516B2 (en) 2002-02-21 2003-02-21 Method of segmenting an audio stream

Country Status (2)

Country Link
US (1) US7346516B2 (en)
KR (1) KR20030070179A (en)

Cited By (120)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050126369A1 (en) * 2003-12-12 2005-06-16 Nokia Corporation Automatic extraction of musical portions of an audio stream
US20080236368A1 (en) * 2007-03-26 2008-10-02 Sanyo Electric Co., Ltd. Recording or playback apparatus and musical piece detecting apparatus
US20110161074A1 (en) * 2009-12-29 2011-06-30 Apple Inc. Remote conferencing center
US20140207456A1 (en) * 2010-09-23 2014-07-24 Waveform Communications, Llc Waveform analysis of speech
US8879761B2 (en) 2011-11-22 2014-11-04 Apple Inc. Orientation-based audio
US8892446B2 (en) 2010-01-18 2014-11-18 Apple Inc. Service orchestration for intelligent automated assistant
US8977584B2 (en) 2010-01-25 2015-03-10 Newvaluexchange Global Ai Llp Apparatuses, methods and systems for a digital conversation management platform
US9262612B2 (en) 2011-03-21 2016-02-16 Apple Inc. Device access using voice authentication
US9300784B2 (en) 2013-06-13 2016-03-29 Apple Inc. System and method for emergency calls initiated by voice command
US9330720B2 (en) 2008-01-03 2016-05-03 Apple Inc. Methods and apparatus for altering audio output signals
US9338493B2 (en) 2014-06-30 2016-05-10 Apple Inc. Intelligent automated assistant for TV user interactions
US9368114B2 (en) 2013-03-14 2016-06-14 Apple Inc. Context-sensitive handling of interruptions
US9430463B2 (en) 2014-05-30 2016-08-30 Apple Inc. Exemplar-based natural language processing
US9483461B2 (en) 2012-03-06 2016-11-01 Apple Inc. Handling speech synthesis of content for multiple languages
US9495129B2 (en) 2012-06-29 2016-11-15 Apple Inc. Device, method, and user interface for voice-activated navigation and browsing of a document
US9502031B2 (en) 2014-05-27 2016-11-22 Apple Inc. Method for supporting dynamic grammars in WFST-based ASR
US9535906B2 (en) 2008-07-31 2017-01-03 Apple Inc. Mobile device having human language translation capability with positional feedback
US9576574B2 (en) 2012-09-10 2017-02-21 Apple Inc. Context-sensitive handling of interruptions by intelligent digital assistant
US9582608B2 (en) 2013-06-07 2017-02-28 Apple Inc. Unified ranking with entropy-weighted information for phrase-based semantic auto-completion
US9620104B2 (en) 2013-06-07 2017-04-11 Apple Inc. System and method for user-specified pronunciation of words for speech synthesis and recognition
US9620105B2 (en) 2014-05-15 2017-04-11 Apple Inc. Analyzing audio input for efficient speech and music recognition
US9626955B2 (en) 2008-04-05 2017-04-18 Apple Inc. Intelligent text-to-speech conversion
US9633674B2 (en) 2013-06-07 2017-04-25 Apple Inc. System and method for detecting errors in interactions with a voice-based digital assistant
US9633004B2 (en) 2014-05-30 2017-04-25 Apple Inc. Better resolution when referencing to concepts
US9633660B2 (en) 2010-02-25 2017-04-25 Apple Inc. User profiling for voice input processing
US9646609B2 (en) 2014-09-30 2017-05-09 Apple Inc. Caching apparatus for serving phonetic pronunciations
US9646614B2 (en) 2000-03-16 2017-05-09 Apple Inc. Fast, language-independent method for user authentication by voice
US9668121B2 (en) 2014-09-30 2017-05-30 Apple Inc. Social reminders
US9697820B2 (en) 2015-09-24 2017-07-04 Apple Inc. Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks
US9697822B1 (en) 2013-03-15 2017-07-04 Apple Inc. System and method for updating an adaptive speech recognition model
US9711141B2 (en) 2014-12-09 2017-07-18 Apple Inc. Disambiguating heteronyms in speech synthesis
US9715875B2 (en) 2014-05-30 2017-07-25 Apple Inc. Reducing the need for manual start/end-pointing and trigger phrases
US9721566B2 (en) 2015-03-08 2017-08-01 Apple Inc. Competing devices responding to voice triggers
US9734193B2 (en) 2014-05-30 2017-08-15 Apple Inc. Determining domain salience ranking from ambiguous words in natural speech
US9760559B2 (en) 2014-05-30 2017-09-12 Apple Inc. Predictive text input
US9785630B2 (en) 2014-05-30 2017-10-10 Apple Inc. Text prediction using combined word N-gram and unigram language models
US9798393B2 (en) 2011-08-29 2017-10-24 Apple Inc. Text correction processing
US9818400B2 (en) 2014-09-11 2017-11-14 Apple Inc. Method and apparatus for discovering trending terms in speech requests
US9842101B2 (en) 2014-05-30 2017-12-12 Apple Inc. Predictive conversion of language input
US9842105B2 (en) 2015-04-16 2017-12-12 Apple Inc. Parsimonious continuous-space phrase representations for natural language processing
US9858925B2 (en) 2009-06-05 2018-01-02 Apple Inc. Using context information to facilitate processing of commands in a virtual assistant
US9865280B2 (en) 2015-03-06 2018-01-09 Apple Inc. Structured dictation using intelligent automated assistants
US9886953B2 (en) 2015-03-08 2018-02-06 Apple Inc. Virtual assistant activation
US9886432B2 (en) 2014-09-30 2018-02-06 Apple Inc. Parsimonious handling of word inflection via categorical stem + suffix N-gram language models
US9899019B2 (en) 2015-03-18 2018-02-20 Apple Inc. Systems and methods for structured stem and suffix language models
US9922642B2 (en) 2013-03-15 2018-03-20 Apple Inc. Training an at least partial voice command system
US9934775B2 (en) 2016-05-26 2018-04-03 Apple Inc. Unit-selection text-to-speech synthesis based on predicted concatenation parameters
US9953088B2 (en) 2012-05-14 2018-04-24 Apple Inc. Crowd sourcing information to fulfill user requests
US9959870B2 (en) 2008-12-11 2018-05-01 Apple Inc. Speech recognition involving a mobile device
US9966068B2 (en) 2013-06-08 2018-05-08 Apple Inc. Interpreting and acting upon commands that involve sharing information with remote devices
US9966065B2 (en) 2014-05-30 2018-05-08 Apple Inc. Multi-command single utterance input method
US9971774B2 (en) 2012-09-19 2018-05-15 Apple Inc. Voice-based media searching
US9972304B2 (en) 2016-06-03 2018-05-15 Apple Inc. Privacy preserving distributed evaluation framework for embedded personalized systems
US10043516B2 (en) 2016-09-23 2018-08-07 Apple Inc. Intelligent automated assistant
US10049663B2 (en) 2016-06-08 2018-08-14 Apple, Inc. Intelligent automated assistant for media exploration
US10049668B2 (en) 2015-12-02 2018-08-14 Apple Inc. Applying neural network language models to weighted finite state transducers for automatic speech recognition
US10057736B2 (en) 2011-06-03 2018-08-21 Apple Inc. Active transport based notifications
US10067938B2 (en) 2016-06-10 2018-09-04 Apple Inc. Multilingual word prediction
US10074360B2 (en) 2014-09-30 2018-09-11 Apple Inc. Providing an indication of the suitability of speech recognition
US10078631B2 (en) 2014-05-30 2018-09-18 Apple Inc. Entropy-guided text prediction using combined word and character n-gram language models
US10079014B2 (en) 2012-06-08 2018-09-18 Apple Inc. Name recognition system
US10083688B2 (en) 2015-05-27 2018-09-25 Apple Inc. Device voice control for selecting a displayed affordance
US10089072B2 (en) 2016-06-11 2018-10-02 Apple Inc. Intelligent device arbitration and control
US10101822B2 (en) 2015-06-05 2018-10-16 Apple Inc. Language input correction
US10127220B2 (en) 2015-06-04 2018-11-13 Apple Inc. Language identification from short strings
US10127911B2 (en) 2014-09-30 2018-11-13 Apple Inc. Speaker identification and unsupervised speaker adaptation techniques
US10134385B2 (en) 2012-03-02 2018-11-20 Apple Inc. Systems and methods for name pronunciation
US10170123B2 (en) 2014-05-30 2019-01-01 Apple Inc. Intelligent assistant for home automation
US10176167B2 (en) 2013-06-09 2019-01-08 Apple Inc. System and method for inferring user intent from speech inputs
US10186254B2 (en) 2015-06-07 2019-01-22 Apple Inc. Context-based endpoint detection
US10185542B2 (en) 2013-06-09 2019-01-22 Apple Inc. Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant
US10192552B2 (en) 2016-06-10 2019-01-29 Apple Inc. Digital assistant providing whispered speech
US10199051B2 (en) 2013-02-07 2019-02-05 Apple Inc. Voice trigger for a digital assistant
US10223066B2 (en) 2015-12-23 2019-03-05 Apple Inc. Proactive assistance based on dialog communication between devices
US10241644B2 (en) 2011-06-03 2019-03-26 Apple Inc. Actionable reminder entries
US10241752B2 (en) 2011-09-30 2019-03-26 Apple Inc. Interface for a virtual digital assistant
US10249300B2 (en) 2016-06-06 2019-04-02 Apple Inc. Intelligent list reading
US10255907B2 (en) 2015-06-07 2019-04-09 Apple Inc. Automatic accent detection using acoustic models
US10269345B2 (en) 2016-06-11 2019-04-23 Apple Inc. Intelligent task discovery
US10276170B2 (en) 2010-01-18 2019-04-30 Apple Inc. Intelligent automated assistant
US10283110B2 (en) 2009-07-02 2019-05-07 Apple Inc. Methods and apparatuses for automatic speech recognition
US10289433B2 (en) 2014-05-30 2019-05-14 Apple Inc. Domain specific language for encoding assistant dialog
US10297253B2 (en) 2016-06-11 2019-05-21 Apple Inc. Application integration with a digital assistant
US10318871B2 (en) 2005-09-08 2019-06-11 Apple Inc. Method and apparatus for building an intelligent automated assistant
US10356243B2 (en) 2015-06-05 2019-07-16 Apple Inc. Virtual assistant aided communication with 3rd party service in a communication session
US10354011B2 (en) 2016-06-09 2019-07-16 Apple Inc. Intelligent automated assistant in a home environment
US10366158B2 (en) 2015-09-29 2019-07-30 Apple Inc. Efficient word encoding for recurrent neural network language models
US10410637B2 (en) 2017-05-12 2019-09-10 Apple Inc. User-specific acoustic models
US10446141B2 (en) 2014-08-28 2019-10-15 Apple Inc. Automatic speech recognition based on user feedback
US10446143B2 (en) 2016-03-14 2019-10-15 Apple Inc. Identification of voice inputs providing credentials
US10482874B2 (en) 2017-05-15 2019-11-19 Apple Inc. Hierarchical belief states for digital assistants
US10490187B2 (en) 2016-06-10 2019-11-26 Apple Inc. Digital assistant providing automated status report
US10496753B2 (en) 2010-01-18 2019-12-03 Apple Inc. Automatically adapting user interfaces for hands-free interaction
US10509862B2 (en) 2016-06-10 2019-12-17 Apple Inc. Dynamic phrase expansion of language input
US10521466B2 (en) 2016-06-11 2019-12-31 Apple Inc. Data driven natural language event detection and classification
US10552013B2 (en) 2014-12-02 2020-02-04 Apple Inc. Data detection
US10553209B2 (en) 2010-01-18 2020-02-04 Apple Inc. Systems and methods for hands-free notification summaries
US10568032B2 (en) 2007-04-03 2020-02-18 Apple Inc. Method and system for operating a multi-function portable electronic device using voice-activation
US10567477B2 (en) 2015-03-08 2020-02-18 Apple Inc. Virtual assistant continuity
US10592095B2 (en) 2014-05-23 2020-03-17 Apple Inc. Instantaneous speaking of content on touch devices
US10593346B2 (en) 2016-12-22 2020-03-17 Apple Inc. Rank-reduced token representation for automatic speech recognition
US10659851B2 (en) 2014-06-30 2020-05-19 Apple Inc. Real-time digital assistant knowledge updates
US10671428B2 (en) 2015-09-08 2020-06-02 Apple Inc. Distributed personal assistant
US10679605B2 (en) 2010-01-18 2020-06-09 Apple Inc. Hands-free list-reading by intelligent automated assistant
US10691473B2 (en) 2015-11-06 2020-06-23 Apple Inc. Intelligent automated assistant in a messaging environment
US10705794B2 (en) 2010-01-18 2020-07-07 Apple Inc. Automatically adapting user interfaces for hands-free interaction
US10706373B2 (en) 2011-06-03 2020-07-07 Apple Inc. Performing actions associated with task items that represent tasks to perform
US10733993B2 (en) 2016-06-10 2020-08-04 Apple Inc. Intelligent digital assistant in a multi-tasking environment
US10747498B2 (en) 2015-09-08 2020-08-18 Apple Inc. Zero latency digital assistant
US10755703B2 (en) 2017-05-11 2020-08-25 Apple Inc. Offline personal assistant
US10762293B2 (en) 2010-12-22 2020-09-01 Apple Inc. Using parts-of-speech tagging and named entity recognition for spelling correction
US10791176B2 (en) 2017-05-12 2020-09-29 Apple Inc. Synchronization and task delegation of a digital assistant
US10789041B2 (en) 2014-09-12 2020-09-29 Apple Inc. Dynamic thresholds for always listening speech trigger
US10791216B2 (en) 2013-08-06 2020-09-29 Apple Inc. Auto-activating smart responses based on activities from remote devices
US10810274B2 (en) 2017-05-15 2020-10-20 Apple Inc. Optimizing dialogue policy decisions for digital assistants using implicit feedback
US11010550B2 (en) 2015-09-29 2021-05-18 Apple Inc. Unified language modeling framework for word prediction, auto-completion and auto-correction
US11025565B2 (en) 2015-06-07 2021-06-01 Apple Inc. Personalized prediction of responses for instant messaging
US11217255B2 (en) 2017-05-16 2022-01-04 Apple Inc. Far-field extension for digital assistant services
US11587559B2 (en) 2015-09-30 2023-02-21 Apple Inc. Intelligent device identification
US11894865B2 (en) * 2013-11-07 2024-02-06 Telefonaktiebolaget Lm Ericsson (Publ) Methods and devices for vector segmentation for coding

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8838452B2 (en) * 2004-06-09 2014-09-16 Canon Kabushiki Kaisha Effective audio segmentation and classification
CA2536976A1 (en) * 2006-02-20 2007-08-20 Diaphonics, Inc. Method and apparatus for detecting speaker change in a voice transaction
JP4665836B2 (en) * 2006-05-31 2011-04-06 日本ビクター株式会社 Music classification device, music classification method, and music classification program
US7680657B2 (en) * 2006-08-15 2010-03-16 Microsoft Corporation Auto segmentation based partitioning and clustering approach to robust endpointing
KR200449646Y1 (en) * 2007-12-17 2010-07-28 비엔지 주식회사 A prefabricated pillar for structure
CN102044244B (en) * 2009-10-15 2011-11-16 华为技术有限公司 Signal classifying method and device
GB2489489B (en) * 2011-03-30 2013-08-21 Toshiba Res Europ Ltd A speech processing system and method
US9535450B2 (en) 2011-07-17 2017-01-03 International Business Machines Corporation Synchronization of data streams with associated metadata streams using smallest sum of absolute differences between time indices of data events and metadata events
US8798996B2 (en) * 2012-03-05 2014-08-05 Coupons.Com Incorporated Splitting term lists recognized from speech
CN105632503B (en) * 2014-10-28 2019-09-03 南宁富桂精密工业有限公司 Information concealing method and system

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6567775B1 (en) * 2000-04-26 2003-05-20 International Business Machines Corporation Fusion of audio and video based speaker identification for multimedia information access
US6931373B1 (en) * 2001-02-13 2005-08-16 Hughes Electronics Corporation Prototype waveform phase modeling for a frequency domain interpolative speech codec system

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS5853352B2 (en) 1979-10-03 1983-11-29 日本電信電話株式会社 speech synthesizer
US5606643A (en) 1994-04-12 1997-02-25 Xerox Corporation Real-time audio recording system for automatic speaker indexing
US5655058A (en) * 1994-04-12 1997-08-05 Xerox Corporation Segmentation of audio data for indexing of conversational speech for real-time or postprocessing applications
KR19990035846U (en) * 1998-02-10 1999-09-15 구자홍 Position and posture adjuster of audio / control head for videocassette recorder
US6185527B1 (en) 1999-01-19 2001-02-06 International Business Machines Corporation System and method for automatic audio content analysis for word spotting, indexing, classification and retrieval
US7120575B2 (en) * 2000-04-08 2006-10-10 International Business Machines Corporation Method and system for the automatic segmentation of an audio stream into semantic or syntactic units

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6567775B1 (en) * 2000-04-26 2003-05-20 International Business Machines Corporation Fusion of audio and video based speaker identification for multimedia information access
US6931373B1 (en) * 2001-02-13 2005-08-16 Hughes Electronics Corporation Prototype waveform phase modeling for a frequency domain interpolative speech codec system

Cited By (169)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9646614B2 (en) 2000-03-16 2017-05-09 Apple Inc. Fast, language-independent method for user authentication by voice
US20050126369A1 (en) * 2003-12-12 2005-06-16 Nokia Corporation Automatic extraction of musical portions of an audio stream
WO2005060337A3 (en) * 2003-12-12 2006-09-21 Nokia Corp Automatic extraction of musical portions of an audio stream
US7179980B2 (en) * 2003-12-12 2007-02-20 Nokia Corporation Automatic extraction of musical portions of an audio stream
KR100840745B1 (en) * 2003-12-12 2008-06-23 노키아 코포레이션 Method and Apparatus for selectively recording music portions of audio stream
US10318871B2 (en) 2005-09-08 2019-06-11 Apple Inc. Method and apparatus for building an intelligent automated assistant
US9117447B2 (en) 2006-09-08 2015-08-25 Apple Inc. Using event alert text as input to an automated assistant
US8930191B2 (en) 2006-09-08 2015-01-06 Apple Inc. Paraphrasing of user requests and results by automated digital assistant
US8942986B2 (en) 2006-09-08 2015-01-27 Apple Inc. Determining user intent based on ontologies of domains
US20080236368A1 (en) * 2007-03-26 2008-10-02 Sanyo Electric Co., Ltd. Recording or playback apparatus and musical piece detecting apparatus
US7745714B2 (en) * 2007-03-26 2010-06-29 Sanyo Electric Co., Ltd. Recording or playback apparatus and musical piece detecting apparatus
US10568032B2 (en) 2007-04-03 2020-02-18 Apple Inc. Method and system for operating a multi-function portable electronic device using voice-activation
US9330720B2 (en) 2008-01-03 2016-05-03 Apple Inc. Methods and apparatus for altering audio output signals
US10381016B2 (en) 2008-01-03 2019-08-13 Apple Inc. Methods and apparatus for altering audio output signals
US9865248B2 (en) 2008-04-05 2018-01-09 Apple Inc. Intelligent text-to-speech conversion
US9626955B2 (en) 2008-04-05 2017-04-18 Apple Inc. Intelligent text-to-speech conversion
US10108612B2 (en) 2008-07-31 2018-10-23 Apple Inc. Mobile device having human language translation capability with positional feedback
US9535906B2 (en) 2008-07-31 2017-01-03 Apple Inc. Mobile device having human language translation capability with positional feedback
US9959870B2 (en) 2008-12-11 2018-05-01 Apple Inc. Speech recognition involving a mobile device
US9858925B2 (en) 2009-06-05 2018-01-02 Apple Inc. Using context information to facilitate processing of commands in a virtual assistant
US11080012B2 (en) 2009-06-05 2021-08-03 Apple Inc. Interface for a virtual digital assistant
US10475446B2 (en) 2009-06-05 2019-11-12 Apple Inc. Using context information to facilitate processing of commands in a virtual assistant
US10795541B2 (en) 2009-06-05 2020-10-06 Apple Inc. Intelligent organization of tasks items
US10283110B2 (en) 2009-07-02 2019-05-07 Apple Inc. Methods and apparatuses for automatic speech recognition
US8560309B2 (en) * 2009-12-29 2013-10-15 Apple Inc. Remote conferencing center
US20110161074A1 (en) * 2009-12-29 2011-06-30 Apple Inc. Remote conferencing center
US8903716B2 (en) 2010-01-18 2014-12-02 Apple Inc. Personalized vocabulary for digital assistant
US10276170B2 (en) 2010-01-18 2019-04-30 Apple Inc. Intelligent automated assistant
US10706841B2 (en) 2010-01-18 2020-07-07 Apple Inc. Task flow identification based on user intent
US12087308B2 (en) 2010-01-18 2024-09-10 Apple Inc. Intelligent automated assistant
US9548050B2 (en) 2010-01-18 2017-01-17 Apple Inc. Intelligent automated assistant
US10553209B2 (en) 2010-01-18 2020-02-04 Apple Inc. Systems and methods for hands-free notification summaries
US10496753B2 (en) 2010-01-18 2019-12-03 Apple Inc. Automatically adapting user interfaces for hands-free interaction
US10705794B2 (en) 2010-01-18 2020-07-07 Apple Inc. Automatically adapting user interfaces for hands-free interaction
US8892446B2 (en) 2010-01-18 2014-11-18 Apple Inc. Service orchestration for intelligent automated assistant
US11423886B2 (en) 2010-01-18 2022-08-23 Apple Inc. Task flow identification based on user intent
US10679605B2 (en) 2010-01-18 2020-06-09 Apple Inc. Hands-free list-reading by intelligent automated assistant
US9318108B2 (en) 2010-01-18 2016-04-19 Apple Inc. Intelligent automated assistant
US9424862B2 (en) 2010-01-25 2016-08-23 Newvaluexchange Ltd Apparatuses, methods and systems for a digital conversation management platform
US8977584B2 (en) 2010-01-25 2015-03-10 Newvaluexchange Global Ai Llp Apparatuses, methods and systems for a digital conversation management platform
US9424861B2 (en) 2010-01-25 2016-08-23 Newvaluexchange Ltd Apparatuses, methods and systems for a digital conversation management platform
US9431028B2 (en) 2010-01-25 2016-08-30 Newvaluexchange Ltd Apparatuses, methods and systems for a digital conversation management platform
US9633660B2 (en) 2010-02-25 2017-04-25 Apple Inc. User profiling for voice input processing
US10049675B2 (en) 2010-02-25 2018-08-14 Apple Inc. User profiling for voice input processing
US20140207456A1 (en) * 2010-09-23 2014-07-24 Waveform Communications, Llc Waveform analysis of speech
US10762293B2 (en) 2010-12-22 2020-09-01 Apple Inc. Using parts-of-speech tagging and named entity recognition for spelling correction
US10102359B2 (en) 2011-03-21 2018-10-16 Apple Inc. Device access using voice authentication
US9262612B2 (en) 2011-03-21 2016-02-16 Apple Inc. Device access using voice authentication
US10241644B2 (en) 2011-06-03 2019-03-26 Apple Inc. Actionable reminder entries
US10057736B2 (en) 2011-06-03 2018-08-21 Apple Inc. Active transport based notifications
US10706373B2 (en) 2011-06-03 2020-07-07 Apple Inc. Performing actions associated with task items that represent tasks to perform
US11120372B2 (en) 2011-06-03 2021-09-14 Apple Inc. Performing actions associated with task items that represent tasks to perform
US9798393B2 (en) 2011-08-29 2017-10-24 Apple Inc. Text correction processing
US10241752B2 (en) 2011-09-30 2019-03-26 Apple Inc. Interface for a virtual digital assistant
US10284951B2 (en) 2011-11-22 2019-05-07 Apple Inc. Orientation-based audio
US8879761B2 (en) 2011-11-22 2014-11-04 Apple Inc. Orientation-based audio
US10134385B2 (en) 2012-03-02 2018-11-20 Apple Inc. Systems and methods for name pronunciation
US9483461B2 (en) 2012-03-06 2016-11-01 Apple Inc. Handling speech synthesis of content for multiple languages
US9953088B2 (en) 2012-05-14 2018-04-24 Apple Inc. Crowd sourcing information to fulfill user requests
US10079014B2 (en) 2012-06-08 2018-09-18 Apple Inc. Name recognition system
US9495129B2 (en) 2012-06-29 2016-11-15 Apple Inc. Device, method, and user interface for voice-activated navigation and browsing of a document
US9576574B2 (en) 2012-09-10 2017-02-21 Apple Inc. Context-sensitive handling of interruptions by intelligent digital assistant
US9971774B2 (en) 2012-09-19 2018-05-15 Apple Inc. Voice-based media searching
US10199051B2 (en) 2013-02-07 2019-02-05 Apple Inc. Voice trigger for a digital assistant
US10978090B2 (en) 2013-02-07 2021-04-13 Apple Inc. Voice trigger for a digital assistant
US9368114B2 (en) 2013-03-14 2016-06-14 Apple Inc. Context-sensitive handling of interruptions
US9922642B2 (en) 2013-03-15 2018-03-20 Apple Inc. Training an at least partial voice command system
US9697822B1 (en) 2013-03-15 2017-07-04 Apple Inc. System and method for updating an adaptive speech recognition model
US9582608B2 (en) 2013-06-07 2017-02-28 Apple Inc. Unified ranking with entropy-weighted information for phrase-based semantic auto-completion
US9633674B2 (en) 2013-06-07 2017-04-25 Apple Inc. System and method for detecting errors in interactions with a voice-based digital assistant
US9966060B2 (en) 2013-06-07 2018-05-08 Apple Inc. System and method for user-specified pronunciation of words for speech synthesis and recognition
US9620104B2 (en) 2013-06-07 2017-04-11 Apple Inc. System and method for user-specified pronunciation of words for speech synthesis and recognition
US10657961B2 (en) 2013-06-08 2020-05-19 Apple Inc. Interpreting and acting upon commands that involve sharing information with remote devices
US9966068B2 (en) 2013-06-08 2018-05-08 Apple Inc. Interpreting and acting upon commands that involve sharing information with remote devices
US10176167B2 (en) 2013-06-09 2019-01-08 Apple Inc. System and method for inferring user intent from speech inputs
US10185542B2 (en) 2013-06-09 2019-01-22 Apple Inc. Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant
US9300784B2 (en) 2013-06-13 2016-03-29 Apple Inc. System and method for emergency calls initiated by voice command
US10791216B2 (en) 2013-08-06 2020-09-29 Apple Inc. Auto-activating smart responses based on activities from remote devices
US11894865B2 (en) * 2013-11-07 2024-02-06 Telefonaktiebolaget Lm Ericsson (Publ) Methods and devices for vector segmentation for coding
US9620105B2 (en) 2014-05-15 2017-04-11 Apple Inc. Analyzing audio input for efficient speech and music recognition
US10592095B2 (en) 2014-05-23 2020-03-17 Apple Inc. Instantaneous speaking of content on touch devices
US9502031B2 (en) 2014-05-27 2016-11-22 Apple Inc. Method for supporting dynamic grammars in WFST-based ASR
US9633004B2 (en) 2014-05-30 2017-04-25 Apple Inc. Better resolution when referencing to concepts
US9715875B2 (en) 2014-05-30 2017-07-25 Apple Inc. Reducing the need for manual start/end-pointing and trigger phrases
US10289433B2 (en) 2014-05-30 2019-05-14 Apple Inc. Domain specific language for encoding assistant dialog
US11257504B2 (en) 2014-05-30 2022-02-22 Apple Inc. Intelligent assistant for home automation
US10083690B2 (en) 2014-05-30 2018-09-25 Apple Inc. Better resolution when referencing to concepts
US11133008B2 (en) 2014-05-30 2021-09-28 Apple Inc. Reducing the need for manual start/end-pointing and trigger phrases
US10078631B2 (en) 2014-05-30 2018-09-18 Apple Inc. Entropy-guided text prediction using combined word and character n-gram language models
US10170123B2 (en) 2014-05-30 2019-01-01 Apple Inc. Intelligent assistant for home automation
US10169329B2 (en) 2014-05-30 2019-01-01 Apple Inc. Exemplar-based natural language processing
US9430463B2 (en) 2014-05-30 2016-08-30 Apple Inc. Exemplar-based natural language processing
US9842101B2 (en) 2014-05-30 2017-12-12 Apple Inc. Predictive conversion of language input
US9734193B2 (en) 2014-05-30 2017-08-15 Apple Inc. Determining domain salience ranking from ambiguous words in natural speech
US9966065B2 (en) 2014-05-30 2018-05-08 Apple Inc. Multi-command single utterance input method
US9760559B2 (en) 2014-05-30 2017-09-12 Apple Inc. Predictive text input
US10497365B2 (en) 2014-05-30 2019-12-03 Apple Inc. Multi-command single utterance input method
US9785630B2 (en) 2014-05-30 2017-10-10 Apple Inc. Text prediction using combined word N-gram and unigram language models
US9338493B2 (en) 2014-06-30 2016-05-10 Apple Inc. Intelligent automated assistant for TV user interactions
US9668024B2 (en) 2014-06-30 2017-05-30 Apple Inc. Intelligent automated assistant for TV user interactions
US10904611B2 (en) 2014-06-30 2021-01-26 Apple Inc. Intelligent automated assistant for TV user interactions
US10659851B2 (en) 2014-06-30 2020-05-19 Apple Inc. Real-time digital assistant knowledge updates
US10446141B2 (en) 2014-08-28 2019-10-15 Apple Inc. Automatic speech recognition based on user feedback
US9818400B2 (en) 2014-09-11 2017-11-14 Apple Inc. Method and apparatus for discovering trending terms in speech requests
US10431204B2 (en) 2014-09-11 2019-10-01 Apple Inc. Method and apparatus for discovering trending terms in speech requests
US10789041B2 (en) 2014-09-12 2020-09-29 Apple Inc. Dynamic thresholds for always listening speech trigger
US10074360B2 (en) 2014-09-30 2018-09-11 Apple Inc. Providing an indication of the suitability of speech recognition
US9668121B2 (en) 2014-09-30 2017-05-30 Apple Inc. Social reminders
US10127911B2 (en) 2014-09-30 2018-11-13 Apple Inc. Speaker identification and unsupervised speaker adaptation techniques
US9886432B2 (en) 2014-09-30 2018-02-06 Apple Inc. Parsimonious handling of word inflection via categorical stem + suffix N-gram language models
US9986419B2 (en) 2014-09-30 2018-05-29 Apple Inc. Social reminders
US9646609B2 (en) 2014-09-30 2017-05-09 Apple Inc. Caching apparatus for serving phonetic pronunciations
US11556230B2 (en) 2014-12-02 2023-01-17 Apple Inc. Data detection
US10552013B2 (en) 2014-12-02 2020-02-04 Apple Inc. Data detection
US9711141B2 (en) 2014-12-09 2017-07-18 Apple Inc. Disambiguating heteronyms in speech synthesis
US9865280B2 (en) 2015-03-06 2018-01-09 Apple Inc. Structured dictation using intelligent automated assistants
US9721566B2 (en) 2015-03-08 2017-08-01 Apple Inc. Competing devices responding to voice triggers
US10311871B2 (en) 2015-03-08 2019-06-04 Apple Inc. Competing devices responding to voice triggers
US11087759B2 (en) 2015-03-08 2021-08-10 Apple Inc. Virtual assistant activation
US9886953B2 (en) 2015-03-08 2018-02-06 Apple Inc. Virtual assistant activation
US10567477B2 (en) 2015-03-08 2020-02-18 Apple Inc. Virtual assistant continuity
US9899019B2 (en) 2015-03-18 2018-02-20 Apple Inc. Systems and methods for structured stem and suffix language models
US9842105B2 (en) 2015-04-16 2017-12-12 Apple Inc. Parsimonious continuous-space phrase representations for natural language processing
US10083688B2 (en) 2015-05-27 2018-09-25 Apple Inc. Device voice control for selecting a displayed affordance
US10127220B2 (en) 2015-06-04 2018-11-13 Apple Inc. Language identification from short strings
US10101822B2 (en) 2015-06-05 2018-10-16 Apple Inc. Language input correction
US10356243B2 (en) 2015-06-05 2019-07-16 Apple Inc. Virtual assistant aided communication with 3rd party service in a communication session
US10186254B2 (en) 2015-06-07 2019-01-22 Apple Inc. Context-based endpoint detection
US11025565B2 (en) 2015-06-07 2021-06-01 Apple Inc. Personalized prediction of responses for instant messaging
US10255907B2 (en) 2015-06-07 2019-04-09 Apple Inc. Automatic accent detection using acoustic models
US11500672B2 (en) 2015-09-08 2022-11-15 Apple Inc. Distributed personal assistant
US10671428B2 (en) 2015-09-08 2020-06-02 Apple Inc. Distributed personal assistant
US10747498B2 (en) 2015-09-08 2020-08-18 Apple Inc. Zero latency digital assistant
US9697820B2 (en) 2015-09-24 2017-07-04 Apple Inc. Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks
US11010550B2 (en) 2015-09-29 2021-05-18 Apple Inc. Unified language modeling framework for word prediction, auto-completion and auto-correction
US10366158B2 (en) 2015-09-29 2019-07-30 Apple Inc. Efficient word encoding for recurrent neural network language models
US11587559B2 (en) 2015-09-30 2023-02-21 Apple Inc. Intelligent device identification
US10691473B2 (en) 2015-11-06 2020-06-23 Apple Inc. Intelligent automated assistant in a messaging environment
US11526368B2 (en) 2015-11-06 2022-12-13 Apple Inc. Intelligent automated assistant in a messaging environment
US10049668B2 (en) 2015-12-02 2018-08-14 Apple Inc. Applying neural network language models to weighted finite state transducers for automatic speech recognition
US10223066B2 (en) 2015-12-23 2019-03-05 Apple Inc. Proactive assistance based on dialog communication between devices
US10446143B2 (en) 2016-03-14 2019-10-15 Apple Inc. Identification of voice inputs providing credentials
US9934775B2 (en) 2016-05-26 2018-04-03 Apple Inc. Unit-selection text-to-speech synthesis based on predicted concatenation parameters
US9972304B2 (en) 2016-06-03 2018-05-15 Apple Inc. Privacy preserving distributed evaluation framework for embedded personalized systems
US10249300B2 (en) 2016-06-06 2019-04-02 Apple Inc. Intelligent list reading
US11069347B2 (en) 2016-06-08 2021-07-20 Apple Inc. Intelligent automated assistant for media exploration
US10049663B2 (en) 2016-06-08 2018-08-14 Apple, Inc. Intelligent automated assistant for media exploration
US10354011B2 (en) 2016-06-09 2019-07-16 Apple Inc. Intelligent automated assistant in a home environment
US11037565B2 (en) 2016-06-10 2021-06-15 Apple Inc. Intelligent digital assistant in a multi-tasking environment
US10733993B2 (en) 2016-06-10 2020-08-04 Apple Inc. Intelligent digital assistant in a multi-tasking environment
US10509862B2 (en) 2016-06-10 2019-12-17 Apple Inc. Dynamic phrase expansion of language input
US10067938B2 (en) 2016-06-10 2018-09-04 Apple Inc. Multilingual word prediction
US10490187B2 (en) 2016-06-10 2019-11-26 Apple Inc. Digital assistant providing automated status report
US10192552B2 (en) 2016-06-10 2019-01-29 Apple Inc. Digital assistant providing whispered speech
US10297253B2 (en) 2016-06-11 2019-05-21 Apple Inc. Application integration with a digital assistant
US10269345B2 (en) 2016-06-11 2019-04-23 Apple Inc. Intelligent task discovery
US10089072B2 (en) 2016-06-11 2018-10-02 Apple Inc. Intelligent device arbitration and control
US10521466B2 (en) 2016-06-11 2019-12-31 Apple Inc. Data driven natural language event detection and classification
US11152002B2 (en) 2016-06-11 2021-10-19 Apple Inc. Application integration with a digital assistant
US10553215B2 (en) 2016-09-23 2020-02-04 Apple Inc. Intelligent automated assistant
US10043516B2 (en) 2016-09-23 2018-08-07 Apple Inc. Intelligent automated assistant
US10593346B2 (en) 2016-12-22 2020-03-17 Apple Inc. Rank-reduced token representation for automatic speech recognition
US10755703B2 (en) 2017-05-11 2020-08-25 Apple Inc. Offline personal assistant
US11405466B2 (en) 2017-05-12 2022-08-02 Apple Inc. Synchronization and task delegation of a digital assistant
US10410637B2 (en) 2017-05-12 2019-09-10 Apple Inc. User-specific acoustic models
US10791176B2 (en) 2017-05-12 2020-09-29 Apple Inc. Synchronization and task delegation of a digital assistant
US10482874B2 (en) 2017-05-15 2019-11-19 Apple Inc. Hierarchical belief states for digital assistants
US10810274B2 (en) 2017-05-15 2020-10-20 Apple Inc. Optimizing dialogue policy decisions for digital assistants using implicit feedback
US11217255B2 (en) 2017-05-16 2022-01-04 Apple Inc. Far-field extension for digital assistant services

Also Published As

Publication number Publication date
KR20030070179A (en) 2003-08-29
US7346516B2 (en) 2008-03-18

Similar Documents

Publication Publication Date Title
US7346516B2 (en) Method of segmenting an audio stream
Chou et al. Robust singing detection in speech/music discriminator design
Lu et al. Content analysis for audio classification and segmentation
Lu et al. A robust audio classification and segmentation method
US6785645B2 (en) Real-time speech and music classifier
Lu et al. Content-based audio classification and segmentation by using support vector machines
US9135929B2 (en) Efficient content classification and loudness estimation
US8793127B2 (en) Method and apparatus for automatically determining speaker characteristics for speech-directed advertising or other enhancement of speech-controlled devices or services
Kos et al. Acoustic classification and segmentation using modified spectral roll-off and variance-based features
US6901362B1 (en) Audio segmentation and classification
US20050228649A1 (en) Method and apparatus for classifying sound signals
US7521622B1 (en) Noise-resistant detection of harmonic segments of audio signals
Butko et al. Audio segmentation of broadcast news in the Albayzin-2010 evaluation: overview, results, and discussion
US20050192795A1 (en) Identification of the presence of speech in digital audio data
Zelenák et al. Speaker diarization of broadcast news in Albayzin 2010 evaluation campaign
Zewoudie et al. The use of long-term features for GMM-and i-vector-based speaker diarization systems
Kwon et al. Speaker change detection using a new weighted distance measure.
Bugatti et al. Audio classification in speech and music: a comparison between a statistical and a neural approach
Delacourt et al. Speaker-based segmentation for audio data indexing
US7680654B2 (en) Apparatus and method for segmentation of audio data into meta patterns
Pinquier et al. Audio indexing: primary components retrieval: robust classification in audio documents
Huijbregts et al. Filtering the unknown: Speech activity detection in heterogeneous video collections
Velayatipour et al. A review on speech-music discrimination methods
Zhan et al. Dragon Systems’ 1998 broadcast news transcription system for Mandarin
Kacprzak et al. Speech/music discrimination for analysis of radio stations

Legal Events

Date Code Title Description
AS Assignment

Owner name: LG ELECTRONICS INC., KOREA, REPUBLIC OF

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SALL, MIKHAEL A.;GRAMNITSKIY, SERGEI N.;MAIBORODA, ALEXANDR L.;AND OTHERS;REEL/FRAME:014058/0840

Effective date: 20030221

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCF Information on status: patent grant

Free format text: PATENTED CASE

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Free format text: PAYER NUMBER DE-ASSIGNED (ORIGINAL EVENT CODE: RMPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

FPAY Fee payment

Year of fee payment: 4

FPAY Fee payment

Year of fee payment: 8

FEPP Fee payment procedure

Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

LAPS Lapse for failure to pay maintenance fees

Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCH Information on status: patent discontinuation

Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362

FP Lapsed due to failure to pay maintenance fee

Effective date: 20200318