CN113537448A - Method and device for processing streaming data, semiconductor chip and computer equipment - Google Patents
Method and device for processing streaming data, semiconductor chip and computer equipment Download PDFInfo
- Publication number
- CN113537448A CN113537448A CN202010320086.1A CN202010320086A CN113537448A CN 113537448 A CN113537448 A CN 113537448A CN 202010320086 A CN202010320086 A CN 202010320086A CN 113537448 A CN113537448 A CN 113537448A
- Authority
- CN
- China
- Prior art keywords
- data
- time interval
- neural network
- calculation
- result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000012545 processing Methods 0.000 title claims abstract description 38
- 239000004065 semiconductor Substances 0.000 title claims abstract description 14
- 238000004364 calculation method Methods 0.000 claims abstract description 101
- 238000013527 convolutional neural network Methods 0.000 claims description 60
- 238000004590 computer program Methods 0.000 claims description 17
- 238000013528 artificial neural network Methods 0.000 claims description 16
- 230000008569 process Effects 0.000 description 15
- 238000010586 diagram Methods 0.000 description 12
- 239000000872 buffer Substances 0.000 description 8
- 239000010410 layer Substances 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 238000003672 processing method Methods 0.000 description 3
- 230000003139 buffering effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000016776 visual perception Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000011229 interlayer Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L17/00—Speaker identification or verification techniques
- G10L17/18—Artificial neural networks; Connectionist approaches
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
- G10L25/30—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique using neural networks
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Acoustics & Sound (AREA)
- Human Computer Interaction (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
Abstract
The application relates to a method and a device for processing streaming data, a semiconductor chip and a computer device, wherein the method for processing the streaming data comprises the following steps: the method comprises the steps of dividing streaming data into a plurality of data blocks according to a time axis of the streaming data, wherein the data blocks correspond to time intervals on the time axis, in a first time interval, a first calculation block calculates and classifies the data blocks in the first time interval to obtain a first recognition result, the first recognition result is cached, in a second time interval, a second calculation block calculates the data blocks in the second time interval to obtain a second calculation result, the second calculation block obtains the cache of the first recognition result, and classifies the first recognition result and the second calculation result to obtain a second recognition result.
Description
Technical Field
The present application relates to the field of information processing technologies, and in particular, to a method and an apparatus for processing streaming data, a semiconductor chip, and a computer device.
Background
With the development of science and technology, more and more users use intelligent devices to communicate with relatives and friends, and in the process of using the intelligent devices, the intelligent devices need to process a large amount of audio or video data, wherein the audio or video data are streaming data.
In the related art, the intelligent device processes streaming data through a well-defined complete convolutional neural network, and the data repetition calculation rate is high, so that the intelligent device is high in power consumption and cost.
At present, no effective solution is provided for the problems of high power consumption and high cost of intelligent equipment caused by high repeated calculation rate of the intelligent equipment for flow type data processing in the related technology.
Disclosure of Invention
The embodiment of the application provides a streaming data processing method and device, a semiconductor chip, a computer device and a computer readable storage medium, so as to at least solve the problems that in the related art, the intelligent device has a high repeated calculation rate for streaming data processing, which results in large power consumption and high cost of the intelligent device.
In a first aspect, an embodiment of the present application provides a method for processing streaming data, where the method includes:
acquiring streaming data in a preset time period, and dividing the streaming data into a plurality of data blocks according to a time axis of the streaming data, wherein the data blocks correspond to time intervals on the time axis;
calculating the data blocks through a convolutional neural network, wherein the convolutional neural network comprises a plurality of calculation blocks, and in a first time interval, a first calculation block of the convolutional neural network calculates and classifies the data blocks in the first time interval to obtain a first identification result, and caches the first identification result;
in a second time interval, a second calculation block of the convolutional neural network calculates the data block in the second time interval to obtain a second calculation result, the second calculation block obtains a cache of the first recognition result, and the first recognition result and the second calculation result are classified to obtain a second recognition result.
In some of these embodiments, the first calculation block and the second calculation block each correspond to a time interval on the time axis.
In some embodiments, after obtaining the second recognition result, the method further includes:
and caching the second identification result in a first-in first-out memory.
In some of these embodiments, said calculating and classifying said data block in said first time interval comprises:
under the condition that the streaming data is audio data, extracting audio features of the audio data, and identifying voiceprints of the audio data according to the audio features;
and according to the voiceprint, carrying out classification and identification on the audio data.
In some of these embodiments, said calculating and classifying said data block in said first time interval further comprises:
and under the condition that the streaming data is video data, carrying out image recognition on a video frame of the video data, extracting a feature vector of the video frame, and carrying out classification recognition on the video frame according to the feature vector.
In a second aspect, an embodiment of the present application provides an apparatus for audio data processing, the apparatus including a microphone and a microprocessor;
the microphone acquires audio data in a preset time period, and the microprocessor divides the audio data into a plurality of data blocks according to a time axis of the audio data, wherein the data blocks correspond to time intervals on the time axis;
the microprocessor calculates and classifies the data blocks in the first time interval in a first time interval to obtain a first identification result, and caches the first identification result to a first-in first-out memory, the microprocessor calculates the data blocks in the second time interval in a second time interval to obtain a second calculation result, the microprocessor obtains the cache of the first identification result, and classifies and identifies the first identification result and the second calculation result to obtain a second identification result.
In some of these embodiments, the microprocessor includes a convolutional neural network that computes and classifies the data blocks in the time interval, wherein the convolutional neural network includes a plurality of computation blocks that correspond to the time intervals on the time axis.
In a third aspect, an embodiment of the present application provides a semiconductor chip for streaming data processing, where the chip includes a neural network accelerator:
the method comprises the steps that a receiver of a semiconductor chip obtains streaming data in a preset time period, and a neural network accelerator divides the streaming data into a plurality of data blocks according to a time axis of the streaming data, wherein the data blocks correspond to time intervals on the time axis;
the neural network accelerator calculates and classifies the data blocks in the first time interval through a first calculation block of a convolutional neural network in the first time interval to obtain a first identification result, and caches the first identification result;
and the neural network accelerator calculates the data blocks in the second time interval through a second calculation block of the convolutional neural network in the second time interval to obtain a second calculation result, acquires the cache of the first recognition result, and classifies the first recognition result and the second calculation result to obtain a second recognition result.
In some of these embodiments, in the neural network accelerator, the first computation block and the second computation block of the convolutional neural network each correspond to a time interval on the time axis.
In a fourth aspect, the present application provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements any one of the above methods when executing the computer program.
In a fifth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement any of the above methods.
Compared with the related art, the streaming data processing method provided in the embodiment of the present application divides streaming data in a preset time period into a plurality of data blocks according to a time axis of the streaming data, where the data blocks correspond to time intervals on the time axis, and calculates the data blocks through a convolutional neural network, where the convolutional neural network includes a plurality of calculation blocks, in a first time interval, a first calculation block of the convolutional neural network calculates and classifies the data blocks in the first time interval to obtain a first recognition result, and buffers the first recognition result, in a second time interval, a second calculation block of the convolutional neural network calculates the data blocks in a second time interval to obtain a second calculation result, and the second calculation block obtains a buffer of the first recognition result, the first recognition result and the second calculation result are classified to obtain the second recognition result, the problems of high repeated calculation rate of intelligent equipment on flow data processing, high power consumption and high cost of the intelligent equipment are solved, the repeated calculation rate of the flow data is reduced, the calculation efficiency is improved, and the cost is reduced.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the application.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a flow diagram of a method of streaming data processing according to an embodiment of the present application;
FIG. 2 is a flow chart of a method of audio data processing according to an embodiment of the present application;
fig. 3 is a block diagram of an apparatus for audio data processing according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a convolutional neural network according to an embodiment of the present application;
fig. 5 is an internal structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments provided in the present application without any inventive step are within the scope of protection of the present application.
It is obvious that the drawings in the following description are only examples or embodiments of the present application, and that it is also possible for a person skilled in the art to apply the present application to other similar contexts on the basis of these drawings without inventive effort. Moreover, it should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another.
Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the specification. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of ordinary skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments without conflict.
Unless defined otherwise, technical or scientific terms referred to herein shall have the ordinary meaning as understood by those of ordinary skill in the art to which this application belongs. Reference to "a," "an," "the," and similar words throughout this application are not to be construed as limiting in number, and may refer to the singular or the plural. The present application is directed to the use of the terms "including," "comprising," "having," and any variations thereof, which are intended to cover non-exclusive inclusions; for example, a process, method, system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to the listed steps or elements, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. Reference to "connected," "coupled," and the like in this application is not intended to be limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" as referred to herein means two or more. "and/or" describes an association relationship of associated objects, meaning that three relationships may exist, for example, "A and/or B" may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. Reference herein to the terms "first," "second," "third," and the like, are merely to distinguish similar objects and do not denote a particular ordering for the objects.
The embodiment provides a method for processing streaming data. Fig. 1 is a flowchart of a method of streaming data processing according to an embodiment of the present application, as shown in fig. 1, the method including the steps of:
step S101, acquiring streaming data in a preset time period, and dividing the streaming data into a plurality of data blocks according to a time axis of the streaming data, where the data blocks correspond to time intervals on the time axis.
The streaming data in this embodiment is a data sequence that arrives in time sequence, in a large amount, quickly, and continuously, such as audio or video data generated during the interaction process of the user through the smart device. The preset time period is a time preset value for processing the streaming data, and is divided along a time axis, and the preset time period comprises a plurality of time intervals. For example, in the case where the streaming data is video data, the preset time period may be 1s, and the time interval may be 30, that is, 30 time intervals are included in the preset time period of 1 s. The streaming data is partitioned according to the time interval, and a plurality of data blocks are obtained in 1s, for example, in video data, the image frame data in each time interval can be regarded as 1 data block, and in 1 data block, the number of the image frames can be 1, 2 or other values.
Step S102, calculating the data block through a convolutional neural network, wherein the convolutional neural network comprises a plurality of calculation blocks, in a first time interval, the first calculation block of the convolutional neural network calculates and classifies the data block in the first time interval to obtain a first identification result, and the first identification result is cached.
The Convolutional Neural Network (CNN) is a type of feed-forward Neural network containing convolution calculation and having a deep structure, and is one of representative algorithms for deep learning, and the Convolutional Neural network has a characteristic learning capability and can perform translation invariant classification on input information according to a hierarchical structure thereof. The convolutional neural network is constructed by simulating a Visual Perception (Visual Perception) mechanism of a living being, and can be used for supervised learning and unsupervised learning, the convolutional kernel parameter sharing in an implicit layer and the sparsity of interlayer connection enable the convolutional neural network to learn lattice characteristics with a small amount of calculation, the learning result is stable, and no additional characteristic Engineering (Feature Engineering) requirement exists for flow data.
In this embodiment, in the first time interval, the first calculation block in the convolutional neural network only calculates the first data block in the first time interval, and then buffers the recognition result.
Step S103, in a second time interval, a second calculation block of the convolutional neural network calculates the data block in the second time interval to obtain a second calculation result, and the second calculation block obtains a cache of the first recognition result, and classifies the first recognition result and the second calculation result to obtain a second recognition result.
In a second time interval, a second calculation block in the convolutional neural network only calculates data in a second data block, and then the first recognition result and the second calculation result are classified and recognized together by combining the first recognition result in the cache to obtain a second recognition result.
Through the steps S101 to S103, based on the time axis, in each time interval, the calculation block in the convolutional neural network only calculates the data block in the time interval, and then classifies and identifies the identification result together with the identification result cached in the previous time interval, so that a large amount of calculation of repeated data is eliminated, the problems of high repeated calculation rate of the intelligent device on the flow data processing, large power consumption and high cost of the intelligent device are solved, the repeated calculation rate of the flow data is reduced, the calculation efficiency is improved, and the cost is reduced.
In some of these embodiments, the first calculation block and the second calculation block each correspond to a time interval on a time axis. In this embodiment, the convolutional neural network is divided into a plurality of calculation blocks according to a time axis, the calculation blocks correspond to time intervals at a classification and identification level, and further correspond to data blocks in streaming data, so that the convolutional neural network can process the streaming data conveniently, and the calculation efficiency of the convolutional neural network is improved while the calculation accuracy of the convolutional neural network is ensured.
In some embodiments, it is necessary to train the weight parameters of multiple computation blocks in the convolutional neural network to find the optimal values of the weight parameters. In the training process, the weight parameters of the calculation blocks are trained simultaneously.
In some embodiments, after obtaining the second recognition result, the method for processing streaming data further includes: and buffering the second recognition result in a first-in first-out memory. The FIFO buffer allows data to enter a queue according to the sequence of the data arriving at the interface in the process of buffering the data, and meanwhile, the FIFO buffer allows the data to be dequeued according to the sequence of the data entering the buffer at the outlet of the queue, the data entering First will be dequeued First, and the data entering later will be dequeued later until all identification results are buffered. In the embodiment, FIFO is adopted for caching the identification result, the data processing is simple, and the data processing cost can be reduced.
The embodiment also provides an audio data processing method. Fig. 2 is a flowchart of a method for processing audio data according to an embodiment of the present application, and in the case that streaming data is audio data, as shown in fig. 2, the method further includes the following steps:
step S201, extracting the audio feature of the audio data, and identifying the voiceprint of the audio data according to the audio feature. The audio data may be sound data acquired by a microphone in real time or sound data cached in the mobile phone, the audio features include signal amplitude, frequency, continuity, and the like of the audio data, and the voiceprint is a frequency spectrum representation of the sound signal. In the present embodiment, audio data may be extracted every 10 milliseconds.
Step S202, according to the voiceprint, the audio data is classified and identified. The voiceprint contains a large amount of information of the sound data, and the processing result of the input audio data can be acquired by calculating, classifying and identifying the information.
Through the steps S201 and S202, after the audio data is acquired, the audio data is filtered and scaled through the calculation block in the convolutional neural network, the audio features in each frame of audio data are extracted, and then the audio features are classified and identified to obtain a final audio data result, so that the processing efficiency of the audio data is improved.
In some embodiments, in the case that the streaming data is video data, image recognition is performed on a video frame of the video data, a feature vector of the video frame is extracted, and classification recognition is performed on the video frame according to the feature vector. For example, in a time interval, only one video frame exists, the calculation block in the convolutional neural network performs feature extraction on the video frame, so that the classification and identification of the video frame are realized, and the processing efficiency of video data is improved.
It should be noted that the steps illustrated in the above-described flow diagrams or in the flow diagrams of the figures may be performed in a computer system, such as a set of computer-executable instructions, and that, although a logical order is illustrated in the flow diagrams, in some cases, the steps illustrated or described may be performed in an order different than here.
The present embodiment further provides an audio data processing apparatus, which is used to implement the foregoing embodiments and preferred embodiments, and the description of the apparatus is omitted for brevity. As used hereinafter, the terms "module," "unit," "subunit," and the like may implement a combination of software and/or hardware for a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
In some embodiments, fig. 3 is a block diagram of an apparatus for audio data processing according to an embodiment of the present application, and as shown in fig. 3, the apparatus includes: a microphone 31 and a microprocessor 32;
the microphone 31 obtains audio data in a preset time period, converts the audio data into digital signals, and sends the digital signals to the microprocessor 32 for processing, the microprocessor 32 divides the audio data into a plurality of data blocks according to a time axis of the audio data, wherein the data blocks correspond to time intervals on the time axis, the microprocessor 32 calculates and classifies the data blocks in a first time interval in the first time interval to obtain a first identification result, and buffers the first identification result into a first-in first-out memory, the microprocessor 32 calculates the data blocks in a second time interval in the second time interval to obtain a second calculation result, the microprocessor 32 obtains a buffer of the first identification result from the first-in first-out memory, and classifies and identifies the first identification result and the second calculation result, and obtaining a second recognition result.
In the related art, in the process of performing classification identification at 4 time intervals, data in 32 time intervals needs to be queried for each identification result, and in the process of calculating streaming data in [ t-31, t ] and [ t-27, t +4], the repetition rate of the data reaches 87%, which is not acceptable for low-power devices and mobile devices. In the embodiment, based on a time axis, in each time interval, the calculation block in the convolutional neural network only calculates the data block in the time interval, and then the recognition result and the recognition result cached in the previous time interval are classified and recognized together, so that the calculation of a large amount of repeated data is eliminated, the problems of high repeated calculation rate of intelligent equipment on the flow data processing, large power consumption and high cost of the intelligent equipment are solved, the repeated calculation rate of the flow data is reduced, the calculation efficiency is improved, and the cost is reduced.
In some embodiments, the microprocessor 32 includes a convolutional neural network that computes and classifies the data blocks in the time interval, wherein the convolutional neural network includes a plurality of computation blocks that correspond to the time intervals on the time axis.
In the related art, the repetition rate may be reduced by increasing the time interval of classification and recognition, for example, classifying and recognition at 4 time intervals is changed to classifying and recognition at 8 time intervals, and the repetition rate may be reduced to 75%, but at this time, since the data signal starts from the middle of the time interval, the recognition accuracy is reduced. In this embodiment, the convolutional neural network is divided into a plurality of calculation blocks according to a time axis, the calculation blocks correspond to time intervals at a classification and identification level, and further correspond to data blocks in streaming data, so that the convolutional neural network can process the streaming data conveniently, and the calculation efficiency of the convolutional neural network is improved while the calculation accuracy of the convolutional neural network is ensured.
In some embodiments, there is also provided a semiconductor chip for streaming data processing, the semiconductor chip comprising a neural network accelerator; the method comprises the steps that a receiver of the semiconductor chip obtains streaming data in a preset time period, and the neural network accelerator divides the streaming data into a plurality of data blocks according to a time axis of the streaming data, wherein the data blocks correspond to time intervals on the time axis; in a first time interval, the neural network accelerator calculates and classifies the data blocks in the first time interval through a first calculation block of a convolutional neural network to obtain a first identification result, and caches the first identification result; and the neural network accelerator calculates the data block in the second time interval through a second calculation block of the convolutional neural network in the second time interval to obtain a second calculation result, acquires the cache of the first recognition result, and classifies the first recognition result and the second calculation result to obtain a second recognition result.
In this embodiment, based on a time axis, in each time interval, the calculation block in the convolutional neural network only calculates the data block in the time interval, and then classifies and identifies the identification result together with the identification result cached in the previous time interval, so that calculation of a large amount of repeated data is eliminated, the problems of high repeated calculation rate of intelligent equipment on processing of streaming data, large power consumption and high cost of the intelligent equipment are solved, the repeated calculation rate of streaming data is reduced, the calculation efficiency is improved, the power consumption of hardware equipment is reduced, and further the cost is reduced.
In some embodiments, in a neural network accelerator of a semiconductor chip, a first calculation block and a second calculation block of a convolutional neural network each correspond to a time interval on a time axis. Fig. 4 is a schematic structural diagram of a convolutional neural network according to an embodiment of the present application, and as shown in fig. 4, along a time axis, the convolutional neural network is divided into a plurality of calculation blocks, the calculation blocks are respectively named as Block0, Block1, Block2, Block3, Block …, Block-n +3, Block-n +2, Block-n +1, Block-n, and so on, and after the calculation blocks, respectively are a flattened Layer (flat Layer), a Fully Connected Layer (FC for short), and a classification (Classes). In this embodiment, the convolutional neural network is divided into a plurality of calculation blocks according to a time axis, the calculation blocks correspond to time intervals at a classification and identification level, and further correspond to data blocks in streaming data, so that the convolutional neural network can process the streaming data conveniently, and the calculation efficiency of the convolutional neural network is improved while the calculation accuracy of the convolutional neural network is ensured.
The above modules may be functional modules or program modules, and may be implemented by software or hardware. For a module implemented by hardware, the modules may be located in the same processor; or the modules can be respectively positioned in different processors in any combination.
In one embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of streaming data processing. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
In one embodiment, fig. 5 is a schematic diagram of an internal structure of an electronic device according to an embodiment of the present application, and as shown in fig. 5, an electronic device is provided, where the electronic device may be a server, and the internal structure diagram may be as shown in fig. 5. The electronic device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic equipment comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the electronic device is used for storing data. The network interface of the electronic device is used for connecting and communicating with an external terminal through a network. The computer program is executed by a processor to implement a method of streaming data processing.
Those skilled in the art will appreciate that the configuration shown in fig. 5 is a block diagram of only a portion of the configuration associated with the present application, and does not constitute a limitation on the electronic device to which the present application is applied, and a particular electronic device may include more or less components than those shown in the drawings, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, the steps in the method for processing streaming data provided in the foregoing embodiments are implemented.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps in the method of streaming data processing provided by the various embodiments described above.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (11)
1. A method of streaming data processing, the method comprising:
acquiring streaming data in a preset time period, and dividing the streaming data into a plurality of data blocks according to a time axis of the streaming data, wherein the data blocks correspond to time intervals on the time axis;
calculating the data blocks through a convolutional neural network, wherein the convolutional neural network comprises a plurality of calculation blocks, and in a first time interval, a first calculation block of the convolutional neural network calculates and classifies the data blocks in the first time interval to obtain a first identification result, and caches the first identification result;
in a second time interval, a second calculation block of the convolutional neural network calculates the data block in the second time interval to obtain a second calculation result, the second calculation block obtains a cache of the first recognition result, and the first recognition result and the second calculation result are classified to obtain a second recognition result.
2. The method of claim 1, wherein the first calculation block and the second calculation block each correspond to a time interval on the time axis.
3. The method of claim 1, wherein after said obtaining the second recognition result, the method further comprises:
and caching the second identification result in a first-in first-out memory.
4. The method of claim 1, wherein the computing and classifying the data block in the first time interval comprises:
under the condition that the streaming data is audio data, extracting audio features of the audio data, and identifying voiceprints of the audio data according to the audio features;
and according to the voiceprint, carrying out classification and identification on the audio data.
5. The method of claim 1, wherein said computing and classifying the data block in the first time interval further comprises:
and under the condition that the streaming data is video data, carrying out image recognition on a video frame of the video data, extracting a feature vector of the video frame, and carrying out classification recognition on the video frame according to the feature vector.
6. An apparatus for audio data processing, the apparatus comprising a microphone and a microprocessor;
the microphone acquires audio data in a preset time period, and the microprocessor divides the audio data into a plurality of data blocks according to a time axis of the audio data, wherein the data blocks correspond to time intervals on the time axis;
the microprocessor calculates and classifies the data blocks in the first time interval in a first time interval to obtain a first identification result, and caches the first identification result to a first-in first-out memory, the microprocessor calculates the data blocks in the second time interval in a second time interval to obtain a second calculation result, the microprocessor obtains the cache of the first identification result, and classifies and identifies the first identification result and the second calculation result to obtain a second identification result.
7. The apparatus of claim 6, wherein the microprocessor comprises a convolutional neural network that computes and classifies the data blocks in the time interval, wherein the convolutional neural network comprises a plurality of computation blocks that correspond to the time interval on the time axis.
8. A semiconductor chip for streaming data processing, the chip comprising a neural network accelerator:
the method comprises the steps that a receiver of a semiconductor chip obtains streaming data in a preset time period, and a neural network accelerator divides the streaming data into a plurality of data blocks according to a time axis of the streaming data, wherein the data blocks correspond to time intervals on the time axis;
the neural network accelerator calculates and classifies the data blocks in the first time interval through a first calculation block of a convolutional neural network in the first time interval to obtain a first identification result, and caches the first identification result;
and the neural network accelerator calculates the data blocks in the second time interval through a second calculation block of the convolutional neural network in the second time interval to obtain a second calculation result, acquires the cache of the first recognition result, and classifies the first recognition result and the second calculation result to obtain a second recognition result.
9. The semiconductor chip according to claim 8, wherein in the neural network accelerator, the first calculation block and the second calculation block of the convolutional neural network each correspond to a time interval on the time axis.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1 to 5 when executing the computer program.
11. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010320086.1A CN113537448A (en) | 2020-04-22 | 2020-04-22 | Method and device for processing streaming data, semiconductor chip and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010320086.1A CN113537448A (en) | 2020-04-22 | 2020-04-22 | Method and device for processing streaming data, semiconductor chip and computer equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113537448A true CN113537448A (en) | 2021-10-22 |
Family
ID=78094006
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010320086.1A Pending CN113537448A (en) | 2020-04-22 | 2020-04-22 | Method and device for processing streaming data, semiconductor chip and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113537448A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023124428A1 (en) * | 2021-12-30 | 2023-07-06 | 上海商汤智能科技有限公司 | Chip, accelerator card, electronic device and data processing method |
-
2020
- 2020-04-22 CN CN202010320086.1A patent/CN113537448A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023124428A1 (en) * | 2021-12-30 | 2023-07-06 | 上海商汤智能科技有限公司 | Chip, accelerator card, electronic device and data processing method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107633207B (en) | AU characteristic recognition methods, device and storage medium | |
| WO2019228317A1 (en) | Face recognition method and device, and computer readable medium | |
| CN111260665B (en) | Image segmentation model training method and device | |
| US11138903B2 (en) | Method, apparatus, device and system for sign language translation | |
| CN110020582B (en) | Face emotion recognition method, device, equipment and medium based on deep learning | |
| CN110765860A (en) | Tumble determination method, tumble determination device, computer apparatus, and storage medium | |
| WO2020024395A1 (en) | Fatigue driving detection method and apparatus, computer device, and storage medium | |
| CN110287836B (en) | Image classification method and device, computer equipment and storage medium | |
| CN109271917B (en) | Face recognition method and device, computer equipment and readable storage medium | |
| CN110364185B (en) | Emotion recognition method based on voice data, terminal equipment and medium | |
| WO2022028147A1 (en) | Image classification model training method and apparatus, computer device, and storage medium | |
| WO2017088434A1 (en) | Human face model matrix training method and apparatus, and storage medium | |
| CN109062715A (en) | The determination method, apparatus and terminal of memory clock frequency | |
| CN112487844A (en) | Gesture recognition method, electronic device, computer-readable storage medium, and chip | |
| CN112613435A (en) | Face image generation method, device, equipment and medium | |
| CN112417985A (en) | Face feature point tracking method, system, electronic equipment and storage medium | |
| CN113537448A (en) | Method and device for processing streaming data, semiconductor chip and computer equipment | |
| WO2021204187A1 (en) | Layout analysis method and electronic device | |
| CN113194281A (en) | Video analysis method and device, computer equipment and storage medium | |
| CN112950497A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN116129881A (en) | Voice task processing method and device, electronic equipment and storage medium | |
| CN116266394A (en) | Multi-modal emotion recognition method, device and storage medium | |
| CN113761004A (en) | Network model data processing method, network model data processing device, network model data display device and storage medium | |
| CN111401317B (en) | Video classification method, device, equipment and storage medium | |
| CN115952830B (en) | Data processing method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |