CN111966721A - Data processing method and device, computer readable storage medium and electronic equipment - Google Patents
Data processing method and device, computer readable storage medium and electronic equipment Download PDFInfo
- Publication number
- CN111966721A CN111966721A CN201910420005.2A CN201910420005A CN111966721A CN 111966721 A CN111966721 A CN 111966721A CN 201910420005 A CN201910420005 A CN 201910420005A CN 111966721 A CN111966721 A CN 111966721A
- Authority
- CN
- China
- Prior art keywords
- data
- data frame
- frame
- processing
- current
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 160
- 230000000737 periodic effect Effects 0.000 claims abstract description 58
- 238000000034 method Methods 0.000 claims abstract description 38
- 230000015654 memory Effects 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 9
- 238000004458 analytical method Methods 0.000 abstract description 5
- 238000012549 training Methods 0.000 description 19
- 238000010586 diagram Methods 0.000 description 13
- 230000008569 process Effects 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 6
- 230000000306 recurrent effect Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 239000000284 extract Substances 0.000 description 4
- 238000010801 machine learning Methods 0.000 description 3
- 238000007781 pre-processing Methods 0.000 description 3
- 239000000126 substance Substances 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000013480 data collection Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000391 smoking effect Effects 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000005215 recombination Methods 0.000 description 1
- 230000006798 recombination Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Probability & Statistics with Applications (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Fuzzy Systems (AREA)
- Image Analysis (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The embodiment of the disclosure discloses a data processing method, a data processing device, a computer readable storage medium and an electronic device, wherein the method comprises the following steps: determining the category of the current data frame; if the type of the current data frame is a first preset type, determining at least one piece of periodic data included in the current data frame; determining at least one target historical data frame from the data frame sequence where the current data frame is located; and obtaining a processing result aiming at the current data frame by utilizing a pre-trained data processing model based on at least one frame of target historical data frame and at least one piece of periodic data. The embodiment of the disclosure can effectively utilize the periodic data, so that the data processing model can combine the current data frame with the historical data frame corresponding to the periodic data according to the periodic data for analysis, namely, different processing is carried out according to different data generation periods, and the accuracy and efficiency of decision making based on the data frame sequence are improved.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a data processing method and apparatus, a computer-readable storage medium, and an electronic device.
Background
In real life, a lot of problems exist in which electronic equipment makes decisions according to data sequences (such as image data sequences and voice data sequences), such as intelligent customer service, intelligent shopping guide, interaction with passengers in intelligent vehicles, and the like. The existing method for processing data sequence mainly writes corresponding decision strategies according to rules manually, including simple logic judgment, finite state machine and the like. A model such as a recurrent neural network may also be used to classify the data sequence into corresponding decisions (e.g., output some information, perform some operation, etc.).
Disclosure of Invention
The embodiment of the disclosure provides a data processing method and device, a computer readable storage medium and an electronic device.
According to an aspect of an embodiment of the present disclosure, there is provided a data processing method, including: determining the category of the current data frame; if the type of the current data frame is a first preset type, determining at least one piece of periodic data included in the current data frame; determining at least one target historical data frame from the data frame sequence where the current data frame is located; and obtaining a processing result aiming at the current data frame by utilizing a pre-trained data processing model based on at least one frame of target historical data frame and at least one piece of periodic data.
According to another aspect of the embodiments of the present disclosure, there is provided a data processing apparatus including: the first determining module is used for determining the category of the current data frame; the second determining module is used for determining at least one piece of periodic data included in the current data frame if the type of the current data frame is a first preset type; the third determining module is used for determining at least one target historical data frame from the data frame sequence where the current data frame is located; and the first processing module is used for obtaining a processing result aiming at the current data frame by utilizing a pre-trained data processing model based on at least one frame of target historical data frame and at least one piece of periodic data.
According to another aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium storing a computer program for executing the above-described data processing method.
According to another aspect of the embodiments of the present disclosure, there is provided an electronic device including: a processor; a memory for storing processor-executable instructions; and the processor is used for executing the data processing method.
Based on the data processing method, device, computer-readable storage medium and electronic device provided by the above embodiments of the present disclosure, at least one piece of period data is determined from a first preset type of current data frame, at least one frame of target historical data frame is determined from a data frame sequence in which the current data frame is located, and finally, a processing result for the current data frame is obtained by using a data processing model based on the at least one frame of target historical data frame and the at least one piece of period data, since the current data frame is prevented from being processed by only using previous frame data, the embodiments of the present disclosure can effectively use the target historical data frame and the period data, so that the data processing model can combine the current data frame and the historical data frame corresponding to the period data according to the period data for analysis, that is, perform different processing according to different data generation periods, the accuracy of decision making based on the data frame sequence is improved.
The technical solution of the present disclosure is further described in detail by the accompanying drawings and examples.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent by describing in more detail embodiments of the present disclosure with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the principles of the disclosure and not to limit the disclosure. In the drawings, like reference numbers generally represent like parts or steps.
Fig. 1 is a system diagram to which the present disclosure is applicable.
Fig. 2 is a schematic flow chart of a data processing method according to an exemplary embodiment of the present disclosure.
Fig. 3 is a schematic diagram of an application scenario of the data processing method according to the embodiment of the present disclosure.
Fig. 4 is a flowchart illustrating a data processing method according to another exemplary embodiment of the present disclosure.
Fig. 5 is a schematic diagram of a data processing model processing a sequence of data frames according to an embodiment of the present disclosure.
Fig. 6 is a schematic structural diagram of a data processing apparatus according to an exemplary embodiment of the present disclosure.
Fig. 7 is a schematic structural diagram of a data processing apparatus according to another exemplary embodiment of the present disclosure.
Fig. 8 is a block diagram of an electronic device provided in an exemplary embodiment of the present disclosure.
Detailed Description
Hereinafter, example embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. It is to be understood that the described embodiments are merely a subset of the embodiments of the present disclosure and not all embodiments of the present disclosure, with the understanding that the present disclosure is not limited to the example embodiments described herein.
It should be noted that: the relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.
It will be understood by those of skill in the art that the terms "first," "second," and the like in the embodiments of the present disclosure are used merely to distinguish one element from another, and are not intended to imply any particular technical meaning, nor is the necessary logical order between them.
It is also understood that in embodiments of the present disclosure, "a plurality" may refer to two or more and "at least one" may refer to one, two or more.
It is also to be understood that any reference to any component, data, or structure in the embodiments of the disclosure, may be generally understood as one or more, unless explicitly defined otherwise or stated otherwise.
In addition, the term "and/or" in the present disclosure is only one kind of association relationship describing an associated object, and means that three kinds of relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" in the present disclosure generally indicates that the former and latter associated objects are in an "or" relationship.
It should also be understood that the description of the various embodiments of the present disclosure emphasizes the differences between the various embodiments, and the same or similar parts may be referred to each other, so that the descriptions thereof are omitted for brevity.
Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.
The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses.
Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.
The disclosed embodiments may be applied to electronic devices such as terminal devices, computer systems, servers, etc., which are operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known terminal devices, computing systems, environments, and/or configurations that may be suitable for use with electronic devices, such as terminal devices, computer systems, servers, and the like, include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, hand-held or laptop devices, microprocessor-based systems, set top boxes, programmable consumer electronics, network pcs, minicomputer systems, mainframe computer systems, distributed cloud computing environments that include any of the above systems, and the like.
Electronic devices such as terminal devices, computer systems, servers, etc. may be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The computer system/server may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.
Summary of the application
The existing method for classifying data sequences to execute corresponding decisions mainly classifies input data sequences by training a recurrent neural network. In different scenarios, the acquired data frames may contain different frequency information. For example, in an intelligent vehicle, the period for collecting the image of the driver and the period for collecting the vehicle state information are different. In the conventional recurrent neural network, when a certain frame of data is processed, only the previous frame of data is used for processing the current data frame, and different analyses are not performed on information obtained in different periods, so that the accuracy of decision (such as classification, prediction and the like) by using a data sequence needs to be improved.
Exemplary System
Fig. 1 shows an exemplary system architecture 100 of a data processing method or data processing apparatus to which embodiments of the present disclosure may be applied.
As shown in fig. 1, system architecture 100 may include terminal device 101, network 102, and server 103. Network 102 is the medium used to provide communication links between terminal devices 101 and server 103. Network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
A user may use terminal device 101 to interact with server 103 over network 102 to receive or send messages and the like. Various communication client applications, such as an image processing application, an audio processing application, and the like, may be installed on the terminal device 101.
The terminal device 101 may be various Electronic devices including, but not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle-mounted terminal (e.g., an ECU (Electronic Control Unit), a vehicle-mounted navigation terminal), etc., and a fixed terminal such as a digital TV, a desktop computer, etc.
The server 103 may be a server that provides various services, such as a background data processing server that processes data frames uploaded by the terminal device 101. The background data processing server can process the received data frame to obtain a processing result. The processing result can also be fed back to the terminal device 101
It should be noted that the data processing method provided by the embodiment of the present disclosure may be executed by the server 103 or the terminal device 101, and accordingly, the data processing apparatus may be disposed in the server 103 or the terminal device 101.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Exemplary method
Fig. 2 is a schematic flow chart of a data processing method according to an exemplary embodiment of the present disclosure. The embodiment can be applied to an electronic device, as shown in fig. 2, and includes the following steps:
In this embodiment, the electronic device may determine the category of the current data frame. The current data frame may be a data frame closest to the current time in the sequence of data frames acquired by the electronic device. As an example, the electronic device receives data collected from various data collection devices in real time and combines the collected data into a data frame, or combines data obtained by processing the collected data into a data frame.
As an example, the sequence of data frames may be vehicle and driver information. The dimension of each frame is the number of types of data contained in each frame, and the value of each dimension is the value of each type of data in the frame. Such as data characterizing whether the vehicle is started or not, data characterizing the state of vehicle windows, the volume of audio played in the vehicle, a user ID, a user age, data characterizing the gender of the user, data characterizing the orientation of a human face, data characterizing points of a human face, data characterizing the state of opening and closing human eyes, data characterizing the smoking state of the user, data characterizing the state of a telephone, data characterizing the result of voice recognition, etc.
The above categories may be used to distinguish whether a data frame includes periodic data. For example, the category of the data frame including the period data may be determined as the first preset category.
In this embodiment, if the type of the current data frame is a first preset type, the electronic device may determine at least one piece of periodic data included in the current data frame. Wherein the data frame belonging to the first preset category may be a data frame including periodic data. The periodic data may be data acquired by the electronic device according to various preset periods, or data obtained by processing the acquired data. For example, the current data frame may include two pieces of periodic data a and B, where the periodic data a is data obtained by periodically performing key point recognition on the acquired face image according to a first preset period (e.g., 100 milliseconds), and the periodic data B is data obtained by periodically performing key point recognition on the acquired voice according to a second preset period (e.g., 10 milliseconds).
In this embodiment, the electronic device may determine at least one target historical data frame from the sequence of data frames in which the current data frame is located. Wherein the target historical data frame is a data frame with which the current data frame is to be processed.
The electronic device may determine at least one target historical data frame from the sequence of data frames in which the current data frame is located using various methods. As an example, a previous data frame adjacent to the current data frame may be determined as the target history data frame, or a preset number of data frames before the current data frame may be determined as the target history data frame.
In some optional implementations, the at least one piece of periodic data respectively corresponds to a preset data generation period, and the data generation period is used for periodically generating the periodic data. For example, the current data frame may include two pieces of period data a and B, where the period data a is data obtained by performing keypoint identification on an acquired face image, and a corresponding data generation period is 100 milliseconds, that is, one piece of period data a is generated every 100 milliseconds. The period data B is data obtained by recognizing the collected voice, and the corresponding data generation period is 10 milliseconds, that is, one period data B is generated every 10 milliseconds. The electronic device may determine at least one frame of target historical data as follows:
first, a target historical data frame is determined from a data frame sequence based on data generation periods respectively corresponding to at least one piece of period data. As an example, assuming that the generation period corresponding to the period data a is 100 milliseconds and the generation period corresponding to the period data B is 10 milliseconds, one frame of the target history data frame is a data frame (here, denoted by data frame x) corresponding to 100 milliseconds before the generation time of the current data frame, and the other frame of the target history data frame is a data frame (here, denoted by data frame y) corresponding to 10 milliseconds before the generation time of the current data frame.
Then, the previous data frame adjacent to the current data frame is determined as a target historical data frame (here, denoted by data frame z). Continuing with the above example, the at least one frame of target historical data determined by the electronic device includes data frame x, data frame y, and data frame z.
The implementation mode can realize that the target historical data frame is determined in a targeted manner according to the periodic data, thereby being beneficial to more accurately analyzing the periodic data included by the current data frame and improving the accuracy of decision making such as classification, prediction and the like of the data frame.
And step 204, obtaining a processing result aiming at the current data frame by utilizing a pre-trained data processing model based on at least one frame of target historical data frame and at least one piece of periodic data.
In this embodiment, the electronic device may obtain a processing result for the current data frame by using a pre-trained data processing model based on at least one frame of the target historical data frame and at least one piece of periodic data. The data processing model is a model obtained by training an initial model in advance by using a preset training sample and a machine learning method. The initial model may be a model for processing sequence data, for example, the initial model may be an RNN (Recurrent Neural Network), and the RNN may include, but is not limited to, at least one of the following: LSTM (Long Short-Term Memory), GRU (gated Recurrent Unit), DRNN (Diagnonal Recurrent Neural network), etc. As an example, the electronic device may input the current data frame into a data processing model, the data processing model extracts pre-stored data related to each target historical data frame (for example, the historical data frame itself, or periodic data included in the target historical data frame, or intermediate data obtained by pre-processing the target historical data by a hidden layer included in the data processing model), and performs processing such as classification and prediction on the related data and the current data frame to obtain a processing result.
Generally, an electronic device can be trained to obtain a data processing model as follows:
firstly, a training sample set is obtained, wherein the training sample comprises a sample data frame sequence, and a sample processing result corresponding to each data frame included in the sample data frame sequence is obtained.
Then, each data frame in a sample data frame sequence included in a training sample in the training sample set is used as input by using a machine learning method, a sample processing result corresponding to the input data frame is used as expected output, and the initial model is trained. The initial model may retain relevant data (e.g., intermediate data determined by a hidden layer of the initial model) for data frames preceding the currently input data frame (e.g., a target historical data frame for the currently input data frame). For each training input data frame, the actual output can be obtained. And the actual output is data actually output by the initial model and is used for representing a processing result. Then, the executing body may adopt a gradient descent method and a back propagation method, adjust parameters of the initial model based on the actual output and the expected output, use the model obtained after each parameter adjustment as the initial model for the next training, and end the training under the condition that a preset training end condition is met, thereby obtaining the data processing model through training. It should be noted that the preset training end condition may include, but is not limited to, at least one of the following: the training time exceeds the preset time; the training times exceed the preset times; the loss value calculated using a predetermined loss function (e.g., a cross entropy loss function) is less than a predetermined loss value threshold.
In some optional implementations, if the category of the current data frame is a second preset category, the electronic device may further determine, as the target historical data frame, a previous data frame adjacent to the current data frame. Wherein the data frame belonging to the second preset category may be a data frame not including periodic data. For example, the data frame belonging to the second preset category may include data such as a temperature collected in real time, a vehicle running speed, and a volume of audio played in the vehicle, but does not include the above-mentioned periodic data.
The electronic device may then use the data processing model to obtain a processing result for the current data frame based on the target historical data frame. Specifically, the electronic device may input the current data frame into a data processing model, and the data processing model extracts pre-stored data related to the target historical data frame (for example, the historical data frame itself, or intermediate data obtained by pre-processing the target historical data by a hidden layer included in the data processing model), and performs processing such as classification and prediction on the related data and the current data frame to obtain a processing result.
In some optional implementations, the electronic device may further perform the following steps:
first, based on the processing result obtained in step 204, a control command corresponding to the processing result is generated. As an example, assuming that the above-described data frame sequence includes a data frame including eye opening/closing state information obtained by identifying eyes of the user, the processing result may include information obtained based on the eye opening/closing state information and used for characterizing whether the user is fatigue driving, and if the processing result includes information used for characterizing that the user is fatigue driving, a control instruction for issuing warning information corresponding to the fatigue driving is generated.
Then, an operation corresponding to the control instruction is executed based on the control instruction. Continuing with the above example, the electronic device may issue an alarm message (e.g., a sound message, a light message, etc.) for reminding the user that the driver is currently tired according to the control instruction.
Referring to fig. 3, fig. 3 is a schematic diagram of an application scenario of the data processing method according to the present embodiment. In the application scenario of fig. 3, the electronic device 301 is disposed in an automobile, the electronic device 301 receives data collected by a camera, a microphone, a sensor, etc. on the automobile and generates a data frame sequence 302, and the electronic device 301 first determines a data frame 3021 closest to the current time from the data frame sequence 302 as a current data frame (i.e., performs step 201). Then, the electronic device determines the category of the current data frame as a first preset category, and determines the period data a and the period data B from the current data frame 3021 (i.e., performs step 202). One period data A is generated every other data frame, and one period data B is generated every other two data frames. Then, the electronic device determines at least one target historical data frame from the sequence of data frames 302, including the data frames 3022, 3023, 3024 shown in the figure (i.e., performs step 203). Finally, the electronic device 301 obtains a processing result 304 (e.g., information characterizing generation of a fatigue driving warning) for the current data frame by using the pre-trained data processing model 303 based on at least one frame of the target historical data frame and at least one piece of periodic data. For example, the data processing model 303 is an RNN model, and includes a hidden layer and an output layer, where the hidden layer processes each target historical data frame in advance to obtain historical intermediate data, and the hidden layer combines the current data frame 3021 and the historical intermediate data to process (for example, performs calculation by using a preset formula) to obtain intermediate data corresponding to the current data frame 3021. And the output layer classifies the intermediate data, and outputs the classification result as a final processing result.
The method provided by the above embodiment of the present disclosure obtains a processing result for a current data frame by determining at least one period data from the current data frame of a first preset category, determining at least one target historical data frame from a data frame sequence in which the current data frame is located, and finally using a data processing model based on the at least one target historical data frame and the at least one period data, since processing of the current data frame with only the previous frame data is avoided, embodiments of the present disclosure may effectively utilize the target historical data frame and the periodic data, enable the data processing model to combine the current data frame with the historical data frame corresponding to the periodic data for analysis based on the periodic data, namely, different processing is carried out according to different data generation periods, so that the accuracy and the efficiency of decision making based on the data frame sequence are improved.
As shown in fig. 4, based on the embodiment shown in fig. 2, step 204 may include the following steps:
In this embodiment, the electronic device may input historical intermediate data corresponding to at least one frame of target historical data frame to a hidden layer of the data processing model, and determine intermediate data corresponding to a current data frame through the hidden layer. The historical intermediate data is generated by the hidden layer according to the target historical data frame.
The data processing model may include a hidden layer and an output layer. The hidden layer is used for processing an input data frame to obtain intermediate data, and the output layer is used for performing prediction, classification and other processing on the intermediate data to obtain a processing result. Generally, the electronic device may store intermediate data corresponding to each data frame, when a hidden layer processes a certain data frame, the hidden layer may extract corresponding historical intermediate data (i.e., intermediate data corresponding to each target historical data frame) according to the type of the data frame, and then the hidden layer processes the historical intermediate data and a currently input data frame to obtain intermediate data corresponding to the current data frame.
In this embodiment, the electronic device may input the intermediate data generated by the hidden layer to the output layer of the data processing model, so as to obtain the processing result. In general, the output layer may include a classifier for classifying the intermediate data to obtain the processing result. The processing results are typically in the form of vectors, with dimensions being the number of decisions that need to be made. For example, the decision may be whether to play music, whether to turn on a phone, whether to output an inattentive reminder, whether to output a fatigue driving reminder, etc., and the value of each dimension is 0 or 1.
As shown in fig. 5, a schematic diagram of a data processing model processing a sequence of data frames is shown. The data processing model in fig. 5 is a model obtained based on RNN training, and H in the figure represents a hidden layer. The part indicated by the reference number 1 in fig. 5 is an input layer of the model for receiving an input data frame, which is in the form of a vector. T1-Tn in the figure are different time points respectively, where Input is a data frame including no period information, InputA is a data frame including period data a, InputB is a data frame including period data B, and InputAB is a data frame including period data a and period data B. The dimension of the Input data frame is X, the dimension of the Input data frame is X + XA, where XA is the dimension of the periodic data a, the dimension of the Input data frame is X + XB, where XB is the dimension of the periodic data B, and the dimension of the Input data frame is X + XA + XB.
The portion indicated by the reference numeral 2 in fig. 5 is an output layer for outputting the processing result at each time. The dimension of each frame of processing result is the number of decisions to be made, and the dimension is Y. For example, playing music, starting a telephone scene, reminding with inattention, reminding with fatigue driving and the like, and the value of each dimension is 0 or 1. Can be calculated based on the following formula (1):
Outputt＝tanh(Ht*WhY+bY) Formula (1)
Wherein, OutputtFor the processing result Output at time t, tanh () is a hyperbolic tangent function, and in this embodiment, Output is performed when tanh () is greater than or equal to 0tTake 1, Output when tanh () is less than 0tTake 0.
Reference numeral 3 in fig. 5 indicates a hidden layer for processing an Input data frame, which can output intermediate data with dimension h, and the value of the hidden layer can be obtained based on the following formula (2):
Ht＝tanh(Input*WXh+Ht-1*Whh+bh) Formula (2)
Wherein HtIntermediate data obtained for the hidden layer at time t, WXhWeight corresponding to Input data frame of X dimension, Ht-1Intermediate data (i.e., historical intermediate data) obtained for the previous frame data frame for the hidden layer, WhhH being H dimensiont-1The corresponding weight.
Reference numeral 4 in fig. 5 indicates a hidden layer for processing an InputA data frame, which can output intermediate data with dimension h, and the value of this hidden layer can be obtained based on the following formula (3):
wherein the content of the first and second substances,
is X + XAWeight corresponding to InputA data frame of dimension, Ht-2To conceal the intermediate data obtained by the layer for the previous second frame data frame (i.e. historical intermediate data),
h being H dimensiont-2Corresponding weight, bhIs the bias term.
The reference number 5 in fig. 5 indicates a hidden layer for processing an InputB data frame, which can output intermediate data with dimension h, and the value of the hidden layer can be obtained based on the following formula (4):
wherein the content of the first and second substances,
is X + XBWeight corresponding to the dimension's InputB data frame, Ht-3To conceal the intermediate data obtained by the layer for the previous third frame data frame (i.e. historical intermediate data),
h being H dimensiont-3The corresponding weight.
wherein the content of the first and second substances,
is X + XA+XBThe corresponding weight of the dimension InputAB data frame.
It should be noted that the weights and bias terms in the above formulas are determined when the data processing model is trained. In the above formulas, the partial weight (W) and the bias term (b) are multiplexed, and the types of data included in different input data frames are different, thereby reducing the input of invalid information, saving the storage space and improving the calculation efficiency.
The above formulas are merely exemplary, and a method (including various formulas and neural networks of various structures) for enabling the hidden layer to determine the intermediate data of the current data frame by using the above historical intermediate data is within the protection scope of the present disclosure.
The method provided by the embodiment corresponding to fig. 4 highlights the step of using the hidden layer of the data processing model to specifically determine the intermediate data of the current data frame by using the historical intermediate data and obtain the processing result according to the intermediate data, so that the hidden layer of the model can perform different processing on different periodic data generated according to different periods in the data frame sequence, the different periodic data can be effectively transmitted, and the dispersion of the long-period periodic data in the forward propagation and backward propagation processes is effectively solved. In addition, when the data frame sequence is used for decision making, corresponding rule strategies do not need to be manually combed and compiled, the relevant data are directly used as training samples to train to obtain the data processing model, and when the strategy or the application scene is changed, the model only needs to be retrained, so that the maintenance and the expansion are easy.
Any of the data processing methods provided by the embodiments of the present disclosure may be performed by any suitable device having data processing capabilities, including but not limited to: terminal equipment, a server and the like. Alternatively, any of the data processing methods provided by the embodiments of the present disclosure may be executed by a processor, for example, the processor may execute any of the data processing methods mentioned in the embodiments of the present disclosure by calling a corresponding instruction stored in a memory. And will not be described in detail below.
Exemplary devices
Fig. 6 is a schematic structural diagram of a data processing apparatus according to an exemplary embodiment of the present disclosure. The present embodiment can be applied to an electronic device, as shown in fig. 6, the data processing apparatus includes: a first determining module 601, configured to determine a category of a current data frame; a second determining module 602, configured to determine at least one piece of periodic data included in the current data frame if the type of the current data frame is a first preset type; a third determining module 603, configured to determine at least one target historical data frame from the data frame sequence in which the current data frame is located; the first processing module 604 is configured to obtain a processing result for a current data frame by using a pre-trained data processing model based on at least one frame of target historical data frame and at least one piece of periodic data.
In this embodiment, the first determining module 601 may determine the category of the current data frame. The current data frame may be a data frame closest to the current time in the sequence of data frames acquired by the electronic device. As an example, the electronic device receives data collected from various data collection devices in real time and combines the collected data into a data frame, or combines data obtained by processing the collected data into a data frame.
As an example, the sequence of data frames may be vehicle and driver information. The dimension of each frame is the number of types of data contained in each frame, and the value of each dimension is the value of each type of data in the frame. Such as data characterizing whether the vehicle is started or not, data characterizing the state of vehicle windows, the volume of audio played in the vehicle, a user ID, a user age, data characterizing the gender of the user, data characterizing the orientation of a human face, data characterizing points of a human face, data characterizing the state of opening and closing human eyes, data characterizing the smoking state of the user, data characterizing the state of a telephone, data characterizing the result of voice recognition, etc.
The above categories may be used to distinguish whether a data frame includes periodic data. For example, a data frame including periodic data may be determined as a first preset category.
In this embodiment, if the type of the current data frame is a first preset type, the second determining module 602 may determine at least one piece of periodic data included in the current data frame. Wherein the data frame belonging to the first preset category may be a data frame including periodic data. The periodic data may be data acquired by the electronic device according to various preset periods, or data obtained by processing the acquired data. For example, the current data frame may include two pieces of periodic data a and B, where the periodic data a is data obtained by periodically performing key point recognition on the acquired face image according to a first preset period (e.g., 100 milliseconds), and the periodic data B is data obtained by periodically performing key point recognition on the acquired voice according to a second preset period (e.g., 10 milliseconds).
In this embodiment, the third determining module 603 may determine at least one target historical data frame from the sequence of data frames in which the current data frame is located. Wherein the target historical data frame is a data frame with which the current data frame is to be processed.
The third determining module 603 may determine at least one target historical data frame from the sequence of data frames in which the current data frame is located using various methods. As an example, a previous data frame adjacent to the current data frame may be determined as the target history data frame, or a preset number of data frames before the current data frame may be determined as the target history data frame.
In this embodiment, the first processing module 604 may obtain a processing result for the current data frame by using a pre-trained data processing model based on at least one frame of the target historical data frame and at least one piece of periodic data. The data processing model is a model obtained by training an initial model in advance by using a preset training sample and a machine learning method. Wherein the initial model may be a model for processing sequence data, for example, the initial model may be an RNN, and the RNN may include, but is not limited to, at least one of: LSTM, GRU, DRNN, etc. As an example, the electronic device may input the current data frame into a data processing model, the data processing model extracts pre-stored data related to each target historical data frame (for example, the historical data frame itself, or periodic data included in the target historical data frame, or intermediate data obtained by pre-processing the target historical data by a hidden layer included in the data processing model), and performs processing such as classification and prediction on the related data and the current data frame to obtain a processing result.
Referring to fig. 7, fig. 7 is a schematic structural diagram of a data processing apparatus according to another exemplary embodiment of the present disclosure.
In some optional implementations, the data processing apparatus may further include: a fourth determining module 605, configured to determine, if the type of the current data frame is a second preset type, a previous data frame adjacent to the current data frame as a target historical data frame; and a second processing module 606, configured to obtain a processing result for the current data frame by using the data processing model based on the target historical data frame.
In some optional implementations, the at least one period data corresponds to a preset data generation period, respectively; and the third determining module 603 may include: a first determination unit 6031 configured to determine a target historical data frame from the sequence of data frames based on data generation periods to which the at least one piece of period data respectively corresponds; a second determining unit 6032 configured to determine a previous data frame adjacent to the current data frame as the target history data frame.
In some optional implementations, the first processing module 604 may include: a third determining unit 6041, configured to input historical intermediate data corresponding to at least one frame of target historical data frame into a hidden layer of the data processing model, and determine intermediate data corresponding to a current data frame through the hidden layer, where the historical intermediate data is intermediate data generated by the hidden layer according to the target historical data frame; and a processing unit 6042, configured to input the intermediate data generated by the hidden layer to an output layer of the data processing model, so as to obtain a processing result.
In some optional implementations, the data processing apparatus may further include: a generating module 606, configured to generate a control instruction corresponding to the processing result based on the processing result; and the execution module 607 is configured to execute an operation corresponding to the control instruction based on the control instruction.
The apparatus provided in the foregoing embodiment of the present disclosure determines at least one period data from a current data frame of a first preset category, determines at least one target historical data frame from a sequence of data frames in which the current data frame is located, and obtains a processing result for the current data frame by using a data processing model based on the at least one target historical data frame and the at least one period data, since processing of the current data frame with only the previous frame data is avoided, embodiments of the present disclosure may effectively utilize the target historical data frame and the periodic data, enable the data processing model to combine the current data frame with the historical data frame corresponding to the periodic data for analysis based on the periodic data, namely, different processing is carried out according to different data generation periods, so that the accuracy and the efficiency of decision making based on the data frame sequence are improved.
Exemplary electronic device
Next, an electronic apparatus according to an embodiment of the present disclosure is described with reference to fig. 8. The electronic device may be either or both of the terminal device 101 and the server 103 as shown in fig. 1, or a stand-alone device separate from them, which may communicate with the terminal device 101 and the server 103 to receive the collected input signals therefrom.
FIG. 8 illustrates a block diagram of an electronic device in accordance with an embodiment of the disclosure.
As shown in fig. 8, an electronic device 800 includes one or more processors 801 and memory 802.
The processor 801 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 800 to perform desired functions.
In one example, the electronic device 800 may further include: an input device 803 and an output device 804, which are interconnected by a bus system and/or other form of connection mechanism (not shown).
For example, when the electronic device is the terminal device 101 or the server 103, the input device 803 may be a camera, a microphone, or the like, for inputting a data frame. When the electronic device is a stand-alone device, the input means 703 may be a communication network connector for receiving data frames from the terminal device 101 and the server 103.
The output device 804 may output various information, including the determined category information, to the outside. The output devices 804 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.
Of course, for simplicity, only some of the components of the electronic device 800 relevant to the present disclosure are shown in fig. 8, omitting components such as buses, input/output interfaces, and the like. In addition, electronic device 800 may include any other suitable components depending on the particular application.
Exemplary computer program product and computer-readable storage Medium
In addition to the above-described methods and apparatus, embodiments of the present disclosure may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the data processing method according to various embodiments of the present disclosure described in the "exemplary methods" section above of this specification.
The computer program product may write program code for carrying out operations for embodiments of the present disclosure in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present disclosure may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform steps in a data processing method according to various embodiments of the present disclosure described in the "exemplary methods" section above of this specification.
The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The foregoing describes the general principles of the present disclosure in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present disclosure are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present disclosure. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the disclosure is not intended to be limited to the specific details so described.
In the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts in the embodiments are referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The block diagrams of devices, apparatuses, systems referred to in this disclosure are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".
The methods and apparatus of the present disclosure may be implemented in a number of ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order for the steps of the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless specifically stated otherwise. Further, in some embodiments, the present disclosure may also be embodied as programs recorded in a recording medium, the programs including machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.
It is also noted that in the devices, apparatuses, and methods of the present disclosure, each component or step can be decomposed and/or recombined. These decompositions and/or recombinations are to be considered equivalents of the present disclosure.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the disclosure to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.
Claims (12)
1. A method of data processing, comprising:
determining the category of the current data frame;
if the type of the current data frame is a first preset type, determining at least one piece of periodic data included in the current data frame;
determining at least one target historical data frame from the data frame sequence in which the current data frame is positioned;
and obtaining a processing result aiming at the current data frame by utilizing a pre-trained data processing model based on the at least one frame of target historical data frame and the at least one piece of periodic data.
2. The method of claim 1, wherein after the determining the category of the current data frame, the method further comprises:
if the type of the current data frame is a second preset type, determining a previous data frame adjacent to the current data frame as a target historical data frame;
and obtaining a processing result aiming at the current data frame by utilizing the data processing model based on the target historical data frame.
3. The method according to claim 1, wherein the at least one piece of periodic data respectively corresponds to a preset data generation period; and
the determining at least one target historical data frame from the data frame sequence in which the current data frame is located includes:
determining target historical data frames from the data frame sequence based on data generation periods respectively corresponding to the at least one piece of periodic data;
and determining the previous data frame adjacent to the current data frame as a target historical data frame.
4. The method of claim 1, wherein the obtaining a processing result for the current data frame based on the at least one frame of target historical data frame and the at least one piece of periodic data by using a pre-trained data processing model comprises:
inputting historical intermediate data respectively corresponding to the at least one frame of target historical data frame into a hidden layer of the data processing model, and determining intermediate data corresponding to the current data frame through the hidden layer, wherein the historical intermediate data is intermediate data generated by the hidden layer according to the target historical data frame;
and inputting the intermediate data generated by the hidden layer into an output layer of the data processing model to obtain a processing result.
5. The method according to one of claims 1-4, wherein the method further comprises:
generating a control instruction corresponding to the processing result based on the processing result;
and executing the operation corresponding to the control instruction based on the control instruction.
6. A data processing apparatus comprising:
the first determining module is used for determining the category of the current data frame;
a second determining module, configured to determine at least one piece of periodic data included in the current data frame if the type of the current data frame is a first preset type;
a third determining module, configured to determine at least one target historical data frame from the data frame sequence in which the current data frame is located;
and the first processing module is used for obtaining a processing result aiming at the current data frame by utilizing a pre-trained data processing model based on the at least one frame of target historical data frame and the at least one piece of periodic data.
7. The apparatus of claim 6, wherein the apparatus further comprises:
the fourth determining module is used for determining a previous data frame adjacent to the current data frame as a target historical data frame if the type of the current data frame is a second preset type;
and the second processing module is used for obtaining a processing result aiming at the current data frame by utilizing the data processing model based on the target historical data frame.
8. The apparatus of claim 6, wherein the at least one piece of periodic data respectively corresponds to a preset data generation period; and
the third determining module includes:
a first determining unit, configured to determine a target historical data frame from the sequence of data frames based on data generation periods respectively corresponding to the at least one piece of period data;
and the second determining unit is used for determining the previous data frame adjacent to the current data frame as the target historical data frame.
9. The apparatus of claim 6, wherein the first processing module comprises:
a third determining unit, configured to input historical intermediate data corresponding to the at least one frame of target historical data frame into a hidden layer of the data processing model, and determine, by using the hidden layer, intermediate data corresponding to the current data frame, where the historical intermediate data is intermediate data generated by the hidden layer according to the target historical data frame;
and the processing unit is used for inputting the intermediate data generated by the hidden layer into an output layer of the data processing model to obtain a processing result.
10. The apparatus according to one of claims 6-9, wherein the apparatus further comprises:
the generating module is used for generating a control instruction corresponding to the processing result based on the processing result;
and the execution module is used for executing the operation corresponding to the control instruction based on the control instruction.
11. A computer-readable storage medium storing a computer program for executing the data processing method of any one of claims 1 to 5.
12. An electronic device, the electronic device comprising:
a processor;
a memory for storing the processor-executable instructions;
the processor for performing the data processing method of any of the preceding claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910420005.2A CN111966721B (en) | 2019-05-20 | 2019-05-20 | Data processing method, data processing device, computer readable storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910420005.2A CN111966721B (en) | 2019-05-20 | 2019-05-20 | Data processing method, data processing device, computer readable storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111966721A true CN111966721A (en) | 2020-11-20 |
| CN111966721B CN111966721B (en) | 2024-05-14 |
Family
ID=73357643
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910420005.2A Active CN111966721B (en) | 2019-05-20 | 2019-05-20 | Data processing method, data processing device, computer readable storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111966721B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104462793A (en) * | 2014-11-25 | 2015-03-25 | 北京数迅科技有限公司 | Real-time time series predicting method and device |
| CN107633254A (en) * | 2017-07-25 | 2018-01-26 | 平安科技（深圳）有限公司 | Establish device, method and the computer-readable recording medium of forecast model |
-
2019
- 2019-05-20 CN CN201910420005.2A patent/CN111966721B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104462793A (en) * | 2014-11-25 | 2015-03-25 | 北京数迅科技有限公司 | Real-time time series predicting method and device |
| CN107633254A (en) * | 2017-07-25 | 2018-01-26 | 平安科技（深圳）有限公司 | Establish device, method and the computer-readable recording medium of forecast model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111966721B (en) | 2024-05-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102137151B1 (en) | Apparatus for noise canceling and method for the same | |
| JP6953464B2 (en) | Information push method and equipment | |
| US11715485B2 (en) | Artificial intelligence apparatus for converting text and speech in consideration of style and method for the same | |
| KR20190123362A (en) | Method and Apparatus for Analyzing Voice Dialogue Using Artificial Intelligence | |
| US20200219384A1 (en) | Methods and systems for ambient system control | |
| Elmalaki et al. | Sentio: Driver-in-the-loop forward collision warning using multisample reinforcement learning | |
| CN112397065A (en) | Voice interaction method and device, computer readable storage medium and electronic equipment | |
| CN112036955B (en) | User identification method and device, computer readable storage medium and electronic equipment | |
| US11531881B2 (en) | Artificial intelligence apparatus for controlling auto stop system based on driving information and method for the same | |
| CN114038457B (en) | Method, electronic device, storage medium, and program for voice wakeup | |
| CN112307816A (en) | In-vehicle image acquisition method and device, electronic equipment and storage medium | |
| CN110555346A (en) | Driver emotion detection method and device, electronic equipment and storage medium | |
| US20210407517A1 (en) | Artificial intelligence robot for providing voice recognition function and method of operating the same | |
| CN115376559A (en) | Emotion recognition method, device and equipment based on audio and video | |
| Darekar et al. | A hybrid meta-heuristic ensemble based classification technique speech emotion recognition | |
| CN113469023A (en) | Method, device, equipment and storage medium for determining alertness | |
| CN113053377A (en) | Voice wake-up method and device, computer readable storage medium and electronic equipment | |
| CN109634554B (en) | Method and device for outputting information | |
| CN111966721B (en) | Data processing method, data processing device, computer readable storage medium and electronic equipment | |
| US20210337274A1 (en) | Artificial intelligence apparatus and method for providing visual information | |
| WO2022193892A1 (en) | Speech interaction method and apparatus, and computer-readable storage medium and electronic device | |
| CN112115325B (en) | Scene category determining method and scene analysis model training method and device | |
| CN115862597A (en) | Method and device for determining character type, electronic equipment and storage medium | |
| Giri et al. | Enhancing Safety in Vehicles using Emotion Recognition with Artificial Intelligence | |
| CN112185367A (en) | Keyword detection method and device, computer readable storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |