CN115527087B

CN115527087B - Method and device for determining behavior information, storage medium and electronic device

Info

Publication number: CN115527087B
Application number: CN202211373495.3A
Authority: CN
Inventors: 倪华健; 易芮; 林亦宁
Original assignee: Beijing Shanma Zhijian Technology Co ltd; Hangzhou Shanma Zhiqing Technology Co Ltd; Shanghai Supremind Intelligent Technology Co Ltd
Current assignee: Beijing Shanma Zhijian Technology Co ltd; Hangzhou Shanma Zhiqing Technology Co Ltd; Shanghai Supremind Intelligent Technology Co Ltd
Priority date: 2022-11-04
Filing date: 2022-11-04
Publication date: 2023-07-14
Anticipated expiration: 2042-11-04
Also published as: CN115527087A

Abstract

The embodiment of the invention provides a method and a device for determining behavior information, a storage medium and an electronic device, wherein the method comprises the following steps: inputting target training data into an initial network model to obtain loss values of N sub-network outputs included in the initial network model; determining a first update gradient of each sub-network based on the loss values output by the N sub-networks and the backbone network parameters of the backbone network; normalizing the first update gradients of the N sub-networks to obtain N second update gradients; determining a target update gradient for updating the sub-network parameters of the N sub-networks based on the N second update gradients; and updating the sub-network parameters of the N sub-networks according to the target updating gradient to obtain a target network model. The invention solves the problem of poor performance of the network model obtained by training in the related technology, achieves the effect of improving the performance of the target network model, and improves the identification accuracy of the target network model.

Description

Method and device for determining behavior information, storage medium and electronic device

Technical Field

The embodiment of the invention relates to the field of computers, in particular to a method and a device for determining behavior information, a storage medium and an electronic device.

Background

In recent years, with further development of artificial intelligence and increase of intelligent management demands, management behavior analysis on target objects is also increasing.

For example, in the field of traffic management, there are increasingly more dregs cars which do not travel along a prescribed route without covering tarpaulin according to the regulations, and thus urban capacity and environment are seriously affected. Or in the financial field, the behavioral changes of risks of agents need to be simulated and analyzed to reduce financial risks and conduct risk planning in advance.

While the challenges faced by behavioral auditing are mainly manifested in: 1. the human resource cost is high, and the traditional auditing mainly depends on illegal pictures captured by manpower from a large number of monitoring cameras or historical behavior data captured from a financial system. 2. The supervision rules are difficult to unify, the auditing efficiency is low, and when illegal data are selected manually, misjudgment or misjudgment is caused due to the fact that judgment standards are not unified, and the auditing efficiency and quality are difficult to guarantee.

The following description will be made by taking the analysis of the behavior of the muck truck as an example:

in the related art, a multitasking network is generally used for identifying the existing illegal actions of the muck, however, when a plurality of tasks are trained in one network, the learning speed of each task is inconsistent due to the differences of data, learning targets and the like, the learning speed of each task is too high, the learning speed of each task is too slow, the training variability is caused, the learned performance of each task is inconsistent, and thus the action of an object identified by using a network model is inaccurate.

As can be seen from the above, the related art has a problem that the behavior of the object to be determined is inaccurate due to poor performance of the network model obtained by training.

In view of the above problems in the related art, no effective solution has been proposed at present.

Disclosure of Invention

The embodiment of the invention provides a method and a device for determining behavior information, a storage medium and an electronic device, which are used for at least solving the problem that the behavior of a determined object is inaccurate due to poor performance of a network model obtained through training in the related technology.

According to an embodiment of the present invention, there is provided a method for determining behavior information, including: inputting target data into a target network model, identifying the target data from different dimensions by utilizing a plurality of sub-networks included in the target network model, and determining behavior information of a target object included in the target data; the target network model is trained by the following modes: inputting target training data into an initial network model to obtain loss values output by N sub-networks included in the initial network model, wherein the N sub-networks are used for identifying the target training data from different dimensions, and N is an integer greater than or equal to 2; determining a first update gradient of each sub-network based on the loss values output by the N sub-networks and a main network parameter of a main network, wherein the main network is a network included in the initial network model; normalizing the first update gradients of the N sub-networks to obtain N second update gradients; determining a target update gradient for updating the sub-network parameters of the N sub-networks based on the N second update gradients; and updating the sub-network parameters of the N sub-networks according to the target updating gradient to obtain a target network model, wherein the target training data and the target data are the same type of data.

According to another embodiment of the present invention, there is provided a behavior information determining apparatus including: the determining module is used for inputting target data into a target network model, identifying the target data from different dimensions by utilizing a plurality of sub-networks included in the target network model, and determining behavior information of a target object included in the target data; the target network model is trained by the following modes: inputting target training data into an initial network model to obtain loss values output by N sub-networks included in the initial network model, wherein the N sub-networks are used for identifying the target training data from different dimensions, and N is an integer greater than or equal to 2; determining a first update gradient of each sub-network based on the loss values output by the N sub-networks and a main network parameter of a main network, wherein the main network is a network included in the initial network model; normalizing the first update gradients of the N sub-networks to obtain N second update gradients; a second determining module, configured to determine a target update gradient for updating the sub-network parameters of the N sub-networks based on the N second update gradients; the updating module is used for updating the sub-network parameters of the N sub-networks according to the target updating gradient to obtain a target network model; the target training data and the target data are the same type of data.

According to yet another embodiment of the present invention, there is also provided a computer-readable storage medium having stored therein a computer program, wherein the computer program when executed by a processor implements the steps of the method as described in any of the above.

According to a further embodiment of the invention, there is also provided an electronic device comprising a memory having stored therein a computer program and a processor arranged to run the computer program to perform the steps of any of the method embodiments described above.

According to the invention, the target data are input into the target network model, the target data are identified from different dimensions by utilizing a plurality of sub-networks included in the target network model, and the behavior information of the target object included in the target data is determined; the target network model is trained by the following modes: inputting the target training data into a value initial network model to obtain N sub-network output loss values included in the initial network model, determining a first update gradient of each sub-network according to the N sub-network output loss values and the main network parameters of the main network included in the initial network model, carrying out normalization processing on the first update gradients of the N sub-networks to obtain N second update gradients, determining a target update gradient for updating the sub-network parameters of the N sub-networks according to the N second update gradients, and updating the sub-network parameters of the N sub-networks according to the target update gradient to obtain the target network model. When the model is trained, the first update gradient of each sub-network can be normalized to obtain a second update gradient, and the target update gradient is determined according to the second update gradient, so that the learning speed among the sub-networks is balanced, the convergence is accelerated, the obtained target network model is more stable, the problem that the behavior of the determined object is inaccurate due to poor performance of the network model obtained by training in the related art can be solved, and the effect of improving the accuracy of the behavior of the determined object is achieved.

Drawings

Fig. 1 is a block diagram of a hardware structure of a mobile terminal according to a method for determining behavior information according to an embodiment of the present invention;

FIG. 2 is a flow chart of a method of determining behavioral information according to an embodiment of the invention;

FIG. 3 is a flowchart of a method for determining behavior information, according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of determining adverse behavior in accordance with an embodiment of the present invention;

fig. 5 is a block diagram of a configuration of a determination apparatus of behavior information according to an embodiment of the present invention.

Detailed Description

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings in conjunction with the embodiments.

It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order.

The method embodiments provided in the embodiments of the present application may be performed in a mobile terminal, a computer terminal or similar computing device. Taking the mobile terminal as an example, fig. 1 is a block diagram of a hardware structure of the mobile terminal according to a method for determining behavior information according to an embodiment of the present invention. As shown in fig. 1, a mobile terminal may include one or more (only one is shown in fig. 1) processors 102 (the processor 102 may include, but is not limited to, a microprocessor MCU or a processing device such as a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 for communication functions and an input-output device 108. It will be appreciated by those skilled in the art that the structure shown in fig. 1 is merely illustrative and not limiting of the structure of the mobile terminal described above. For example, the mobile terminal may also include more or fewer components than shown in fig. 1, or have a different configuration than shown in fig. 1.

The memory 104 may be used to store a computer program, for example, a software program of application software and a module, such as a computer program corresponding to a method for determining behavior information in an embodiment of the present invention, and the processor 102 executes the computer program stored in the memory 104 to perform various functional applications and data processing, that is, to implement the above-described method. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory remotely located relative to the processor 102, which may be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission device 106 is used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, simply referred to as NIC) that can connect to other network devices through a base station to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is configured to communicate with the internet wirelessly.

In this embodiment, a method for determining behavior information is provided, fig. 2 is a flowchart of a method for determining behavior information according to an embodiment of the present invention, and as shown in fig. 2, the flowchart includes the following steps:

step S202, inputting target data into a target network model, identifying the target data from different dimensions by utilizing a plurality of sub-networks included in the target network model, and determining behavior information of a target object included in the target data;

the target network model is trained by the following modes: inputting target training data into an initial network model to obtain loss values output by N sub-networks included in the initial network model, wherein the N sub-networks are used for identifying the target training data from different dimensions, and N is an integer greater than or equal to 2;

determining a first update gradient of each sub-network based on the loss values output by the N sub-networks and a main network parameter of a main network, wherein the main network is a network included in the initial network model;

normalizing the first update gradients of the N sub-networks to obtain N second update gradients;

determining a target update gradient for updating the sub-network parameters of the N sub-networks based on the N second update gradients;

and updating the sub-network parameters of the N sub-networks according to the target updating gradient to obtain a target network model, wherein the target training data and the target data are the same type of data.

In the above embodiment, the target training data includes N types of training data, each type of training data is used for training one sub-network, and the types of training data of each sub-network are different. The number of training data of each type is of the same order of magnitude, e.g., the number of training data of each type is the same. The training data includes images and behavior types of objects included in the images. Wherein the training object may comprise a vehicle, a person, etc. The vehicle may include a road and earth vehicle, a non-motor vehicle, a car or other type of motor vehicle, etc.

In the above embodiment, the initial network model may be trained first, when the loss value of the sub-network is greater than a predetermined threshold value, or the number of training times is less than a predetermined number of times, a target update gradient for updating the sub-network parameters of the sub-network included in the initial network model is determined, the sub-network parameters are updated according to the target update gradient, the updated initial network model is obtained, and training is performed again until the loss value of the sub-network is less than or equal to the predetermined threshold value, or the number of training times reaches the predetermined number of times, the training is exited, and the finally obtained initial network model is determined as the target network model.

In the above embodiment, when the initial network model is trained, the target training data may be first input into the initial network model, and the loss value of each sub-network output may be determined. In the above embodiment, after determining the loss value output by each sub-network, the first update gradient of each sub-network may be determined according to the loss value and the backbone network parameter of the backbone network. The backbone network may be a convolutional neural network, which is used to extract features of the target training data. And inputting the extracted features into each sub-network so that each sub-network can obtain a recognition result according to the features.

In the above embodiment, after the first update gradient is obtained, normalization processing may be performed on the plurality of first update gradients to obtain a second update gradient, and the target update gradient may be determined according to the second update gradient. The learning speed of each sub-network is balanced, and the convergence speed and the learning effect of the initial network model are improved.

In the above embodiment, after the target network model is obtained, the obtained target data may be input into the target network model, and behavior information of the target object included in the target data is identified from different dimensions using a plurality of sub-networks included in the target network model, that is, one sub-network is used to perform one type of task, and each sub-network performs a different type of task. The target data may be an image captured by an image capturing device in a traffic post. The target object may include a vehicle, a person, and the like. The vehicle may include a road and earth vehicle, a non-motor vehicle, a car or other type of motor vehicle, etc. The task type may be determined according to an application scenario or a target object of the target network model. For example, when the target network model is used to determine whether there is an offence in the earth-moving vehicle, the task types may include: whether retrograde, on-road time, whether to carry cargo, whether to cover tarpaulin, etc. When the target network model is used to determine whether there is a violation by a non-maneuver, the task types may include: whether retrograde, whether to wear a helmet, whether to carry people, etc.

In the above embodiment, after determining the behavior of the target object by using the target network model, when there is a violation in the behavior of the object, the target data and the violation type may be uploaded to the cloud platform, and an alarm or prompt operation is performed.

In the above embodiment, the target data and the target training data for training the target network model belong to the same type of data, for example, when the target data is a picture, the target training data is also a picture. When the target data is a feature map, the target training data is also a feature map.

Alternatively, the main body of execution of the above steps may be a background processor, or other devices with similar processing capability, and may also be a machine integrated with at least a data processing device, where the data processing device may include, but is not limited to, a terminal such as a computer, a mobile phone, and the like.

In one exemplary embodiment, determining the first update gradient for each of the sub-networks based on the loss values output by the N sub-networks and the backbone network parameters of the backbone network comprises: for each of the sub-networks, performing the following operations to obtain the first update gradient of each sub-network: determining a partial derivative of the loss value with respect to the backbone network parameter; the partial derivative is determined as the first update gradient. In this embodiment, the formula may be used

To determine a first update gradient. Wherein (1)>

Representing a first update gradient, loss, of an ith subnetwork included in the N subnetworks _i Representing the loss value of the ith sub-network, w represents the backbone network parameters of the backbone network.

In an exemplary embodiment, normalizing the first update gradients of the N sub-networks to obtain N second update gradients includes: determining a first average of the N first update gradients; determining variances of N of the first update gradients based on the first average; and determining the second update gradient corresponding to each first update gradient based on the variance and the first average value. In this embodiment, when performing multiple first update gradient normalization, one can use

To represent a first average value. Determining N first updates from the first averageAnd determining the variance of the gradients, and determining a second updating gradient corresponding to each first updating gradient according to the variance and the first average value.

In one exemplary embodiment, determining the second update gradient corresponding to each of the first update gradients based on the variance and the first average value includes: the following operations are executed for each first update gradient, and the second update gradient corresponding to each first update gradient is obtained: determining a first difference of the first update gradient and the first average value; a ratio of the first difference to the variance is determined as the second update gradient. In this embodiment, the second update gradient may be represented as

. Wherein (1)>

Represents a first average,/->

A first update gradient representing an ith sub-network, < >>

Representing the variance.

In one exemplary embodiment, determining the variance of each of the first update gradients based on the first average value comprises: determining a second difference for each of the first update gradients from the first average; determining the square value of each second difference value to obtain N square values; determining a second average of the N square values; the arithmetic square root of the second mean is determined as the variance. In this embodiment, the variance can be expressed as

。

In one exemplary embodiment, a target update for updating the subnet parameters of the N subnets is determined based on the N second update gradientsThe gradient includes: determining a sum of N second update gradients; determining a ratio of each of the second update gradients to the sum value as a network weight of the sub-network corresponding to the second update gradient; determining the product of each second updating gradient and the network weight to obtain N products; and determining the sum of N products as the target update gradient. In this embodiment, the network weights may be expressed as

The target update gradient may be expressed as +.>

。

In an exemplary embodiment, before inputting the target training data into the initial network model, obtaining the loss values of the N sub-network outputs included in the initial network model, the method further includes: acquiring N types of training data, wherein one type of training data is used for training one sub-network, different sub-networks correspond to different types of training data, and the quantity of each type of training data is the same; preprocessing target type training data included in the N types of training data to obtain processed training data; and determining the processed training data and other training data as the target training data, wherein the other training data are training data of N types except the training data of the target type. In this embodiment, training data may be acquired prior to training the initial network model. The initial network model includes N sub-networks, so that N types of training data can be acquired, where each type of training data is used to train one sub-network, and the training data of each sub-network is different. For example, when the target network model is used for identifying the illegal behavior of the urban sediment and soil truck and comprises 3 sub-networks, the input network is a training data set of 3 tasks, which are respectively: (1) a head orientation data set of a muck truck, (2) a city vehicle data set, and (3) a muck truck coverage condition data set. And carrying out class balancing on the data sets of the 3 tasks, carrying out class balancing on different class data sampling repeated sampling strategies of the same task, and ensuring that different class data of the same task are in the same range, wherein some class data of a certain task are not excessive and other class data are too little. Aiming at the covering condition data set of the slag car of the 3 rd task, each picture is cut and reserved for 1/2 of the upper half picture in order to improve the recognition precision, the view angle proportion of the car hopper part area is increased, and the recognition precision is improved. I.e. the target type may be an overlay type. The preprocessing may be clipping processing.

The method for determining behavior information is described below with reference to the specific embodiments:

fig. 3 is a flowchart of a method for determining behavior information according to an embodiment of the present invention, as shown in fig. 3,

the multi-task network for training urban muck vehicle violation identification is characterized in that the input network is a data set of 3 tasks, and the data sets are respectively as follows: (1) a head orientation data set of a muck truck, (2) a city vehicle data set, and (3) a muck truck coverage condition data set. And carrying out class balancing on the data sets of the 3 tasks, carrying out class balancing on different class data sampling repeated sampling strategies of the same task, and ensuring that different class data of the same task are in the same range, wherein some class data of a certain task are not excessive and other class data are too little.

Aiming at the covering condition data set of the slag car of the 3 rd task, each picture is cut and reserved for 1/2 of the upper half picture in order to improve the recognition precision, the view angle proportion of the car hopper part area is increased, and the recognition precision is improved.

3. The processed 3 task data sets are sent to a multi-task network for training, and the learning speeds of a plurality of tasks are possibly different due to different task data conditions and learning targets, so that the self-adaptive gradient updating method can be used for updating the learning speeds of the 3 tasks, ensuring the learning speeds of the 3 tasks to be balanced, and accelerating the convergence speed and the learning effect.

（1）

（2）/>

（3）

（4）

（5）

（6）

Wherein, formula (1)

Gradient representing the ith task, i=1, 2,3, +.>

The Loss indicating the ith task and W indicating the parameters shared in the backup. +.>

The regularized gradient of the ith task is extremely small in the training process, and the gradient value difference range of the 3 tasks is uncertain, if the magnitude order of gradient difference between the tasks is large, the effect is deficient because the gradient value is directly balanced. For this purpose, gradient normalization operations are performed, wherein the gradient of each task is mapped into the same scale, and the gradient normalization operations are shown in formulas (2) - (4), respectively>

Represents the mean value of the gradient,/->

Representing the variance of the gradient, n represents the total number of tasks. Gradient +.3 regularized for each iteration>

Solving gradient duty ratio, see formula (5), to obtain weight of each gradient>

. Weighting based on the weight of each gradient to obtain adaptively weighted gradient, obtaining total gradient +.>

Compared with the previous gradient, the method sets corresponding weight for each task, so that each task obtains learning speed suitable for the user, network convergence is faster, and performance is improved.

The network outputs the results of 3 tasks, which are respectively: (1) the head of the dregs car faces forward and backward. (2) a muck truck and a non-muck truck category. (3) Covered muck trucks, uncovered empty muck trucks, uncovered non-empty muck trucks. The 2 categories of task 1 are used to determine whether the muck vehicle is driving in reverse, and the 2 categories of task 2 are used to determine whether the muck vehicle is driving on a prescribed time period and a prescribed road, and a general city is not allowed to run in urban areas during the daytime. The 3 categories of task 3 are used to determine whether the earth and slag vehicle is covered as required.

Carrying out violation analysis on the results of the 3 tasks, and judging that the vehicle runs in reverse if the current vehicle head orientation is not in accordance with the requirements by capturing the current vehicle head orientation by the cameras controlled on the basis of the current road for the output result of the task 1; the schematic diagram for determining the adverse behavior can be seen in fig. 4. If the output result of the task 2 is that the muck vehicle is identified based on the current urban road, and the specified daytime traffic peak period which does not allow the muck vehicle to run is the time, determining that the rule is illegal; and (3) if the output result of the task 3 identifies that the muck truck is not empty and not covered, judging that the muck truck is illegal.

Uploading the violation information to a cloud end, and notifying a supervisor to carry out supervision treatment.

In the embodiment, a plurality of soil and slag vehicle illegal behaviors are integrated into one network for training, so that the training task is simplified, and the performance of the network is ensured. The regularized self-adaptive gradient is used in multi-task classification, so that the learning speed among tasks is balanced, the convergence is accelerated, and the model is more stable and has better performance. Compared with the traditional method by manual identification, the method for identifying the urban muck vehicle violations by using the deep learning algorithm has the advantages that the efficiency is greatly improved, and the labor cost is reduced. The intelligent supervision is implemented on the illegal behaviors of the dregs car, so that the sense of safety of the city can be increased, the image of the city can be improved, and the concept of green city is practiced.

From the description of the above embodiments, it will be clear to a person skilled in the art that the method according to the above embodiments may be implemented by means of software plus the necessary general hardware platform, but of course also by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present invention.

The embodiment also provides a device for determining behavior information, which is used for implementing the foregoing embodiments and preferred embodiments, and is not described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

Fig. 5 is a block diagram of a configuration of a device for determining behavior information according to an embodiment of the present invention, as shown in fig. 5, the device including:

a determining module 52, configured to input target data into a target network model, identify the target data from different dimensions by using a plurality of sub-networks included in the target network model, and determine behavior information of a target object included in the target data;

the target network model is trained by the following modes: inputting target training data into an initial network model to obtain loss values output by N sub-networks included in the initial network model, wherein the N sub-networks are used for identifying the target training data from different dimensions, and N is an integer greater than or equal to 2; determining a first update gradient of each sub-network based on the loss values output by the N sub-networks and a main network parameter of a main network, wherein the main network is a network included in the initial network model; normalizing the first update gradients of the N sub-networks to obtain N second update gradients; determining a target update gradient for updating the sub-network parameters of the N sub-networks based on the N second update gradients; and updating the sub-network parameters of the N sub-networks according to the target updating gradient to obtain a target network model, wherein the target training data and the target data are the same type of data.

In an exemplary embodiment, the apparatus may determine the first update gradient of each of the sub-networks based on the loss values of the N sub-network outputs and the backbone network parameters of the backbone network by: for each of the sub-networks, performing the following operations to obtain the first update gradient of each sub-network: determining a partial derivative of the loss value with respect to the backbone network parameter; the partial derivative is determined as the first update gradient.

In an exemplary embodiment, the apparatus may perform normalization processing on the first update gradients of the N sub-networks to obtain N second update gradients by: determining a first average of the N first update gradients; determining variances of N of the first update gradients based on the first average; and determining the second update gradient corresponding to each first update gradient based on the variance and the first average value.

In an exemplary embodiment, the apparatus may determine the second update gradient corresponding to each of the first update gradients based on the variance and the first average by: the following operations are executed for each first update gradient, and the second update gradient corresponding to each first update gradient is obtained: determining a first difference of the first update gradient and the first average value; a ratio of the first difference to the variance is determined as the second update gradient.

In an exemplary embodiment, the apparatus may be configured to determine the variance of each of the first update gradients based on the first average value by: determining a second difference for each of the first update gradients from the first average; determining the square value of each second difference value to obtain N square values; determining a second average of the N square values; the arithmetic square root of the second mean is determined as the variance.

In an exemplary embodiment, the apparatus may determine the target update gradient for updating the sub-network parameters of the N sub-networks based on the N second update gradients by: determining a sum of N second update gradients; determining a ratio of each of the second update gradients to the sum value as a network weight of the sub-network corresponding to the second update gradient; determining the product of each second updating gradient and the network weight to obtain N products; and determining the sum of N products as the target update gradient.

In an exemplary embodiment, the apparatus is further configured to, before inputting the target training data into the initial network model, obtain loss values of N sub-network outputs included in the initial network model, include: acquiring N types of training data, wherein one type of training data is used for training one sub-network, different sub-networks correspond to different types of training data, and the quantity of each type of training data is the same; preprocessing target type training data included in the N types of training data to obtain processed training data; and determining the processed training data and other training data as the target training data, wherein the other training data are training data of N types except the training data of the target type.

It should be noted that each of the above modules may be implemented by software or hardware, and for the latter, it may be implemented by, but not limited to: the modules are all located in the same processor; alternatively, the above modules may be located in different processors in any combination.

Embodiments of the present invention also provide a computer readable storage medium having a computer program stored therein, wherein the computer program when executed by a processor implements the steps of the method described in any of the above.

In one exemplary embodiment, the computer readable storage medium may include, but is not limited to: a usb disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing a computer program.

An embodiment of the invention also provides an electronic device comprising a memory having stored therein a computer program and a processor arranged to run the computer program to perform the steps of any of the method embodiments described above.

In an exemplary embodiment, the electronic apparatus may further include a transmission device connected to the processor, and an input/output device connected to the processor.

Specific examples in this embodiment may refer to the examples described in the foregoing embodiments and the exemplary implementation, and this embodiment is not described herein.

It will be appreciated by those skilled in the art that the modules or steps of the invention described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may be implemented in program code executable by computing devices, so that they may be stored in a storage device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than that shown or described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.

The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention should be included in the protection scope of the present invention.

Claims

1. The method for determining the behavior information is characterized by being applied to the field of vehicle behavior recognition and comprising the following steps of:

inputting target data into a target network model, identifying the target data from different dimensions by utilizing a plurality of sub-networks included in the target network model, and determining behavior information of a target object included in the target data, wherein the target data is an acquired vehicle image;

the target network model is trained by the following modes: inputting target training data into an initial network model to obtain loss values output by N sub-networks included in the initial network model, wherein the N sub-networks are used for identifying the target training data from different dimensions, and N is an integer greater than or equal to 2; determining a first update gradient of each sub-network based on the loss values output by the N sub-networks and a main network parameter of a main network, wherein the main network is a network included in the initial network model; normalizing the first update gradients of the N sub-networks to obtain N second update gradients; determining a target update gradient for updating the sub-network parameters of the N sub-networks based on the N second update gradients; updating the sub-network parameters of N sub-networks according to the target updating gradient to obtain a target network model, wherein the target training data and the target data are the same type of data;

the normalization processing is performed on the first update gradients of the N sub-networks, and obtaining N second update gradients includes:

determining a first average of the N first update gradients;

determining variances of N of the first update gradients based on the first average;

determining the second update gradient corresponding to each first update gradient based on the variance and the first average value;

wherein determining a target update gradient for updating the sub-network parameters of the N sub-networks based on the N second update gradients comprises:

determining a sum of N second update gradients;

determining a ratio of each of the second update gradients to the sum value as a network weight of the sub-network corresponding to the second update gradient;

determining the product of each second updating gradient and the network weight to obtain N products;

determining a sum of N said products as said target update gradient;

before inputting the target training data into the initial network model to obtain the loss values of the N sub-network outputs included in the initial network model, the method further includes:

dividing the target training data into N types of training data;

and acquiring the N types of training data, wherein one type of training data is used for training one sub-network, different sub-networks correspond to different types of training data, and the quantity of each type of training data is the same.

2. The method of claim 1, wherein determining a first update gradient for each of the sub-networks based on the loss values of the N sub-network outputs and the backbone network parameters of the backbone network comprises:

for each of the sub-networks, performing the following operations to obtain the first update gradient of each sub-network:

determining a partial derivative of the loss value with respect to the backbone network parameter;

the partial derivative is determined as the first update gradient.

3. The method of claim 1, wherein determining the second update gradient for each of the first update gradients based on the variance and the first average comprises:

the following operations are executed for each first update gradient, and the second update gradient corresponding to each first update gradient is obtained:

determining a first difference of the first update gradient and the first average value;

a ratio of the first difference to the variance is determined as the second update gradient.

4. The method of claim 1, wherein determining the variance of each of the first update gradients based on the first average value comprises:

determining a second difference for each of the first update gradients from the first average;

determining the square value of each second difference value to obtain N square values;

determining a second average of the N square values;

the arithmetic square root of the second mean is determined as the variance.

5. The method of claim 1, wherein prior to inputting the target training data into the initial network model to obtain the loss values for the N sub-network outputs included in the initial network model, the method further comprises:

preprocessing target type training data included in the N types of training data to obtain processed training data;

and determining the processed training data and other training data as the target training data, wherein the other training data are training data of N types except the training data of the target type.

6. A behavior information determining apparatus, comprising:

a determining module, configured to input target data into a target network model, identify the target data from different dimensions by using a plurality of sub-networks included in the target network model, and determine behavior information of a target object included in the target data, where the target data is an acquired vehicle image;

the apparatus is further for determining a first average of the N first update gradients; determining variances of N of the first update gradients based on the first average; determining the second update gradient corresponding to each first update gradient based on the variance and the first average value;

the apparatus is further for determining a sum of N of the second update gradients; determining a ratio of each of the second update gradients to the sum value as a network weight of the sub-network corresponding to the second update gradient; determining the product of each second updating gradient and the network weight to obtain N products; determining a sum of N said products as said target update gradient;

the device is further used for dividing the target training data into N types of training data before inputting the target training data into an initial network model to obtain loss values of N sub-network outputs included in the initial network model;

7. A computer readable storage medium, characterized in that a computer program is stored in the computer readable storage medium, wherein the computer program, when being executed by a processor, implements the steps of the method according to any of the claims 1 to 5.

8. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to run the computer program to perform the method of any of the claims 1 to 5.