CN112733901B - Structured action classification method and device based on federal learning and blockchain - Google Patents
Structured action classification method and device based on federal learning and blockchain Download PDFInfo
- Publication number
- CN112733901B CN112733901B CN202011621661.8A CN202011621661A CN112733901B CN 112733901 B CN112733901 B CN 112733901B CN 202011621661 A CN202011621661 A CN 202011621661A CN 112733901 B CN112733901 B CN 112733901B
- Authority
- CN
- China
- Prior art keywords
- action classification
- action
- blockchain
- structured
- classification result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000009471 action Effects 0.000 title claims abstract description 107
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000012549 training Methods 0.000 claims description 31
- 230000006870 function Effects 0.000 claims description 29
- 238000004590 computer program Methods 0.000 claims description 18
- 230000004927 fusion Effects 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 4
- 238000012886 linear function Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 description 9
- 239000011159 matrix material Substances 0.000 description 7
- 230000005540 biological transmission Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000005457 optimization Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000000638 solvent extraction Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/94—Hardware or software architectures specially adapted for image or video understanding
- G06V10/95—Hardware or software architectures specially adapted for image or video understanding structured as a network, e.g. client-server architectures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Multimedia (AREA)
- Bioethics (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention belongs to the technical field of blockchains and provides a structured action classification method and a structured action classification device based on federal learning and blockchains.
Description
Technical Field
The present application relates to the field of blockchain technology, and in particular, to a structured action classification method, system, apparatus, computer device and computer readable storage medium based on federal learning and blockchain.
Background
At present, a motion classification method based on a two-dimensional image is paid attention to, for example, application of human motion classification in the fields of security and protection and the like. However, the action-issuing subject (e.g., human body) is a non-rigid body and is often in a complex scene. Therefore, image acquisition from different angles by using a plurality of devices to obtain comprehensive classification results is an important method for improving classification effect.
However, in the prior art, the original main image and other information are transmitted between the edge computing device and the server, privacy is easy to be revealed, and the transmission cost is high; because no proper training model is adopted, the action classification precision is not high, a consensus mechanism is not established between the devices through the block chain, the action classification can be requested without classification authority, the load of the server is increased, and the server is easy to be abused.
In summary, the prior art has the technical problems of low security, low classification precision, high transmission cost, easy abuse of the server, and the like.
Disclosure of Invention
To solve the above technical problems, the present invention provides a structured action classification method, system, apparatus, computer device and computer readable storage medium based on federal learning and blockchain.
A structured action classification method based on federal learning and blockchain, comprising:
receiving image characteristics of a plurality of edge computing devices of an information uploading block chain and action classification requests aiming at the image characteristics;
synchronizing the equipment for classifying the same image characteristic request in the action classification request;
acquiring the image feature training loss function model to obtain an action classification result of a single image feature close to a real value;
and modeling the structured output of the action classification result by using a boundary structured output learning framework to obtain a fused action classification result and sending the fused action classification result to the request equipment.
A structured action classification device based on federal learning and blockchain, comprising:
the receiving module is used for receiving image characteristics of a plurality of edge computing devices of the information uploading block chain and action classification requests aiming at the image characteristics;
the synchronization module is used for synchronizing the equipment which is required to be classified aiming at the same image characteristic in the action classification request;
the training module is used for acquiring the image feature training loss function model and obtaining an action classification result of the single image feature close to the real value;
and the modeling module is used for modeling the structured output of the action classification result by using the boundary structured output learning framework so as to obtain a fusion action classification result and sending the fusion action classification result to the request equipment.
A structured action classification system based on federal learning and blockchain, comprising: a plurality of edge computing devices, servers, and blockchains;
the plurality of edge computing devices communicate through the blockchain; the server internally runs a receiving module, a synchronizing module, a training module and a modeling module;
the receiving module receives image characteristics of a plurality of edge computing devices of an information uploading block chain and action classification requests aiming at the image characteristics;
the synchronization module synchronizes the equipment which is required to be classified aiming at the same image characteristic in the action classification request;
the training module acquires the image feature training loss function model to obtain an action classification result of a single image feature close to a real value;
the modeling module uses a boundary structured output learning framework to model the structured output of the action classification result so as to obtain a fused action classification result and send the fused action classification result to the request device.
A computer device comprising a memory and a processor, the memory storing a computer program that performs an implementable method in the processor.
A computer readable storage medium storing a computer program, the computer program being executed in a processor to implement the method described above.
The invention provides a structured action classification method, a structured action classification device and a structured action classification system based on federal learning and blockchains, which are characterized in that by receiving image features of a plurality of edge computing devices of information uploading blockchains and action classification requests aiming at the image features, synchronizing the devices aiming at the same image feature request classification in the action classification requests, acquiring an image feature training loss function model to obtain action classification results of single image features close to real values, modeling structured output of the action classification results by using a boundary structured output learning frame, so as to obtain a fused action classification result and send the fused action classification result to requesting devices, so that only the image features are uploaded and the identification result is downloaded between edge computing devices and a server, the original image and other information is not transmitted, privacy is guaranteed, the data volume of the features is smaller than that of the original image, transmission cost is reduced, training data on different edge computing devices is collected, the precision of the model is improved, and the model is prevented from being abused by unverified devices by using a blockchain protection model.
Drawings
FIG. 1 is a schematic diagram of a structured action classification system based on federal learning and blockchain according to an embodiment;
FIG. 2 is a flow chart of a structured action classification method based on federal learning and blockchain according to an embodiment;
FIG. 3 is a schematic diagram of a similar image frame;
FIG. 4 is a schematic diagram of a structured motion classification device based on federal learning and blockchain according to an embodiment;
fig. 5 is a schematic diagram of a framework of a computer device according to an embodiment.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The structured action classification method based on federal learning and blockchain provided in this embodiment may be applied in an application environment as shown in fig. 1, where an edge computing device communicates with a server through a network, and the server operates in a blockchain system. Among other things, edge computing devices may be, but are not limited to, various personal computers, notebook computers, smartphones, tablet computers, and portable wearable devices. The server may be implemented as a stand-alone server or as a server cluster composed of a plurality of servers.
In one embodiment, as shown in fig. 2, a structured action classification method based on federal learning and blockchain is provided, and the method is applied to the server in fig. 1 for illustration, and includes the following steps:
s1, receiving image characteristics of a plurality of edge computing devices of an information uploading block chain and action classification requests aiming at the image characteristics;
s2, synchronizing the equipment for classifying the same image characteristic request in the action classification request;
s3, acquiring an image feature training loss function model to obtain an action classification result of a single image feature close to a real value;
and S4, modeling the structured output of the action classification result by using a boundary structured output learning framework to obtain a fusion action classification result and sending the fusion action classification result to the request equipment.
In the embodiment, by receiving the image features of a plurality of edge computing devices of the information uploading blockchain and the action classification requests aiming at the image features, synchronizing the devices aiming at the same image feature request classification in the action classification requests, acquiring an image feature training loss function model, obtaining the action classification result of a single image feature close to a real value, modeling the structured output of the action classification result by using a boundary structured output learning frame, and obtaining a fusion action classification result to be sent to a requesting device, so that only the image features and the download identification result are uploaded between the edge computing devices and a server, the original image and other information are not transmitted, privacy is ensured, the data quantity of the features is smaller than that of the original image, transmission cost is reduced, training data on different edge computing devices are collected, the precision of the model is improved, and the model is prevented from being abused by unverified devices by using a blockchain protection model.
In step S1, the edge computing device sends its own device serial number to the server, the serial number verifies on the edge computing device information chain, and the device passing verification can send an image feature and a request for identification, which need to identify the motion, to the server.
It should be noted that the edge computing device may upload information to the blockchain through the server, where the uploaded information includes, but is not limited to, a device serial number, a contribution data amount, and the like. When the image features and the identification request which need to be identified are sent to the server, the server firstly initiates information verification to the blockchain, clearly requests the identity and the data volume contribution of the equipment, and responds to the request of the equipment or refuses the request of the equipment. Thus, not only the authenticity and privacy of the uplink information can be ensured through the blockchain, but also the server can be prevented from being abused.
It should be further noted that, the image data provided to the server by the edge computing device is image feature data, and the image feature data may be feature extracted data corresponding to the image data one by one, which has an advantage of small data volume compared with the original image data, so that transmission cost can be effectively reduced in transmission. The feature extraction can be performed by prior art means, for example by means of a gradient direction histogram algorithm.
It can be appreciated that the data source manner provided by the edge computing device can use any camera to take images, and can also directly input existing pictures, video screens and the like.
In step S2, information synchronization is performed on the plurality of request devices that initiate the identification request to the same object in step S1 by using the federal learning manner, for example, synchronization is performed by using a device timestamp, which indicates that the identification request is initiated for the same identification object. Therefore, the technical effects of guaranteeing the privacy of the training data and collecting more training data can be achieved.
It should be noted that federal learning is a prior art, and the main idea is to construct a machine learning model based on data sets distributed over a plurality of devices, while preventing data leakage.
In step S3, an image feature training loss function model is obtained by using a loss function model technique, and an action classification result of a single image feature close to a real value is obtained. The loss function model may be specifically characterized as:
wherein the function isRepresenting classification output->And (3) real value->Loss between; assuming that an action is represented by a frame or series of keyframes containing action capture information, a query set Q and a category set C are assumed; for each query sample q i ＝x i ，y i ∈Q，x i E X is the image feature relevant to the query sample, X i Is D, i.e. x i ∈R D ；/>Is an output list containing the class |c| and its confidence.
It should be noted that the data result output by the loss function model processing is a single image featureAction classification results, the image characteristics are related to the query samples, the collection of the query samples reflects the image characteristics for model training, and the larger the data volume is, the more favorable the classification output is reducedAnd (3) real value->Loss between them, realize classified outputAnd (3) real value->The method is infinitely close to achieve the technical effect that the action classification result of the single image feature is more accurate.
It should be noted that, the real value is a preset value or an ideal value of the output data of the loss function model, and the purpose of training the loss function model is to make the action classification result of the image feature more approximate to the real value.
In step S4, the structured output of the action classification result is modeled by using the boundary structured output learning framework, so as to obtain a fused action classification result, and the fused action classification result is sent to the requesting device. Wherein the boundary structured output learning framework can be characterized as:
where w is a model parameter vector; f (x) i ，y i The method comprises the steps of carrying out a first treatment on the surface of the w) is defined as a linear function of w; f (x) i ，y i ；w)＝w T Ψ(x i ，y i )，Ψ(x i ，y i ) Is a joint feature map, which maps the feature x i Confidence prediction y i Mapping to a real value; psi (x) i ,y i ) Defined as->Deformation into->
Is query sample q i Action confidence list of (2).
It should be noted that, based on the action classification result of the single image feature, the boundary structured output learning framework may implement fusion of the classification result to process continuous image feature classification, for example, processing the video image feature.
Further, referring to fig. 3, since training samples are typically assigned a single label during the learning process of the conventional method. In this way, the different categories are trained separately. However, some training samples in different classes may be very similar. Taking the image frame illustrated in fig. 3 as an example, two frames in the figure, one from ThrowCatch and the other from Walking, look very similar in both the original video and motion data, and simply dividing the training samples by category may lead to a cluttered classifier. To avoid this clutter, the boundary structured output learning framework may be enhanced by applying local constraints. Wherein, to preserve locality, a hypergraph may be constructed to describe the correlation of samples. The specific implementation can be as follows:
redefining the boundary structured output learning framework as:
where α is used to balance the structured correlation w T Ψ(x i ，y i ) And local constraint item->Is a trade-off parameter of (2); minimizing the local constraint term is considered a manifold regularization-based hypergraph, allowing similar training samples to obtain similar confidence.
It should be noted that, based on regularization theory, a definition of the laplace matrix L may be introduced, and the redefined boundary structured output learning framework is characterized as:
wherein c= ((z) i -y i ) /(2α). Solutions can be divided into calculations and optimizations. The laplace matrix L may be calculated in two steps.
First, subsection optimization: in the subdivision optimization phase, a hypergraph tile may be defined as being defined as a vertex that is associated by a hyperedge. Thus, the definition of hypergraph slices in this learning process is:for a hypergraph tile, the calculation is required: />This means that two vertices in the vertex set comprised by one superside e are randomly selected and summed: />By extending the formula: />And combining the terms to obtain a piece optimization for each superside: />
Wherein, matrix E is defined as:i is an n identity matrix.
Second, the overall alignment: in hypergraphs, the weight of a hyperedge is calculated by summing the similarity scores of all pairs of vertices contained in the hyperedge. The similarity score for any pair of vertices is defined as the distance of the image feature:where, feat (u) represents the image feature vector of vertex u, dist (x, y) is typically set to a distance of L2. In the super-edge weighting matrix, the multi-view super-graph Laplacian matrix can be obtained by performing a cross-over onThe optimized summation of all the image slices of the superb defined in (a) is calculated: />
To solve the optimizationLagrangian multiplier λ can be introduced to eliminate the equality constraint and get +.>Where I is N i ×N i An identity matrix. Lambda can be from->The solution in the dual problem of (2) is: />Wherein η=u T c and L i ＝U∑U T Is L taking U as characteristic vector i Is characterized by decomposition of->Is the corresponding characteristic value lambda, denoted +.>The analysis of (a) is known, and for example, a binarization search technique is used, so that an optimum can be obtained. For specific procedures, see document B.Geng, Y.Yang, C.Xu, and X. -S.Hua, "Content-aware ranking for visual search," in IEEE International Conference on Computer Vision and Pattern Recognition.IEEE Press,June 2010,pp.3400–3407.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present invention.
In one embodiment, referring to fig. 4, a structured action classification device based on federal learning and blockchain is provided, which corresponds to the structured action classification method based on federal learning and blockchain in the above embodiment one-to-one. As shown in fig. 4, the apparatus comprises a receiving module 1, a synchronizing module 2, a training module 3 and a modeling module 4.
The receiving module 1 is used for receiving image characteristics of a plurality of edge computing devices establishing a block uplink and action classification requests aiming at the image characteristics;
the synchronization module 2 is used for synchronizing the equipment which requests classification aiming at the same image characteristic in the action classification request;
the training module 3 is used for acquiring an image characteristic training loss function model and obtaining an action classification result of a single image characteristic which is close to the theoretical output;
and the modeling module 4 is used for modeling the structured output of the action classification result by using the boundary structured output learning framework so as to obtain a fused action classification result and sending the fused action classification result to the request equipment.
It should be noted that the terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or modules is not necessarily limited to those steps or modules that are expressly listed or inherent to such process, method, article, or apparatus, but may include other steps or modules that may not be expressly listed or inherent to such process, method, article, or apparatus, and that the partitioning of such modules by which such process, article, or apparatus is presented is merely a logical partitioning and may be implemented in a practical application in other manners.
It should also be noted that, regarding the specific definition and description of the above apparatus, reference may be made to the definition and description of the structured action classification method based on federal learning and blockchain hereinabove, and the detailed description thereof will be omitted herein. Each of the modules in the above-described apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, referring to fig. 4 and 5, a computer device is provided, which may be a server running a receiving module 1, a synchronizing module 2, a training module 3, and a modeling module 4. Wherein the internal structural diagram of the computer device may be as shown in fig. 5. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used to store data involved in the structured action classification method based on federal learning and blockchain. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program, when executed by a processor, implements a structured action classification method based on federal learning and blockchain.
In one embodiment, a computer device is provided that includes a memory, a processor, and a computer program stored on the memory and executable on the processor, when executing the computer program, implementing the steps of the structured action classification method based on federal learning and blockchain in the above embodiments, such as steps 1 through 4 shown in fig. 2 and other extensions of the method and extensions of related steps. Alternatively, the processor, when executing the computer program, implements the functions of the modules/units of the federal learning and blockchain-based structured action classification apparatus of the above embodiments, such as the functions of modules 1 through 4 shown in fig. 4. In order to avoid repetition, a description thereof is omitted.
The processor may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, the processor being a control center of the computer device, and the various interfaces and lines connecting the various parts of the overall computer device.
The memory may be used to store computer programs and/or modules, and the processor implements various functions of the computer device by running or executing the computer programs and/or modules stored in the memory, and invoking data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data (such as audio data, video data, etc.) created according to the use of the cellular phone, etc.
The memory may be integrated with the processor or may be separate from the processor.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon which, when executed by a processor, implements the steps of the federal learning and blockchain based structured action classification method of the above embodiments, such as steps 1 through 4 and other extensions of the method and extensions of related steps shown in fig. 2. Alternatively, the computer program when executed by the processor implements the functions of the modules/units of the federal learning and blockchain based structured action classification apparatus of the above embodiments, such as the functions of modules 1 through 4 shown in fig. 4. In order to avoid repetition, a description thereof is omitted.
Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.
Claims (9)
1. A structured action classification method based on federal learning and blockchain, comprising:
receiving image features of a plurality of edge computing devices of an information uploading blockchain and action classification requests aiming at the image features, wherein the edge computing devices upload information to the blockchain through a server and send the image features needing action recognition to the server;
synchronizing the equipment for classifying the same image characteristic request in the action classification request;
acquiring the image feature training loss function model to obtain an action classification result of a single image feature close to a real value;
modeling the structured output of the action classification result by using a boundary structured output learning framework to obtain a fused action classification result and sending the fused action classification result to the request equipment;
wherein the boundary structured output learning framework is characterized by:
wherein W is a model parameter vector;
F(x i ，y i the method comprises the steps of carrying out a first treatment on the surface of the w) is defined as a linear function of w; f (x) i ，y i ；w)＝w T Ψ(x i ，y i )，Ψ(x i ，y i ) Is a joint feature map, which maps the feature x i Confidence prediction y i Mapping to a real value; psi (x) i ，y i ) Is defined asIs deformed into
Is a list of action confidence for the query sample qi.
2. The method as recited in claim 1, further comprising:
local constraints are applied to the boundary structured output learning framework to enhance the boundary structured output learning framework.
3. The method of claim 1, wherein the information that uploads the blockchain includes a device serial number and a contribution data amount.
4. The method of claim 2, wherein the loss function model is characterized by:wherein the function->Representing classification output->And (3) real value->Loss between; assuming that an action is represented by a frame or series of keyframes containing action capture information, a query set Q and a category set C are assumed; for each query sample q i ＝x i ，y i ∈Q，x i E X is the image feature relevant to the query sample, X i Is D, i.e. x i ∈R D ；Is an output list containing the class |c| and its confidence.
5. The method of claim 4, comprising redefining:
where α is used to balance the structured correlation w T Ψ(x i ，y i ) And local constraint item->Is a trade-off parameter of (2); minimizing the local constraint term is considered a manifold regularization-based hypergraph, allowing similar training samples to obtain similar confidence.
6. A structured action classification device based on federal learning and blockchain, comprising:
the device comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving image characteristics of a plurality of edge computing devices of an information uploading blockchain and action classification requests aiming at the image characteristics, the edge computing devices upload information to the blockchain through a server, and the edge computing devices send the image characteristics needing action recognition to the server;
the synchronization module is used for synchronizing the equipment which is required to be classified aiming at the same image characteristic in the action classification request;
the training module is used for acquiring the image feature training loss function model and obtaining an action classification result of the single image feature close to the real value;
the modeling module is used for modeling the structured output of the action classification result by using a boundary structured output learning framework so as to obtain a fusion action classification result and sending the fusion action classification result to the request equipment;
wherein the boundary structured output learning framework is characterized by:
wherein W is a model parameter vector; f (x) i ，y i The method comprises the steps of carrying out a first treatment on the surface of the w) is defined as the linearity of wA function; f (x) i ，y i ；w)＝w T Ψ(x i ，y i )，Ψ(x i ，y i ) Is a joint feature map, which maps the feature x i Confidence prediction y i Mapping to a real value; psi (x) i ，y i ) Defined as->Deformation into-> Is query sample q i Action confidence list of (2).
7. A structured action classification system based on federal learning and blockchain, comprising: a plurality of edge computing devices, servers, and blockchains;
the plurality of edge computing devices communicate through the blockchain; the edge computing equipment uploads information to the blockchain through the server, and sends image features needing to be subjected to action recognition to the server;
the receiving module receives image characteristics of a plurality of edge computing devices of an information uploading block chain and action classification requests aiming at the image characteristics;
the synchronization module synchronizes the equipment which is required to be classified aiming at the same image characteristic in the action classification request;
the training module acquires the image feature training loss function model to obtain an action classification result of a single image feature close to a real value;
the modeling module uses a boundary structured output learning framework to model structured output of the action classification result so as to obtain a fusion action classification result and send the fusion action classification result to the request equipment;
wherein the boundary structured output learning framework is characterized by:
wherein W is a model parameter vector;
F(x i ，y i the method comprises the steps of carrying out a first treatment on the surface of the w) is defined as a linear function of w; f (x) i ，y i ；w)＝w T Ψ(x i ，y i )，Ψ(x i ，y i ) Is a joint feature map, which maps the feature x i Confidence prediction y i Mapping to a real value; psi (x) i ，y i ) Is defined asIs deformed into Is query sample q i Action confidence list of (2).
8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the computer program is executed in the processor to implement the method of any one of claims 1-5.
9. A computer readable storage medium storing a computer program, characterized in that the computer program is executed in a processor for implementing the method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011621661.8A CN112733901B (en) | 2020-12-30 | 2020-12-30 | Structured action classification method and device based on federal learning and blockchain |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011621661.8A CN112733901B (en) | 2020-12-30 | 2020-12-30 | Structured action classification method and device based on federal learning and blockchain |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112733901A CN112733901A (en) | 2021-04-30 |
| CN112733901B true CN112733901B (en) | 2024-01-12 |
Family
ID=75608246
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011621661.8A Active CN112733901B (en) | 2020-12-30 | 2020-12-30 | Structured action classification method and device based on federal learning and blockchain |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112733901B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115701071A (en) * | 2021-07-16 | 2023-02-07 | 中移物联网有限公司 | Model training method and device, electronic equipment and storage medium |
| CN115424079B (en) * | 2022-09-30 | 2023-11-24 | 深圳市大数据研究院 | Image classification method based on federal edge learning and related equipment |
| CN116502732B (en) * | 2023-06-29 | 2023-10-20 | 杭州金智塔科技有限公司 | Federal learning method and system based on trusted execution environment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020199693A1 (en) * | 2019-03-29 | 2020-10-08 | 中国科学院深圳先进技术研究院 | Large-pose face recognition method and apparatus, and device |
| CN111931242A (en) * | 2020-09-30 | 2020-11-13 | 国网浙江省电力有限公司电力科学研究院 | Data sharing method, computer equipment applying same and readable storage medium |
| CN112100295A (en) * | 2020-10-12 | 2020-12-18 | 平安科技（深圳）有限公司 | User data classification method, device, equipment and medium based on federal learning |
-
2020
- 2020-12-30 CN CN202011621661.8A patent/CN112733901B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020199693A1 (en) * | 2019-03-29 | 2020-10-08 | 中国科学院深圳先进技术研究院 | Large-pose face recognition method and apparatus, and device |
| CN111931242A (en) * | 2020-09-30 | 2020-11-13 | 国网浙江省电力有限公司电力科学研究院 | Data sharing method, computer equipment applying same and readable storage medium |
| CN112100295A (en) * | 2020-10-12 | 2020-12-18 | 平安科技（深圳）有限公司 | User data classification method, device, equipment and medium based on federal learning |
Non-Patent Citations (1)
| Title |
|---|
| 面向移动终端智能的自治学习系统;徐梦炜;刘渊强;黄康;刘譞哲;黄罡;;软件学报(第10期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112733901A (en) | 2021-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112733901B (en) | Structured action classification method and device based on federal learning and blockchain | |
| WO2021043168A1 (en) | Person re-identification network training method and person re-identification method and apparatus | |
| CN110574077B (en) | Image analysis device and method using virtual three-dimensional deep neural network | |
| US10796452B2 (en) | Optimizations for structure mapping and up-sampling | |
| JP7208408B2 (en) | Detection model training method, apparatus, computer device and computer program | |
| US20220130130A1 (en) | Video tracking with deep siamese networks and bayesian optimization | |
| CN110135406B (en) | Image recognition method and device, computer equipment and storage medium | |
| CN110580482B (en) | Image classification model training, image classification and personalized recommendation method and device | |
| WO2019228317A1 (en) | Face recognition method and device, and computer readable medium | |
| WO2021227726A1 (en) | Methods and apparatuses for training face detection and image detection neural networks, and device | |
| CN109886077B (en) | Image recognition method and device, computer equipment and storage medium | |
| CN110569721A (en) | Recognition model training method, image recognition method, device, equipment and medium | |
| US20220375213A1 (en) | Processing Apparatus and Method and Storage Medium | |
| WO2022179581A1 (en) | Image processing method and related device | |
| WO2021013095A1 (en) | Image classification method and apparatus, and method and apparatus for training image classification model | |
| CN110287836B (en) | Image classification method and device, computer equipment and storage medium | |
| CN111091075B (en) | Face recognition method and device, electronic equipment and storage medium | |
| US11816877B2 (en) | Method and apparatus for object detection in image, vehicle, and robot | |
| WO2021114612A1 (en) | Target re-identification method and apparatus, computer device, and storage medium | |
| CN111680675B (en) | Face living body detection method, system, device, computer equipment and storage medium | |
| WO2022052782A1 (en) | Image processing method and related device | |
| Lu et al. | Rethinking prior-guided face super-resolution: A new paradigm with facial component prior | |
| Chang et al. | Salgaze: Personalizing gaze estimation using visual saliency | |
| WO2022179603A1 (en) | Augmented reality method and related device thereof | |
| Chen et al. | StateNet: Deep state learning for robust feature matching of remote sensing images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |