CN109685041A

CN109685041A - Image analysis method and device, electronic equipment and storage medium

Info

Publication number: CN109685041A
Application number: CN201910063728.1A
Authority: CN
Inventors: 冯伟; 刘文韬; 李通; 钱晨
Original assignee: Beijing Sensetime Technology Development Co Ltd
Current assignee: Beijing Sensetime Technology Development Co Ltd
Priority date: 2019-01-23
Filing date: 2019-01-23
Publication date: 2019-04-26
Anticipated expiration: 2039-01-23
Also published as: CN109685041B

Abstract

This disclosure relates to a kind of image analysis method and device, electronic equipment and storage medium, the described method includes: carrying out feature extraction to image to be analyzed, the characteristic information of target object in the image to be analyzed is obtained, the characteristic information includes behavioural characteristic and posture feature；According to the characteristic information, relation recognition is carried out to the target object, obtains the relation recognition of the target object as a result, the relation recognition result includes at least one of the location information of behavioural information and object relevant to behavior.The embodiment of the present disclosure carries out relation recognition to target object by the behavioural characteristic and posture feature of target object in image, to improve the accuracy of relation recognition.

Description

Image analysis method and device, electronic equipment and storage medium

Technical field

This disclosure relates to field of computer technology more particularly to a kind of image analysis method and device, electronic equipment and deposit Storage media.

Background technique

In the fields such as image understanding and human-computer interaction, people estimates to have obtained with object relationship identification and human body attitude to be answered extensively With.But traditional people and object relationship recognition methods depend only on the barment tag of people, recognition result is easy by appearance The influence of changing features, and traditional estimation method of human posture also tends to independently predict each human body key point, has ignored Positional relationship between key point is easy to be blocked, the influence of the factors such as erroneous detection.

Summary of the invention

The present disclosure proposes a kind of technical solutions of image analysis.

According to the disclosure in a first aspect, providing a kind of image analysis method, which comprises to image to be analyzed Feature extraction is carried out, obtains the characteristic information of target object in the image to be analyzed, the characteristic information includes behavioural characteristic And posture feature；According to the characteristic information, relation recognition is carried out to the target object, obtains the relationship of the target object Recognition result, the relation recognition result include at least one in the location information of behavioural information and object relevant to behavior It is a.

In one possible implementation, the method also includes: according to the relation recognition result and the appearance State feature carries out Attitude estimation to the target object, obtains the Attitude estimation of the target object as a result, the Attitude estimation It as a result include the posture information of the target object.

In one possible implementation, the relation recognition result includes N grades of relation recognition as a result, the posture Estimated result includes N grades of Attitude estimation as a result, N is the integer greater than 1, wherein according to the characteristic information, to the target Object carries out relation recognition, obtains the relation recognition result of the target object, comprising: according to the characteristic information, to described Target object carries out relation recognition, obtains the relation recognition result of the first order；In the case where N is equal to 2, according to the pass of the first order It is the Attitude estimation of recognition result and the first order as a result, carrying out relation recognition to the target object, obtains the relationship of the second level Recognition result.

In one possible implementation, according to the characteristic information, relation recognition is carried out to the target object, is obtained To the relation recognition result of the target object, further includes: in the case where N is greater than 2, according to (n-1)th grade of relation recognition knot Fruit and (n-1)th grade of Attitude estimation obtain n-th grade of relation recognition knot as a result, to target object progress relation recognition Fruit, n are integer and 1 < n < N；According to N-1 grades of relation recognition result and N-1 grades of Attitude estimation as a result, to the mesh It marks object and carries out relation recognition, obtain N grades of relation recognition result.

In one possible implementation, according to the relation recognition result and the posture feature, to the mesh It marks object and carries out Attitude estimation, obtain the Attitude estimation result of the target object, comprising: according to the relation recognition knot of the first order Fruit and the posture feature carry out Attitude estimation to the target object, obtain the Attitude estimation result of the first order；In N etc. In the case where 2, according to the Attitude estimation of the relation recognition result of the second level and the first order as a result, to the target object into Row Attitude estimation obtains the Attitude estimation result of the second level.

In one possible implementation, according to the relation recognition result and the posture feature, to the mesh It marks object and carries out Attitude estimation, obtain the Attitude estimation result of the target object, further includes: in the case where N is greater than 2, root According to n-th grade of relation recognition result and (n-1)th grade of Attitude estimation as a result, carrying out Attitude estimation to the target object, obtain To n-th grade of Attitude estimation as a result, n is integer and 1 < n < N；According to N grades of relation recognition result and N-1 grades of posture Estimated result carries out Attitude estimation to the target object, obtains N grades of Attitude estimation result.

In one possible implementation, according to (n-1)th grade of relation recognition result and (n-1)th grade of Attitude estimation As a result, carrying out relation recognition to the target object, n-th grade of relation recognition result is obtained, comprising: to (n-1)th grade of relationship Recognition result and (n-1)th grade of Attitude estimation result carry out full connection processing, obtain n-th grade of connection features；To n-th grade Connection features carry out Activity recognition processing, obtain n-th grade of behavioural information.

In one possible implementation, according to (n-1)th grade of relation recognition result and (n-1)th grade of Attitude estimation As a result, carrying out relation recognition to the target object, n-th grade of relation recognition result is obtained, further includes: according to described n-th grade Behavioural information, relation recognition processing is carried out to n-th grade of connection features, obtains n-th grade of location information.

In one possible implementation, described n-th grade of relation recognition result further includes that n-th grade of intermediate interactions are special Sign, wherein according to (n-1)th grade of relation recognition result and (n-1)th grade of Attitude estimation as a result, being carried out to the target object Relation recognition obtains n-th grade of relation recognition result, further includes: to (n-1)th grade of relation recognition result and (n-1)th grade Attitude estimation result is connected entirely and process of convolution, obtains n-th grade of intermediate interactions feature.

In one possible implementation, according to n-th grade of relation recognition result and (n-1)th grade of Attitude estimation knot Fruit carries out Attitude estimation to the target object, obtains n-th grade of Attitude estimation result, comprising: it is based on attention mechanism, it is right Described n-th grade of relation recognition result carries out convolution and activation processing, obtains n-th grade of attention and tries hard to；To n-th grade of attention Figure and (n-1)th grade of Attitude estimation result carry out dot product, obtain n-th grade of input feature vector；Appearance is carried out to n-th grade of input feature vector State estimation, obtains n-th grade of posture information.

In one possible implementation, described n-th grade of Attitude estimation result further includes that n-th grade of middle attitude is special Sign, wherein according to n-th grade of relation recognition result and (n-1)th grade of Attitude estimation as a result, carrying out appearance to the target object State estimation, obtains n-th grade of Attitude estimation result, further includes: carries out process of convolution to n-th grade of input feature vector, obtains n-th grade Middle attitude feature.

In one possible implementation, the characteristic information further includes barment tag, wherein is believed according to the feature Breath carries out relation recognition to the target object, obtains the relation recognition result of the target object, comprising: according to described outer Table feature, behavioural characteristic and posture feature carry out relation recognition to the target object, and the relationship for obtaining the target object is known Other result.

In one possible implementation, according to the relation recognition result and the posture feature, to the mesh It marks object and carries out Attitude estimation, obtain the Attitude estimation result of the target object, comprising: according to the barment tag, described Relation recognition result and the posture feature carry out Attitude estimation to the target object, obtain the appearance of the target object State estimated result.

In one possible implementation, for the method by neural fusion, the neural network includes relationship Identify network and Attitude estimation network, wherein the relation recognition network is used to carry out relation recognition, institute to the characteristic information Attitude estimation network is stated for carrying out Attitude estimation to the relation recognition result and the posture feature.

In one possible implementation, for the method by neural fusion, the neural network includes N grades of passes System's identification network and N grades of Attitude estimation networks, wherein n-th grade of relation recognition network is used for the relation recognition knot to (n-1)th grade Fruit and (n-1)th grade of Attitude estimation result carry out relation recognition, and n-th grade of Attitude estimation network is used for the relationship to n-th grade Recognition result and (n-1)th grade of Attitude estimation result carry out Attitude estimation.

In one possible implementation, for the method by neural fusion, the neural network includes feature Network is extracted, the feature extraction network is used to carry out feature extraction to image to be analyzed.

In one possible implementation, the method also includes: according to preset training set, the training nerve net Network.

In one possible implementation, the behavioural information includes the confidence of the current behavior of the target object Degree.

According to the second aspect of the disclosure, provide a kind of image analysis apparatus, comprising: characteristic extracting module, for pair Image to be analyzed carries out feature extraction, obtains the characteristic information of target object in the image to be analyzed, the characteristic information packet Include behavioural characteristic and posture feature；Relation recognition module, for carrying out relationship to the target object according to the characteristic information Identification, obtain the relation recognition of the target object as a result, the relation recognition result include behavioural information and with behavior phase At least one of location information of object of pass.

In one possible implementation, described device further include: the first Attitude estimation module, for according to the pass It is recognition result and the posture feature, Attitude estimation is carried out to the target object, obtains the posture of the target object Estimated result, the Attitude estimation result include the posture information of the target object.

In one possible implementation, the relation recognition result includes N grades of relation recognition as a result, the posture Estimated result includes N grades of Attitude estimation as a result, N is the integer greater than 1, wherein the relation recognition module, comprising: first Relation recognition submodule, for carrying out relation recognition to the target object, obtaining the pass of the first order according to the characteristic information It is recognition result；Second relation recognition submodule, in the case where N is equal to 2, according to the relation recognition result of the first order and The Attitude estimation of the first order obtains the relation recognition result of the second level as a result, to target object progress relation recognition.

In one possible implementation, the relation recognition module, further includes: third relation recognition submodule is used In N be greater than 2 in the case where, according to (n-1)th grade of relation recognition result and (n-1)th grade of Attitude estimation as a result, to described Target object carries out relation recognition, obtains n-th grade of relation recognition as a result, n is integer and 1 < n < N；4th relation recognition submodule Block, for according to N-1 grades of relation recognition result and N-1 grades of Attitude estimation as a result, to the target object carry out Relation recognition obtains N grades of relation recognition result.

In one possible implementation, the first Attitude estimation module, comprising: the first Attitude estimation submodule, For according to the first order relation recognition result and the posture feature, to the target object carry out Attitude estimation, obtain The Attitude estimation result of the first order；Second Attitude estimation submodule is used in the case where N is equal to 2, according to the relationship of the second level Recognition result and the Attitude estimation of the first order obtain the posture of the second level as a result, to target object progress Attitude estimation Estimated result.

In one possible implementation, the first Attitude estimation module, further includes: third Attitude estimation submodule Block is used in the case where N is greater than 2, according to n-th grade of relation recognition result and (n-1)th grade of Attitude estimation as a result, to institute It states target object and carries out Attitude estimation, obtain n-th grade of Attitude estimation as a result, n is integer and 1 < n < N；4th Attitude estimation Module, for according to N grades of relation recognition result and N-1 grades of Attitude estimation as a result, to the target object carry out Attitude estimation obtains N grades of Attitude estimation result.

In one possible implementation, the third relation recognition submodule is used for: to (n-1)th grade of relation recognition As a result and (n-1)th grade of Attitude estimation result carries out full connection processing, obtains n-th grade of connection features；Connection to n-th grade Feature carries out Activity recognition processing, obtains n-th grade of behavioural information.

In one possible implementation, the third relation recognition submodule is also used to: according to n-th grade of the row For information, relation recognition processing is carried out to n-th grade of connection features, obtains n-th grade of location information.

In one possible implementation, described n-th grade of relation recognition result further includes that n-th grade of intermediate interactions are special Sign, wherein the third relation recognition submodule is also used to: to (n-1)th grade of relation recognition result and (n-1)th grade of posture Estimated result is connected entirely and process of convolution, obtains n-th grade of intermediate interactions feature.

In one possible implementation, the third Attitude estimation submodule is used for: attention mechanism is based on, to institute It states n-th grade of relation recognition result and carries out convolution and activation processing, obtain n-th grade of attention and try hard to；Attention to n-th grade is tried hard to And (n-1)th grade of Attitude estimation result carries out dot product, obtains n-th grade of input feature vector；Posture is carried out to n-th grade of input feature vector Estimation, obtains n-th grade of posture information.

In one possible implementation, described n-th grade of Attitude estimation result further includes that n-th grade of middle attitude is special Sign, wherein the third Attitude estimation submodule is also used to: process of convolution is carried out to n-th grade of input feature vector, obtains n-th grade Middle attitude feature.

In one possible implementation, the characteristic information further includes barment tag.

In one possible implementation, described device further include: the second Attitude estimation module, for according to described outer Table feature, the relation recognition result and the posture feature carry out Attitude estimation to the target object, obtain the mesh Mark the Attitude estimation result of object.

In one possible implementation, described device includes neural network, and the neural network includes relation recognition Network and Attitude estimation network, wherein the relation recognition network is used to carry out relation recognition, the appearance to the characteristic information State estimates that network is used to carry out Attitude estimation to the relation recognition result and the posture feature.

In one possible implementation, described device includes neural network, and the neural network includes that N grades of relationships are known Other network and N grades of Attitude estimation networks, wherein n-th grade of relation recognition network be used for (n-1)th grade of relation recognition result with And (n-1)th grade of Attitude estimation result carries out relation recognition, n-th grade of Attitude estimation network is used for the relation recognition to n-th grade As a result and (n-1)th grade Attitude estimation result carry out Attitude estimation.

In one possible implementation, described device includes neural network, and the neural network includes feature extraction Network, the feature extraction network are used to carry out feature extraction to image to be analyzed.

In one possible implementation, described device further include: training module is used for according to preset training set, The training neural network.

According to the third aspect of the disclosure, a kind of electronic equipment is provided, comprising: processor；It can for storage processor The memory executed instruction；Wherein, the processor is configured to: execute above-mentioned image analysis method.

According to the fourth aspect of the disclosure, a kind of computer readable storage medium is provided, is stored thereon with computer journey Sequence instruction, the computer program instructions realize above-mentioned image analysis method when being executed by processor.

In the embodiments of the present disclosure, by the behavioural characteristic of target object in image and posture feature to target object into Row relation recognition obtains the behavior score of target object and the location information of object relevant to behavior, to improve pass It is the accuracy of identification.

It should be understood that above general description and following detailed description is only exemplary and explanatory, rather than Limit the disclosure.According to below with reference to the accompanying drawings to detailed description of illustrative embodiments, the other feature and aspect of the disclosure will It becomes apparent.

Detailed description of the invention

The drawings herein are incorporated into the specification and forms part of this specification, and those figures show meet this public affairs The embodiment opened, and together with specification it is used to illustrate the technical solution of the disclosure.

Fig. 1 shows the flow chart of the image analysis method according to the embodiment of the present disclosure.

Fig. 2 shows the flow charts according to the image analysis method of the embodiment of the present disclosure.

Fig. 3 shows the structural schematic diagram of the neural network according to the embodiment of the present disclosure.

Fig. 4 shows the structural schematic diagram of the relation recognition network according to the embodiment of the present disclosure.

Fig. 5 shows the structural schematic diagram of the Attitude estimation network according to the embodiment of the present disclosure.

Fig. 6 shows the block diagram of the image analysis apparatus according to the embodiment of the present disclosure.

Fig. 7 shows the block diagram of a kind of electronic equipment according to the embodiment of the present disclosure.

Fig. 8 shows the block diagram of a kind of electronic equipment according to the embodiment of the present disclosure.

Specific embodiment

Various exemplary embodiments, feature and the aspect of the disclosure are described in detail below with reference to attached drawing.It is identical in attached drawing Appended drawing reference indicate element functionally identical or similar.Although the various aspects of embodiment are shown in the attached drawings, remove It non-specifically points out, it is not necessary to attached drawing drawn to scale.

Dedicated word " exemplary " means " being used as example, embodiment or illustrative " herein.Here as " exemplary " Illustrated any embodiment should not necessarily be construed as preferred or advantageous over other embodiments.

The terms "and/or", only a kind of incidence relation for describing affiliated partner, indicates that there may be three kinds of passes System, for example, A and/or B, can indicate: individualism A exists simultaneously A and B, these three situations of individualism B.In addition, herein Middle term "at least one" indicate a variety of in any one or more at least two any combination, it may for example comprise A, B, at least one of C can indicate to include any one or more elements selected from the set that A, B and C are constituted.

In addition, giving numerous details in specific embodiment below in order to which the disclosure is better described. It will be appreciated by those skilled in the art that without certain details, the disclosure equally be can be implemented.In some instances, for Method, means, element and circuit well known to those skilled in the art are not described in detail, in order to highlight the purport of the disclosure.

Fig. 1 shows the flow chart of the image analysis method according to the embodiment of the present disclosure, as shown in Figure 1, described image is analyzed Method includes:

In step s 11, feature extraction is carried out to image to be analyzed, obtains the spy of target object in the image to be analyzed Reference breath, the characteristic information includes behavioural characteristic and posture feature；

In step s 12, according to the characteristic information, relation recognition is carried out to the target object, obtains the target The relation recognition of object is as a result, the relation recognition result includes the location information of behavioural information and object relevant to behavior At least one of.

In accordance with an embodiment of the present disclosure, by the behavioural characteristic and posture feature of target object in image to target object Relation recognition is carried out, the behavior score of target object and the location information of object relevant to behavior are obtained, to improve The accuracy of relation recognition.

In one possible implementation, described image analysis method can be set by electronics such as terminal device or servers Standby to execute, terminal device can be user equipment (User Equipment, UE), mobile device, user terminal, terminal, honeycomb Phone, wireless phone, personal digital assistant (Personal Digital Assistant, PDA), handheld device, calculate equipment, Mobile unit, wearable device etc., the method can call the computer-readable instruction stored in memory by processor Mode realize.Alternatively, the method can be executed by server.

In one possible implementation, the image to be analyzed can be pre-stored figure in terminal or server Picture or the image downloaded from network, are also possible to the image acquired by shooting unit.For example, carrying out human-computer interaction or interaction During amusement, image can be acquired by shooting unit (such as camera).There can be one in the image to be analyzed Or multiple target objects (people or object) and the one or more objects interacted with the target object.The disclosure is treated The type and acquisition modes for analyzing image are with no restriction.In one possible implementation, target object can be people or dynamic The objects such as object, the characteristic information may include behavioural characteristic (preliminary behavioural characteristic) and posture feature (preliminary posture spy Sign).Wherein, behavioural characteristic and posture feature can be for example including the positions of multiple human body key points of target object.

In one possible implementation, feature extraction can be carried out to image to be analyzed in step s 11, obtains institute State the characteristic information of target object in image to be analyzed.The disclosure to the mode of feature extraction with no restriction.

In one possible implementation, neural network reality is being passed through according to the image analysis method of the embodiment of the present disclosure Now, image can be analyzed by feature extraction network handles carry out feature extraction.It should be appreciated that feature extraction network can be for example Convolutional neural networks, deep neural network etc., the disclosure to the concrete type of feature extraction network with no restriction.

For example, feature extraction network can be for example including multiple convolutional layers and full articulamentum etc..It can be analysed to Image input feature vector extracts in network, extracts the characteristic pattern of image to be analyzed and determines one or more human regions, wherein It may include target object (human body) in each human region；Then different processing is carried out to the characteristic pattern of human region, point Do not obtain target object behavioural characteristic (for example can handle to obtain by convolution and full connection) and posture feature (can for example by Process of convolution obtains).

It in one possible implementation, can be in step s 12 to the mesh according to the characteristic information of target object It marks object and carries out relation recognition, obtain the relation recognition result of the target object.The relation recognition result includes behavior letter At least one of the location information of breath and object relevant to behavior.

In one possible implementation, the behavioural information may include the confidence level of the current behavior of target object. For example, the current behavior of people in image to be analyzed is the confidence level (such as 60%) of " playing soccer ".It should be appreciated that behavioural information It also may include that target object currently carries out the probability of certain behavior or obtains grading information, the disclosure is to the specific of behavioural information Content is with no restriction.

In one possible implementation, the location information of object relevant to behavior may include current with target object Picture position (region) of the associated object of behavior in image to be analyzed.For example, the people in image to be analyzed is current Behavior be " playing soccer " in the case where, object relevant to the behavior be " football ", can be according to " football " in image to be analyzed Position determine the location information.

In one possible implementation, neural network reality is being passed through according to the image analysis method of the embodiment of the present disclosure Now, relation recognition can be carried out to the target object by relation recognition network.Relation recognition network can be for example convolution mind Through network C NN, the disclosure to the concrete type of relation recognition network with no restriction.

For example, characteristic information (behavioural characteristic and posture feature) can be inputted in relation recognition network and carries out behavior Identification, obtains the behavioural information (such as confidence level) of every kind of behavior act of target object；According to behavioural information and human body area The position in domain, the location information of available object relevant to behavior.In image to be analyzed, every kind of behavior of target object Have relevant object corresponding, therefore, relation recognition result may include multiple groups behavioural information and with behavior correlative The location information of body.

Fig. 2 shows the flow charts according to the image analysis method of the embodiment of the present disclosure.As shown in Fig. 2, the method may be used also Include:

In step s 13, according to the relation recognition result and the posture feature, appearance is carried out to the target object State estimation, obtains the Attitude estimation of the target object as a result, the Attitude estimation result includes the posture of the target object Information.That is, can carry out Attitude estimation by relation recognition result and posture feature simultaneously as inputting, obtain target pair The Attitude estimation result of elephant.

In one possible implementation, neural network reality is being passed through according to the image analysis method of the embodiment of the present disclosure Now, Attitude estimation can be carried out to the target object by Attitude estimation network.Attitude estimation network can be for example convolution mind Through network C NN etc., the disclosure to the concrete type of Attitude estimation network with no restriction.

For example, Attitude estimation network can be to the relation recognition result of input (behavioural information and/or related to behavior Object location information) and posture feature handled and (can be handled such as convolution and activation), obtain Attitude estimation knot Fruit, wherein Attitude estimation result can include at least the posture information (such as position of human body key point) of target object.

In this way, using relation recognition result as the input of Attitude estimation, so that the position between human body key point Relationship is set definitely, to improve the accuracy of Attitude estimation.

It in one possible implementation, can be in such a way that turbine learns (turbine study framework), by relation recognition It is combined with Attitude estimation, multiple relation recognition and Attitude estimation is carried out to image to be analyzed.It can be by the result of Attitude estimation Input as relation recognition；Again using the result of relation recognition as the input of Attitude estimation, successive ignition is carried out.The process class It is similar to the turbocharger of engine, output is fed back into input, exhaust gas is recycled to improve engine efficiency, feedback procedure can be with Gradually improve this two tasks as a result, improving the accuracy of relation recognition and Attitude estimation simultaneously.

In one possible implementation, the relation recognition result includes N grades of relation recognition as a result, the posture Estimated result includes N grades of Attitude estimation result.N is the integer greater than 1, such as N=3.The disclosure does not make the specific value of N Limitation.

In one possible implementation, step S12 can include: according to the characteristic information, to the target object Relation recognition is carried out, the relation recognition result of the first order is obtained；

In the case where N is equal to 2, according to the Attitude estimation of the relation recognition result of the first order and the first order as a result, to institute It states target object and carries out relation recognition, obtain the relation recognition result of the second level.

For example, N grades of relation recognitions can be carried out to image to be analyzed.It, can be according to spy in first order relation recognition Reference ceases (behavioural characteristic and posture feature) and carries out relation recognition to target object, obtains the relation recognition result (row of the first order For information and/or the location information of object relevant to behavior)；It, can be according to the relationship of the first order in the relation recognition of the second level Recognition result and the Attitude estimation of the first order obtain the relation recognition knot of the second level as a result, to target object progress relation recognition Fruit.N be equal to 2 in the case where, can be using the relation recognition result of the second level as final output result.

In one possible implementation, step S12 may also include that

N be greater than 2 in the case where, according to (n-1)th grade of relation recognition result and (n-1)th grade of Attitude estimation as a result, Relation recognition is carried out to the target object, obtains n-th grade of relation recognition as a result, n is integer and 1 < n < N；

According to N-1 grades of relation recognition result and N-1 grades of Attitude estimation as a result, being carried out to the target object Relation recognition obtains N grades of relation recognition result.

It for example,, can be according to (n-1)th grade in n-th (1 < n < N) grade relation recognition in the case where N is greater than 2 Relation recognition result and (n-1)th grade of Attitude estimation obtain n-th grade of relationship as a result, to target object progress relation recognition Recognition result, wherein n-th grade of any level that can be in the second level to N-1 grades；It, can basis in N grades of relation recognitions N-1 grades of relation recognition result and N-1 grades of Attitude estimation obtain N as a result, to target object progress relation recognition The relation recognition result of grade.

In one possible implementation, neural network reality is being passed through according to the image analysis method of the embodiment of the present disclosure Now, neural network may include the N grade relation recognition network for carrying out relation recognition to the target object.The disclosure is to N The concrete type of grade relation recognition network is with no restriction.

By the above-mentioned means, can be using Attitude estimation result as the input of relation recognition, so that during relation recognition The barment tag for not only relying on people also relies on finer human body attitude feature, to avoid the influence of the appearance difference of people, mentions The precision of high relation recognition.

In one possible implementation, step S13 can include: according to the relation recognition result of the first order and described Posture feature carries out Attitude estimation to the target object, obtains the Attitude estimation result of the first order；

In the case where N is equal to 2, according to the Attitude estimation of the relation recognition result of the second level and the first order as a result, right The target object carries out Attitude estimation, obtains the Attitude estimation result of the second level.

For example, N grades of Attitude estimations can be carried out to image to be analyzed.It, can basis in first order Attitude estimation The relation recognition result (location information of behavioural information and/or object relevant to behavior) of the first order and the posture are special Sign carries out Attitude estimation to the target object, obtains the Attitude estimation result (posture information) of the first order；In second level posture It, can be according to the relation recognition result of the second level and the Attitude estimation of the first order as a result, being carried out to the target object in estimation Attitude estimation obtains the Attitude estimation result of the second level.In the case where N is equal to 2, the Attitude estimation result of the second level can be made For final output result.

In one possible implementation, step S13 may also include that

In the case where N is greater than 2, according to n-th grade of relation recognition result and (n-1)th grade of Attitude estimation as a result, right The target object carries out Attitude estimation, obtains n-th grade of Attitude estimation result；

According to N grades of relation recognition result and N-1 grades of Attitude estimation as a result, carrying out appearance to the target object State estimation, obtains N grades of Attitude estimation result.

It for example,, can be according to n-th grade of pass in n-th (1 < n < N) grade Attitude estimation in the case where N is greater than 2 It is recognition result and (n-1)th grade of Attitude estimation as a result, carrying out Attitude estimation to the target object, obtains n-th grade of appearance State estimated result, wherein n-th grade of any level that can be in the second level to N-1 grades；In N grades of Attitude estimations, Ke Yigen According to N grades of relation recognition result and N-1 grades of Attitude estimation as a result, carrying out Attitude estimation to the target object, obtain To N grades of Attitude estimation result.

In one possible implementation, neural network reality is being passed through according to the image analysis method of the embodiment of the present disclosure Now, neural network may include the N grade Attitude estimation network for carrying out Attitude estimation to the target object.The disclosure is to N The concrete type of grade Attitude estimation network is with no restriction.

The behavior of people is also relied on while by the above-mentioned means, the barment tag of people can be relied on during Attitude estimation And the location information of object relevant to behavior improves posture so that the positional relationship between human body key point is more clear The precision of estimation.

Fig. 3 shows the structural schematic diagram of the neural network according to the embodiment of the present disclosure.As described in Figure 3, neural network can wrap Include N grades of relation recognition networks: the relation recognition network 31 of the first order, the second level relation recognition network 32 ..., N grades of pass System's identification network 33.Neural network may also include N grades of Attitude estimation networks: the Attitude estimation network 61 of the first order, the second level Attitude estimation network 62 ..., N grades of Attitude estimation network 63.

In one possible implementation, in the first order of relation recognition, can will by feature extraction network 34 to The preliminary behavioural characteristic of the target object obtained in analysis imageWith preliminary posture featureInput the relationship of the first order It is handled in identification network 31, obtains the relation recognition result of the first orderThe relation recognition result of the first orderIncluding target pair The behavioural information of elephantAnd the location information of object relevant to behavior

It in one possible implementation, can be by the relation recognition result of the first order in the second level of relation recognition With the Attitude estimation result of the first orderIt inputs in the relation recognition network 32 of the second level and handles, obtain the relation recognition of the second level As a resultIt, can be by (n-1)th grade of relation recognition result and (n-1)th grade of gesture recognition result at n-th grade of relation recognition The relation recognition network of n-th grade of input carries out relation recognition, obtains n-th grade of relation recognition result.

It in one possible implementation, can be by N-1 grades of relation recognition result at N grades of relation recognitionAnd N-1 grades of gesture recognition resultIt inputs in N grades of relation recognition network 33 and handles, obtain N grades Relation recognition result(includingWith), and by relation recognition resultWithOutput as image analysis.

In this way, using the high correlation of people and object relationship identification and human body attitude estimation, with turbine The mode of habit iteratively optimizes the recognition result of people and object relationship identification, can step up accuracy of identification.

Fig. 4 shows the structural schematic diagram of the relation recognition network according to the embodiment of the present disclosure.As shown in figure 4, one kind can Can implementation in, relation recognition network may include the operations such as complete connection, convolution, can to target object progress Activity recognition with And relation recognition.

In one possible implementation, according to (n-1)th grade of relation recognition result and (n-1)th grade of Attitude estimation As a result, the step of carrying out relation recognition to the target object, obtaining n-th grade of relation recognition result can include:

Full connection processing is carried out to the Attitude estimation result of (n-1)th grade of relation recognition result and (n-1)th grade, obtains the N grades of connection features；

Activity recognition processing is carried out to n-th grade of connection features, obtains n-th grade of behavioural information.

As shown in figure 4, in relation recognition network, it can be first to (n-1)th grade of relation recognition result of inputWith (n-1)th grade of gesture recognition resultIt carries out full connection processing, obtains n grades of connection features (as shown in 41 in Fig. 4)；So Activity recognition is carried out to n-th grade of connection features afterwards, obtains n-th grade of behavioural informationWherein, following formula can be passed through (1) connection features h is obtained:

In formula (1), [] indicates the connection of characteristic pattern, and I () indicates full attended operation.

In this way, it using the relation recognition result and gesture recognition result progress Activity recognition after connection, mentions The precision of high Activity recognition.

In one possible implementation, according to (n-1)th grade of relation recognition result and (n-1)th grade of Attitude estimation As a result, the step of carrying out relation recognition to the target object, obtaining n-th grade of relation recognition result, may also include that

According to n-th grade of the behavioural information, relation recognition processing is carried out to n-th grade of connection features, obtains n-th grade Location information.

As shown in figure 4, in relation recognition network, can according to n-th grade of behavioural information, to n-th grade of connection features into Row relation recognition obtains n-th grade of location information

In this way, it using the relation recognition result and gesture recognition result progress relation recognition after connection, mentions The precision of high relation recognition.

In one possible implementation, described n-th grade of relation recognition result further includes that n-th grade of intermediate interactions are special Sign,

Wherein, according to (n-1)th grade of relation recognition result and (n-1)th grade of Attitude estimation as a result, to the target pair As the step of carrying out relation recognition, obtaining n-th grade of relation recognition result, further includes:

To (n-1)th grade of relation recognition result and (n-1)th grade of Attitude estimation result is connected entirely and process of convolution, Obtain n-th grade of intermediate interactions feature.

It, can be to (n-1)th grade of relation recognition result as shown in figure 4, in relation recognition networkWith (n-1)th grade Gesture recognition resultIt is connected entirely and process of convolution, obtains n-th grade of intermediate interactions featureAccording to n-th grade Recognition result(including n-th grade of behavioural informationWith n-th grade of location information) and n-th grade of intermediate interactions FeatureObtain n-th grade of relation recognition result

In this way, the recognition result (intermediate interactions feature) of previous stage can be added in relation recognition result, keep away Exempt from Character losing, further improves the precision of relation recognition.

In one possible implementation, as shown in figure 3, Attitude estimation can be carried out by N grades of Attitude estimation networks.? The first order of Attitude estimation, can be by the relation recognition result of the first order of target objectWith preliminary posture featureInput It is handled in the Attitude estimation network 61 of the first order, obtains the Attitude estimation result of the first orderIn the second level of Attitude estimation, It can be by the relation recognition result of the second levelWith the Attitude estimation result of the first orderInput the Attitude estimation network 62 of the second level Middle processing obtains the posture feature of the second level

It in one possible implementation, can be by N grades of relation recognition result at N grades of Attitude estimationWith And N-1 grades of recognition resultIt inputs in N grades of Attitude estimation network 63 and handles, obtain N grades of Attitude estimation knot FruitAnd by Attitude estimation resultOutput as image analysis.

In this way, iteratively excellent using the high correlation of people and object relationship identification and human body attitude estimation The recognition result for changing Attitude estimation, can step up accuracy of identification.

Fig. 5 shows the structural schematic diagram of the Attitude estimation network according to the embodiment of the present disclosure.As shown in figure 5, one kind can In the implementation of energy, Attitude estimation network may include the operations such as full connection, convolution, activation and dot product, can be to target object Carry out Attitude estimation.

In one possible implementation, according to n-th grade of relation recognition result and (n-1)th grade of Attitude estimation knot Fruit, the step of carrying out Attitude estimation to the target object, obtain n-th grade of Attitude estimation result, it may include:

Based on attention mechanism, convolution is carried out to described n-th grade of relation recognition result and activation is handled, obtains n-th grade Attention try hard to；

Attention to n-th grade is tried hard to and (n-1)th grade of Attitude estimation result carries out dot product, obtains n-th grade of input feature vector；

Attitude estimation is carried out to n-th grade of input feature vector, obtains n-th grade of posture information.

For example, in Attitude estimation network, it can be primarily based on attention mechanism, to n-th grade of relation recognition knot FruitConvolution and activation processing are carried out, n-th grade of attention is obtained and tries hard to Att_action(as shown in 51 in Fig. 5).Such as formula (2) institute Show:

In formula (2), sigmoid indicates that activation primitive, R () indicate deformation operation.

In one possible implementation, n-th grade of attention can be tried hard to, (n-1)th grade of Attitude estimation result Dot product is carried out, n-th grade of input feature vector p is obtained, as shown in formula (3):

In one possible implementation, in Attitude estimation network, posture can be carried out to n-th grade of input feature vector p Estimation, obtains n-th grade of posture information

In one possible implementation, described n-th grade of Attitude estimation result further includes that n-th grade of middle attitude is special Sign,

Wherein, according to n-th grade of relation recognition result and (n-1)th grade of Attitude estimation as a result, to the target object The step of carrying out Attitude estimation, obtaining n-th grade of Attitude estimation result, may also include that

Process of convolution is carried out to n-th grade of input feature vector, obtains n-th grade of middle attitude feature.

That is, the input feature vector p to n-th grade carries out process of convolution, n-th grade of middle attitude feature is obtained According to n-th grade of posture informationAnd n-th grade of middle attitude featureObtain n-th grade of Attitude estimation result

As described above, every level-one of turbine study framework may include level-one relation recognition network and level-one Attitude estimation net Network, can be by the input of previous stage exported as rear stage.N-th grade of input tⁿIt can be as shown in formula (4):

Successive ignition in this way, can gradually improve relations identification mission and Attitude estimation task as a result, mentioning simultaneously The accuracy of high relation recognition and Attitude estimation.

In one possible implementation, neural network (including feature extraction network, N grades of relation recognition nets are being used Network and N grades of Attitude estimation networks) to image to be analyzed carry out image analysis before, neural network can be trained.

In one possible implementation, the method also includes: according to preset training set, the training nerve net Network.In the training process, network parameter values can be adjusted according to the direction for minimizing loss function, when loss function is reduced to When to a certain degree or converging in certain threshold value, stops adjustment, obtain N grades of neural network adjusted.The disclosure was to training Loss function used in journey is with no restriction.

In one possible implementation, following loss function can be used to be trained to neural network:

In formula (5), L indicates that the total losses of neural network, N indicate the grade of relation recognition network and Attitude estimation network Number,Indicate the loss of i-stage Attitude estimation network,Indicate the weight of i-stage Attitude estimation network,Table Show the loss of i-stage relation recognition network,Indicate the weight of i-stage relation recognition network, 1≤i≤N, L_detIndicate mesh The loss of mark detection network.Wherein, target detection network is for determining region of the target object in image to be analyzed, the disclosure With no restriction to the network structure of target detection network.

Neural network is trained by using loss function described in formula (5), obtains neural network adjusted, Performance when neural network carries out image analysis can be improved.

In one possible implementation, characteristic information may also include barment tag, wherein barment tag can be wrapped for example Include the features such as dress, the appearance of people.When carrying out feature extraction to image to be analyzed in step s 11, target object can be extracted Barment tag.For example, can extract the characteristic pattern of image to be analyzed and determine one or more human regions, to human region It is analyzed, the barment tag of target object can be obtained.

In one possible implementation, step S12 can include: according to the barment tag, behavioural characteristic and posture Feature carries out relation recognition to the target object, obtains the relation recognition result of the target object.

For example, barment tag can be added as input when carrying out relation recognition to the target object.It is treating It, can be using barment tag as the input of every level-one relation recognition when analyzing image N grades of relation recognitions of progress.For example, at n-th grade It, can be according to barment tag, (n-1)th grade of relation recognition result and (n-1)th grade of Attitude estimation as a result, right in relation recognition Target object carries out relation recognition, obtains n-th grade of relation recognition result, wherein n-th grade can be the second level to N-1 grades In any level.

In this way, the appearance information that object (personage) can be introduced in relation recognition, avoids the appearance difference of people Influence to relation recognition result further increases the precision of relation recognition.

In one possible implementation, step S13 can include: according to the barment tag, the relation recognition knot Fruit and the posture feature carry out Attitude estimation to the target object, obtain the Attitude estimation result of the target object.

For example, barment tag can be added as input when carrying out Attitude estimation to the target object.It is treating It, can be using barment tag as the input of every level-one Attitude estimation when analyzing image N grades of Attitude estimations of progress.For example, at n-th grade It, can be according to barment tag, n-th grade of relation recognition result and (n-1)th grade of Attitude estimation as a result, to mesh in Attitude estimation It marks object and carries out Attitude estimation, obtain n-th grade of Attitude estimation result, wherein n-th grade can be the second level in N-1 grades Any level.

In this way, the appearance information that object (personage) can be introduced in Attitude estimation, avoids the appearance difference of people Influence to Attitude estimation result further increases the precision of Attitude estimation.

Similarly, include barment tag in characteristic information, and relation recognition and appearance are carried out using turbine study framework simultaneously In the case that state is estimated, turbine learns n-th grade of input t of frameworkⁿIt can be as shown in formula (6):

In this way, barment tag can be introduced in an iterative process, avoid the appearance difference of people to the shadow of result It rings, further increases the accuracy of relation recognition and Attitude estimation.

According to the image analysis method of the embodiment of the present disclosure, the position of (object) and object of owner in image can be obtained It sets, the positional relationship for the object that everyone is interacted with it and the human body key point position (posture feature) of the people enhance People and object relationship identify and human body attitude estimates the robustness to the appearance difference of people, available more accurate identification knot Fruit.It can be applied to human-computer interaction class product in accordance with an embodiment of the present disclosure, the products such as interaction entertainment product and use field accordingly Jing Zhong improves the accuracy of interbehavior identification.

It is appreciated that above-mentioned each embodiment of the method that the disclosure refers to, without prejudice to principle logic, To engage one another while the embodiment to be formed after combining, as space is limited, the disclosure is repeated no more.

Fig. 6 shows the block diagram of the image analysis apparatus according to the embodiment of the present disclosure, as shown in fig. 6, described device includes:

Characteristic extracting module 71 obtains target in the image to be analyzed for carrying out feature extraction to image to be analyzed The characteristic information of object, the characteristic information include behavioural characteristic and posture feature；

Relation recognition module 72, for carrying out relation recognition to the target object, obtaining institute according to the characteristic information The relation recognition of target object is stated as a result, the relation recognition result includes the position of behavioural information and object relevant to behavior At least one of confidence breath.

In one possible implementation, the relation recognition result includes N grades of relation recognition as a result, the posture Estimated result includes N grades of Attitude estimation as a result, N is the integer greater than 1, wherein the relation recognition module 72, comprising: the One relation recognition submodule, for carrying out relation recognition to the target object, obtaining the first order according to the characteristic information Relation recognition result；Second relation recognition submodule is used in the case where N is equal to 2, according to the relation recognition result of the first order And the Attitude estimation of the first order obtains the relation recognition result of the second level as a result, to target object progress relation recognition.

In one possible implementation, the relation recognition module 72, further includes: third relation recognition submodule, For N be greater than 2 in the case where, according to (n-1)th grade of relation recognition result and (n-1)th grade of Attitude estimation as a result, to institute It states target object and carries out relation recognition, obtain n-th grade of relation recognition as a result, n is integer and 1 < n < N；4th relation recognition Module, for according to N-1 grades of relation recognition result and N-1 grades of Attitude estimation as a result, to the target object into Row relation recognition obtains N grades of relation recognition result.

In some embodiments, the embodiment of the present disclosure provides the function that has of device or comprising module can be used for holding The method of row embodiment of the method description above, specific implementation are referred to the description of embodiment of the method above, for sake of simplicity, this In repeat no more.

The embodiment of the present disclosure also proposes a kind of computer readable storage medium, is stored thereon with computer program instructions, institute It states when computer program instructions are executed by processor and realizes the above method.Computer readable storage medium can be non-volatile meter Calculation machine readable storage medium storing program for executing.

The embodiment of the present disclosure also proposes a kind of electronic equipment, comprising: processor；For storage processor executable instruction Memory；Wherein, the processor is configured to the above method.

The equipment that electronic equipment may be provided as terminal, server or other forms.

Fig. 7 is the block diagram of a kind of electronic equipment 800 shown according to an exemplary embodiment.For example, electronic equipment 800 can To be mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, Medical Devices are good for Body equipment, the terminals such as personal digital assistant.

Referring to Fig. 7, electronic equipment 800 may include following one or more components: processing component 802, memory 804, Power supply module 806, multimedia component 808, audio component 810, the interface 812 of input/output (I/O), sensor module 814, And communication component 816.

The integrated operation of the usual controlling electronic devices 800 of processing component 802, such as with display, call, data are logical Letter, camera operation and record operate associated operation.Processing component 802 may include one or more processors 820 to hold Row instruction, to perform all or part of the steps of the methods described above.In addition, processing component 802 may include one or more moulds Block, convenient for the interaction between processing component 802 and other assemblies.For example, processing component 802 may include multi-media module, with Facilitate the interaction between multimedia component 808 and processing component 802.

Memory 804 is configured as storing various types of data to support the operation in electronic equipment 800.These data Example include any application or method for being operated on electronic equipment 800 instruction, contact data, telephone directory Data, message, picture, video etc..Memory 804 can by any kind of volatibility or non-volatile memory device or it Combination realize, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM) is erasable Except programmable read only memory (EPROM), programmable read only memory (PROM), read-only memory (ROM), magnetic memory, fastly Flash memory, disk or CD.

Power supply module 806 provides electric power for the various assemblies of electronic equipment 800.Power supply module 806 may include power supply pipe Reason system, one or more power supplys and other with for electronic equipment 800 generate, manage, and distribute the associated component of electric power.

Multimedia component 808 includes the screen of one output interface of offer between the electronic equipment 800 and user. In some embodiments, screen may include liquid crystal display (LCD) and touch panel (TP).If screen includes touch surface Plate, screen may be implemented as touch screen, to receive input signal from the user.Touch panel includes one or more touches Sensor is to sense the gesture on touch, slide, and touch panel.The touch sensor can not only sense touch or sliding The boundary of movement, but also detect duration and pressure associated with the touch or slide operation.In some embodiments, Multimedia component 808 includes a front camera and/or rear camera.When electronic equipment 800 is in operation mode, as clapped When taking the photograph mode or video mode, front camera and/or rear camera can receive external multi-medium data.It is each preposition Camera and rear camera can be a fixed optical lens system or have focusing and optical zoom capabilities.

Audio component 810 is configured as output and/or input audio signal.For example, audio component 810 includes a Mike Wind (MIC), when electronic equipment 800 is in operation mode, when such as call mode, recording mode, and voice recognition mode, microphone It is configured as receiving external audio signal.The received audio signal can be further stored in memory 804 or via logical Believe that component 816 is sent.In some embodiments, audio component 810 further includes a loudspeaker, is used for output audio signal.

I/O interface 812 provides interface between processing component 802 and peripheral interface module, and above-mentioned peripheral interface module can To be keyboard, click wheel, button etc..These buttons may include, but are not limited to: home button, volume button, start button and lock Determine button.

Sensor module 814 includes one or more sensors, for providing the state of various aspects for electronic equipment 800 Assessment.For example, sensor module 814 can detecte the state that opens/closes of electronic equipment 800, the relative positioning of component, example As the component be electronic equipment 800 display and keypad, sensor module 814 can also detect electronic equipment 800 or The position change of 800 1 components of electronic equipment, the existence or non-existence that user contacts with electronic equipment 800, electronic equipment 800 The temperature change of orientation or acceleration/deceleration and electronic equipment 800.Sensor module 814 may include proximity sensor, be configured For detecting the presence of nearby objects without any physical contact.Sensor module 814 can also include optical sensor, Such as CMOS or ccd image sensor, for being used in imaging applications.In some embodiments, which may be used also To include acceleration transducer, gyro sensor, Magnetic Sensor, pressure sensor or temperature sensor.

Communication component 816 is configured to facilitate the communication of wired or wireless way between electronic equipment 800 and other equipment. Electronic equipment 800 can access the wireless network based on communication standard, such as WiFi, 2G or 3G or their combination.Show at one In example property embodiment, communication component 816 receives broadcast singal or broadcast from external broadcasting management system via broadcast channel Relevant information.In one exemplary embodiment, the communication component 816 further includes near-field communication (NFC) module, short to promote Cheng Tongxin.For example, radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra wide band can be based in NFC module (UWB) technology, bluetooth (BT) technology and other technologies are realized.

In the exemplary embodiment, electronic equipment 800 can be by one or more application specific integrated circuit (ASIC), number Word signal processor (DSP), digital signal processing appts (DSPD), programmable logic device (PLD), field programmable gate array (FPGA), controller, microcontroller, microprocessor or other electronic components are realized, for executing the above method.

In the exemplary embodiment, a kind of non-volatile computer readable storage medium storing program for executing is additionally provided, for example including calculating The memory 804 of machine program instruction, above-mentioned computer program instructions can be executed by the processor 820 of electronic equipment 800 to complete The above method.

Fig. 8 is the block diagram of a kind of electronic equipment 1900 shown according to an exemplary embodiment.For example, electronic equipment 1900 It may be provided as a server.Referring to Fig. 8, electronic equipment 1900 includes processing component 1922, further comprise one or Multiple processors and memory resource represented by a memory 1932, can be by the execution of processing component 1922 for storing Instruction, such as application program.The application program stored in memory 1932 may include it is one or more each Module corresponding to one group of instruction.In addition, processing component 1922 is configured as executing instruction, to execute the above method.

Electronic equipment 1900 can also include that a power supply module 1926 is configured as executing the power supply of electronic equipment 1900 Management, a wired or wireless network interface 1950 is configured as electronic equipment 1900 being connected to network and an input is defeated (I/O) interface 1958 out.Electronic equipment 1900 can be operated based on the operating system for being stored in memory 1932, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or similar.

In the exemplary embodiment, a kind of non-volatile computer readable storage medium storing program for executing is additionally provided, for example including calculating The memory 1932 of machine program instruction, above-mentioned computer program instructions can by the processing component 1922 of electronic equipment 1900 execute with Complete the above method.

The disclosure can be system, method and/or computer program product.Computer program product may include computer Readable storage medium storing program for executing, containing for making processor realize the computer-readable program instructions of various aspects of the disclosure.

Computer readable storage medium, which can be, can keep and store the tangible of the instruction used by instruction execution equipment Equipment.Computer readable storage medium for example can be-- but it is not limited to-- storage device electric, magnetic storage apparatus, optical storage Equipment, electric magnetic storage apparatus, semiconductor memory apparatus or above-mentioned any appropriate combination.Computer readable storage medium More specific example (non exhaustive list) includes: portable computer diskette, hard disk, random access memory (RAM), read-only deposits It is reservoir (ROM), erasable programmable read only memory (EPROM or flash memory), static random access memory (SRAM), portable Compact disk read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical coding equipment, for example thereon It is stored with punch card or groove internal projection structure and the above-mentioned any appropriate combination of instruction.Calculating used herein above Machine readable storage medium storing program for executing is not interpreted that instantaneous signal itself, the electromagnetic wave of such as radio wave or other Free propagations lead to It crosses the electromagnetic wave (for example, the light pulse for passing through fiber optic cables) of waveguide or the propagation of other transmission mediums or is transmitted by electric wire Electric signal.

Computer-readable program instructions as described herein can be downloaded to from computer readable storage medium it is each calculate/ Processing equipment, or outer computer or outer is downloaded to by network, such as internet, local area network, wide area network and/or wireless network Portion stores equipment.Network may include copper transmission cable, optical fiber transmission, wireless transmission, router, firewall, interchanger, gateway Computer and/or Edge Server.Adapter or network interface in each calculating/processing equipment are received from network to be counted Calculation machine readable program instructions, and the computer-readable program instructions are forwarded, for the meter being stored in each calculating/processing equipment In calculation machine readable storage medium storing program for executing.

Computer program instructions for executing disclosure operation can be assembly instruction, instruction set architecture (ISA) instructs, Machine instruction, machine-dependent instructions, microcode, firmware instructions, condition setup data or with one or more programming languages The source code or object code that any combination is write, the programming language include the programming language-of object-oriented such as Smalltalk, C++ etc., and conventional procedural programming languages-such as " C " language or similar programming language.Computer Readable program instructions can be executed fully on the user computer, partly execute on the user computer, be only as one Vertical software package executes, part executes on the remote computer or completely in remote computer on the user computer for part Or it is executed on server.In situations involving remote computers, remote computer can pass through network-packet of any kind It includes local area network (LAN) or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as benefit It is connected with ISP by internet).In some embodiments, by utilizing computer-readable program instructions Status information carry out personalized customization electronic circuit, such as programmable logic circuit, field programmable gate array (FPGA) or can Programmed logic array (PLA) (PLA), the electronic circuit can execute computer-readable program instructions, to realize each side of the disclosure Face.

Referring herein to according to the flow chart of the method, apparatus (system) of the embodiment of the present disclosure and computer program product and/ Or block diagram describes various aspects of the disclosure.It should be appreciated that flowchart and or block diagram each box and flow chart and/ Or in block diagram each box combination, can be realized by computer-readable program instructions.

These computer-readable program instructions can be supplied to general purpose computer, special purpose computer or other programmable datas The processor of processing unit, so that a kind of machine is produced, so that these instructions are passing through computer or other programmable datas When the processor of processing unit executes, function specified in one or more boxes in implementation flow chart and/or block diagram is produced The device of energy/movement.These computer-readable program instructions can also be stored in a computer-readable storage medium, these refer to It enables so that computer, programmable data processing unit and/or other equipment work in a specific way, thus, it is stored with instruction Computer-readable medium then includes a manufacture comprising in one or more boxes in implementation flow chart and/or block diagram The instruction of the various aspects of defined function action.

Computer-readable program instructions can also be loaded into computer, other programmable data processing units or other In equipment, so that series of operation steps are executed in computer, other programmable data processing units or other equipment, to produce Raw computer implemented process, so that executed in computer, other programmable data processing units or other equipment Instruct function action specified in one or more boxes in implementation flow chart and/or block diagram.

The flow chart and block diagram in the drawings show system, method and the computer journeys according to multiple embodiments of the disclosure The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation One module of table, program segment or a part of instruction, the module, program segment or a part of instruction include one or more use The executable instruction of the logic function as defined in realizing.In some implementations as replacements, function marked in the box It can occur in a different order than that indicated in the drawings.For example, two continuous boxes can actually be held substantially in parallel Row, they can also be executed in the opposite order sometimes, and this depends on the function involved.It is also noted that block diagram and/or The combination of each box in flow chart and the box in block diagram and or flow chart, can the function as defined in executing or dynamic The dedicated hardware based system made is realized, or can be realized using a combination of dedicated hardware and computer instructions.

The presently disclosed embodiments is described above, above description is exemplary, and non-exclusive, and It is not limited to disclosed each embodiment.Without departing from the scope and spirit of illustrated each embodiment, for this skill Many modifications and changes are obvious for the those of ordinary skill in art field.The selection of term used herein, purport In the principle, practical application or improvement to the technology in market for best explaining each embodiment, or make the art Other those of ordinary skill can understand each embodiment disclosed herein.

Claims

1. a kind of image analysis method, which is characterized in that the described method includes:

Feature extraction is carried out to image to be analyzed, obtains the characteristic information of target object in the image to be analyzed, the feature Information includes behavioural characteristic and posture feature；

According to the characteristic information, relation recognition is carried out to the target object, obtains the relation recognition knot of the target object Fruit, the relation recognition result include at least one of the location information of behavioural information and object relevant to behavior.

2. the method according to claim 1, wherein the method also includes:

According to the relation recognition result and the posture feature, Attitude estimation is carried out to the target object, is obtained described The Attitude estimation of target object is as a result, the Attitude estimation result includes the posture information of the target object.

3. according to the method described in claim 2, it is characterized in that, the relation recognition result includes N grades of relation recognition knot Fruit, the Attitude estimation result includes N grades of Attitude estimation as a result, N is the integer greater than 1,

Wherein, according to the characteristic information, relation recognition is carried out to the target object, the relationship for obtaining the target object is known Other result, comprising:

According to the characteristic information, relation recognition is carried out to the target object, obtains the relation recognition result of the first order；

In the case where N is equal to 2, according to the Attitude estimation of the relation recognition result of the first order and the first order as a result, to the mesh It marks object and carries out relation recognition, obtain the relation recognition result of the second level.

4. according to the method described in claim 3, it is characterized in that, being carried out according to the characteristic information to the target object Relation recognition obtains the relation recognition result of the target object, further includes:

In the case where N is greater than 2, according to (n-1)th grade of relation recognition result and (n-1)th grade of Attitude estimation as a result, to institute It states target object and carries out relation recognition, obtain n-th grade of relation recognition as a result, n is integer and 1 < n < N；

According to N-1 grades of relation recognition result and N-1 grades of Attitude estimation as a result, carrying out relationship to the target object Identification, obtains N grades of relation recognition result.

5. the method according to claim 3 or 4, which is characterized in that according to the relation recognition result and the posture Feature carries out Attitude estimation to the target object, obtains the Attitude estimation result of the target object, comprising:

According to the relation recognition result of the first order and the posture feature, Attitude estimation is carried out to the target object, is obtained The Attitude estimation result of the first order；

In the case where N is equal to 2, according to the Attitude estimation of the relation recognition result of the second level and the first order as a result, to described Target object carries out Attitude estimation, obtains the Attitude estimation result of the second level.

6. according to the method described in claim 5, it is characterized in that, special according to the relation recognition result and the posture Sign carries out Attitude estimation to the target object, obtains the Attitude estimation result of the target object, further includes:

In the case where N is greater than 2, according to n-th grade of relation recognition result and (n-1)th grade of Attitude estimation as a result, to described Target object carries out Attitude estimation, obtains n-th grade of Attitude estimation as a result, n is integer and 1 < n < N；

Estimated according to N grades of relation recognition result and N-1 grades of Attitude estimation as a result, carrying out posture to the target object Meter, obtains N grades of Attitude estimation result.

7. method according to any one of claims 4 to 6, which is characterized in that according to (n-1)th grade of relation recognition knot Fruit and (n-1)th grade of Attitude estimation obtain n-th grade of relation recognition knot as a result, to target object progress relation recognition Fruit, comprising:

Full connection processing is carried out to (n-1)th grade of relation recognition result and (n-1)th grade of Attitude estimation result, obtains n-th grade Connection features；

8. a kind of image analysis apparatus characterized by comprising

Characteristic extracting module obtains target object in the image to be analyzed for carrying out feature extraction to image to be analyzed Characteristic information, the characteristic information include behavioural characteristic and posture feature；

Relation recognition module, for carrying out relation recognition to the target object, obtaining the target according to the characteristic information The relation recognition of object is as a result, the relation recognition result includes the location information of behavioural information and object relevant to behavior At least one of.

9. a kind of electronic equipment characterized by comprising

Processor；

Memory for storage processor executable instruction；

Wherein, the processor is configured to: perform claim require any one of 1 to 7 described in method.

10. a kind of computer readable storage medium, is stored thereon with computer program instructions, which is characterized in that the computer Method described in any one of claim 1 to 7 is realized when program instruction is executed by processor.