CN111476214A - Image area matching method and related device - Google Patents
Image area matching method and related device Download PDFInfo
- Publication number
- CN111476214A CN111476214A CN202010435120.XA CN202010435120A CN111476214A CN 111476214 A CN111476214 A CN 111476214A CN 202010435120 A CN202010435120 A CN 202010435120A CN 111476214 A CN111476214 A CN 111476214A
- Authority
- CN
- China
- Prior art keywords
- region
- image
- area
- target
- distance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Human Computer Interaction (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application discloses an image area matching method and a related device, which are used for determining a first image area and a second image area which correspond to a plurality of objects in an image respectively; a target distance between the first image region and the second image region is determined. And determining the sum of the optimal distances between the first image area and the second image area according to the target distance, wherein the sum of the optimal distances is used for showing that the matching degree between the first image area and the second image area is the highest. And determining that the matching relationship between the first image area and the second image area corresponding to the sum of the optimal distances is a target matching relationship between the first image area and the second image area. According to the method, the target distance between the first image area and the second image area which belong to the same object accords with the relevant condition, and the matching relation between the first image area and the second image area is accurately determined in a target distance mode.
Description
Technical Field
The present application relates to the field of image processing, and in particular, to an image region matching method and a related apparatus.
Background
When tracking a target object such as a human body through videos, images and the like, human body information, corresponding human face information and the like are generally required to realize accurate recognition and tracking of the human body. Therefore, when tracking, it is necessary to accurately match the human body and the human face in the image.
At present, face and body matching is mainly performed through a greedy algorithm. The greedy algorithm is that when solving a problem, the best choice can be made at present. Greedy algorithms do not consider global optimality but rather determine a locally optimal solution in some sense. However, for a scene with dense people, matching errors are easy to occur when the matching is performed by the algorithm, and the actual effect is not good.
Therefore, how to improve the matching accuracy of human bodies and human faces is a problem which needs to be solved urgently at present.
Disclosure of Invention
In order to solve the above technical problem, the present application provides an image region matching method and a related apparatus, which achieve accurate determination of a matching relationship between a first image region and a second image region.
The embodiment of the application discloses the following technical scheme:
in one aspect, an embodiment of the present application provides an image region matching method, where the method includes:
determining a first image area and a second image area which correspond to a plurality of objects in an image respectively; the first image region and the second image region respectively correspond to different target sites in a subject;
determining a target distance between the first image area and the second image area, wherein the target distance is used for embodying the matching degree between the first image area and the second image area; the target distance is determined according to any one or more combinations of area overlapping rate and distance;
determining the sum of the optimal distances between the first image area and the second image area according to the target distance, wherein the sum of the optimal distances is used for showing that the matching degree between the first image area and the second image area is the highest;
and determining that the matching relationship between the first image area and the second image area corresponding to the sum of the optimal distances is a target matching relationship between the first image area and the second image area.
In another aspect, an embodiment of the present application provides an image region matching method, where the method includes:
determining a first region, a second region and a third region respectively corresponding to a plurality of objects in an image, wherein the first region, the second region and the third region respectively correspond to different target parts in the objects;
determining a first matching relationship between the first region and the second region and a second matching relationship between the second region and the third region, which are target matching relationships, respectively; the target matching relationship is determined by the method, and when the target matching relationship is the first matching relationship, the first image area is the first area, and the second image area is the second area; when the target matching relationship is the second matching relationship, the first image area is the second area, and the second image area is the third area;
and determining a third matching relationship between the first region and the third region according to the first matching relationship and the second matching relationship.
In another aspect, an embodiment of the present application provides an image region matching apparatus, where the apparatus includes:
the device comprises a first determining unit, a second determining unit and a display unit, wherein the first determining unit is used for determining a first image area and a second image area which correspond to a plurality of objects in an image respectively; the first image region and the second image region respectively correspond to different target sites in a subject;
a second determining unit, configured to determine a target distance between the first image region and the second image region, where the target distance is used to represent a matching degree between the first image region and the second image region; the target distance is determined according to any one or more combinations of area overlapping rate and distance;
a third determining unit, configured to determine, according to the target distance, a sum of optimal distances between the first image region and the second image region, where the sum of optimal distances is used to show that a matching degree between the first image region and the second image region is the highest;
a fourth determining unit, configured to determine that a matching relationship between the first image region and the second image region corresponding to the sum of the optimal distances is a target matching relationship between the first image region and the second image region.
In another aspect, an embodiment of the present application provides an image region matching apparatus, where the apparatus includes:
a first determining unit, configured to determine a first region, a second region, and a third region corresponding to a plurality of objects in an image, respectively, where the first region, the second region, and the third region correspond to different target portions in the objects, respectively;
a second determination unit configured to determine a first matching relationship between the first region and the second region and a second matching relationship between the second region and the third region, which are target matching relationships, respectively; the target matching relationship is determined by the method, and when the target matching relationship is the first matching relationship, the first image area is the first area, and the second image area is the second area; when the target matching relationship is the second matching relationship, the first image area is the second area, and the second image area is the third area;
a third determining unit, configured to determine a third matching relationship between the first region and the third region according to the first matching relationship and the second matching relationship.
In another aspect, an embodiment of the present application provides an apparatus, where the apparatus includes a processor and a memory:
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is used for executing the image area matching method according to instructions in the program codes.
In another aspect, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, where the computer program is used to execute the above method.
According to the technical scheme, the first image area and the second image area corresponding to a plurality of objects in the image are determined; wherein the first image region and the second image region correspond to different target sites in the object, respectively. And determining a target distance between the first image area and the second image area, wherein the target distance is used for embodying the matching degree between the first image area and the second image area, and the target distance is determined according to any one or more combinations of area overlapping rate and distance. And determining the sum of the optimal distances between the first image area and the second image area according to the target distance, wherein the sum of the optimal distances is used for showing that the matching degree between the first image area and the second image area is the highest. And determining that the matching relationship between the first image area and the second image area corresponding to the sum of the optimal distances is a target matching relationship between the first image area and the second image area. In the method, the target distance between the first image area and the second image area which belong to the same object accords with the relevant condition, so that the matching relation between the first image area and the second image area is accurately determined in a target distance mode.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a flowchart of an image region matching method according to an embodiment of the present disclosure;
fig. 2a is a schematic image diagram provided in an embodiment of the present application;
FIG. 2b is a schematic view of another image provided by an embodiment of the present application;
fig. 2c is a schematic diagram of an image region according to an embodiment of the present disclosure;
fig. 3 is a schematic view of a scene for calculating a distance according to an embodiment of the present disclosure;
fig. 4 is a schematic view of a scene overlapped between a first image region and a second image region according to an embodiment of the present disclosure;
FIG. 5 is a schematic view of a scene overlapping a first image region and a second image region according to an embodiment of the present disclosure;
fig. 6a is a schematic diagram illustrating a first matching relationship between a human body frame and a human head frame according to an embodiment of the present application;
fig. 6b is a schematic diagram illustrating a second matching relationship between a human head frame and a human face frame according to an embodiment of the present application;
fig. 6c is a schematic diagram illustrating a third matching relationship between a human body frame and a human face frame according to an embodiment of the present application;
fig. 7 is a flowchart of an image region matching method according to an embodiment of the present disclosure;
fig. 8 is a schematic diagram of an image area matching apparatus according to an embodiment of the present disclosure;
fig. 9 is a schematic diagram of an image area matching apparatus according to an embodiment of the present disclosure.
Detailed Description
Embodiments of the present application are described below with reference to the accompanying drawings.
At present, face and body matching is mainly performed through a greedy algorithm. However, for a scene with dense people, matching errors are easy to occur when the matching is performed by the algorithm, and the actual effect is not good. Therefore, how to improve the matching accuracy of human bodies and human faces is a problem which needs to be solved urgently at present.
Therefore, the embodiment of the application provides an image area matching method, which realizes accurate determination of the matching relationship between the first image area and the second image area.
First, an execution subject of the image region matching method of the embodiment of the present application is described. The image area method may be performed by a data processing device, which may be a terminal device or a server.
The matching method provided in the embodiment of the present application is described below.
Referring to fig. 1, which shows a flowchart of an image region matching method provided in an embodiment of the present application, as shown in fig. 1, the method includes:
s101: a first image area and a second image area corresponding to a plurality of objects in the image are determined.
The image may include a plurality of objects, which are not limited in this embodiment, and the object may be any physical object, such as a human, an animal, an article, and the like.
The first image region and the second image region correspond to different target parts in the object, and the target part may refer to any one part in the object or a combination of two or more parts in the object, wherein when the target part is the part combination in the object, the parts in the part combination have a connection relationship.
In one possible implementation, the object may be a human body, and the first image region (or the second image region) may be an image region within a human body frame, an image region within a human head frame, or an image region within a human face frame.
For example, referring to fig. 2a, a schematic image diagram provided by an embodiment of the present application is shown, as shown in fig. 2a, which includes three persons. Referring to fig. 2b, which shows another schematic diagram of an image provided by the embodiment of the present application, as shown in fig. 2b, the body frame, the head frame and the face frame of the three persons are detected from the image. The human body frame can be a cuboid frame determined by the length and the width of a human body in an image, the human head frame can be a cuboid frame determined by the length and the width of a human head in the image, and the human face frame can be a cuboid frame determined by the length and the width of a human face in the image. Referring to fig. 2c, which shows a schematic diagram of an image region provided in the embodiment of the present application, as shown in fig. 2c, a body frame, a head frame, and a face frame of the three persons are respectively shown.
S102: a target distance between the first image region and the second image region is determined.
The target distance is used for showing the matching degree between the first image area and the second image area. The target distance may be determined based on any one or more combinations of the area overlapping ratio and the distance.
The target distance may be used to represent a matching degree between the first image region and the second image region. The target distance between the first image region and the second image region may also be recorded as a weight between the two image regions.
In a possible implementation manner, the method for determining the target distance between the first image area and the second image area in S102 includes:
in the following, the first image area may correspond to a complete object, and the second image area may correspond to a part of the object. For example, the first image region is an image region within a human body frame, and the second image region is an image region within a human head frame.
A distance between the target position of the first image area and the target position of the second image area is determined. The target position may be a position determined on the first image area and the second image area according to a preset manner. In some embodiments, the target position may be a position of a center point of a top edge of the human body frame corresponding to the first image region, or may be a position of a center point of a top edge of the human body frame corresponding to the second image region.
That is, for multiple persons in the image, the distance between each body frame and the top edge center point position of each head frame may be determined.
It should be noted that the present embodiment does not limit the type of the distance, and in a possible implementation, the distance may be any one of a vertical distance, a euclidean distance, a manhattan distance, and a manhattan distance including a weight. Wherein the vertical distance may be a distance in a vertical direction.
Referring to FIG. 3, a diagram illustrating a method provided by an embodiment of the present application is shownA scene schematic diagram for calculating distance is to make the center point of the top side of the human body frame in the image coordinate system be (x)bc,ybc) The center point of the top edge of the human head frame is (x)hc,yhc) Then, determining the vertical distance, Euclidean distance, Manhattan distance and Manhattan distance based on weight between the top edge center points of the human body frame and the human head frame:
vertical distance: d ═ ybc-yhc|；
Euclidean distance:
manhattan distance: d ═ xbc-xhc|+|ybc-yhc|；
Manhattan distances of different weights: d ═ t | xbc-xhc|+w*|ybc-yhcWhere t and w are weights.
The distance between the target position of the first image area and the target position of the second image area can represent the matching degree between the first image area and the second image area, and when the distance is smaller, the first image area and the second image area are more likely to be the same object, namely the matching degree between the first image area and the second image area is higher.
Thus, the target distance between the first image area and the second image area is determined based on the distance between the target position of the first image area and the target position of the second image area.
In a specific implementation, the distance may be directly used as the target distance between the first image region and the second image region.
Since the first image region, the second image region, and the like obtained by detecting the model are not so perfect, they may have various problems, and thus there may be a case of missing detection or false detection. Based on this, in a possible implementation manner, the method for determining the target distance according to the distance may include:
when it is determined that the distance satisfies the first condition, the distance is increased.
The present embodiment does not limit the degree of increase in the distance. In one possible implementation, the distance may be increased to infinity.
Wherein the first condition comprises:
the distance is greater than a distance threshold.
In addition, when the target portion corresponding to the first image area includes the target portion corresponding to the second image area, the target condition further includes:
the area overlapping ratio between the first image area and the second image area is smaller than a first overlapping ratio threshold, and the area overlapping ratio may be determined according to the first image area, the second image area, and an overlapping area between the first image area and the second image area.
The area intersection, which is the area of the overlapping region between the first image region and the second image region, may be denoted as a, the minimum area between the first image region and the second image region may be denoted as B, and the union of the areas between the first image region and the second image region may be denoted as C, so that the area overlapping ratio may be a/B or a/C. Wherein, A/B can be expressed as single overlapping rate, and A/C can be expressed as double overlapping rate.
For example, referring to fig. 4, which shows a schematic view of a scene where a first image region and a second image region overlap with each other according to an embodiment of the present disclosure, as shown in fig. 4, since an area of the first image region is the same as an area of a body frame and an area of the second image region is the same as an area of a head frame, an area overlapping rate between the first image region and the second image region may be determined by the body frame and the head frame, as shown in fig. 4, a gray region identifies an overlapping region area between the body frame and the head frame, and the area is a body frame area ∩, an area overlapping rate between the body frame and the head frame is, for example, a single overlapping rate (body frame area ∩ head frame area)/(min (body frame area, head frame area)), and a double overlapping rate (body frame area ∩ head frame area)/(body frame area ∪ head frame area).
The area overlapping ratio may be a single overlapping ratio or a double overlapping ratio between the first image region and the second image region, and preferably, the single overlapping ratio may be the area overlapping ratio.
Thus, the target distance between the first image area and the second image area is determined based on the increased distance.
That is, two constraints are added to the distance between the target position of the first image region and the target position of the second image region. The first constraint is: under normal conditions, the top center points of the body frame and the head frame of the same human body are coincident or relatively close, so that the vertical distance between the top center points of the body frame and the head frame should be less than or equal to a set distance threshold (such as 200), and if the vertical distance is greater than the distance threshold, it indicates that the detected body frame and the head frame are not matched by one person, so that the distance between the body frame and the body frame is set to be infinite. The second constraint is: normally, the human body frame of the same person contains the human head frame of the person, that is, the first overlapping rate of the human body frame and the human head frame should be greater than or equal to a set first overlapping rate threshold (e.g. 0.8), if the first overlapping rate is smaller than the first overlapping rate threshold, it indicates that the overlapping area of the human body frame and the human head frame is too small, that is, the detected human body frame and the detected human head frame are more likely not to belong to the same person, thereby setting the distance between the human body frame and the human head frame to be infinite.
The set target distance between the human body frame and the human head frame is as follows: when the distance between the top edge center points of the human body frame and the human body frame is smaller than or equal to a distance threshold value and/or the area overlapping rate between the human body frame and the human body frame is smaller than or equal to a first overlapping rate threshold value, determining the target distance as the determined distance; and when the distance between the center points of the top edges of the human body frame and the human head frame is greater than a distance threshold or the area overlapping rate between the human body frame and the human head frame is greater than a first overlapping rate threshold, determining that the target distance is infinite (namely infinity).
In the method, when the distance between the first image region and the second image region satisfies the first condition, that is, the probability that the first image region and the second image region are matched is low, so that the distance is increased, and the target distance between the first image region and the second image region is determined according to the increased distance, so that when the sum of the optimal distances between the first image region and the second image region is determined according to the target distance in subsequent S103, the probability that the matching relationship between the first image region and the second image region is determined to be the matching relationship corresponding to the sum of the optimal distances is reduced, and the probability of mismatching is reduced.
In a specific implementation, for each of the x first image regions (corresponding to the body frame) and the y second image regions (corresponding to the head frame), the determined target distances therebetween may be represented in a matrix form:
in a possible implementation manner, when the target portion corresponding to the first image region includes the target portion corresponding to the second image region, the method for determining the target distance between the first image region and the second image region in S102 includes:
determining an area overlap ratio between the first image region and the second image region.
Wherein the area overlapping ratio is determined according to a first image region, the second image region, and an overlapping region between the first image region and the second image region. The overlapping area between the first image region and the second image region may be denoted as a ', the minimum area between the first image region and the second image region may be denoted as B ', and the union of the areas between the first image region and the second image region may be denoted as C ', and the area overlapping ratio may be a '/B ', or a '/C '. In this case, A '/B' may be expressed as a single overlapping rate, and A '/C' may be expressed as a double overlapping rate.
The following description will be given taking the first image region as an image region in the face frame and the second image region as an image region in the face frame as an example.
For example, referring to fig. 5, which shows a schematic view of a scene where a first image region and a second image region overlap with each other according to an embodiment of the present disclosure, as shown in fig. 5, since the area of the first image region is the same as the area of a human head frame, and the area of the second image region is the same as the area of a human face frame, as such, an area overlapping rate between the first image region and the second image region may be determined by the human head frame and the human face frame, as shown in fig. 5, a gray region identifies an overlapping region between the human head frame and the human face frame, and the area is a human head frame area ∩, an area overlapping rate between the human head frame and the human face frame is, for example, a single overlapping rate (human head frame area ∩ human face area)/(min (human head frame area, human face area)), and a double overlapping rate (human head frame area ∩ human face area)/(human head frame area ∪ human face area).
The area overlapping ratio may be a single overlapping ratio or a double overlapping ratio between the first image region and the second image region, and preferably, the single overlapping ratio may be the area overlapping ratio.
Thus, the target distance is determined based on the area overlap ratio. In a specific implementation, the reciprocal of the area overlap ratio may be taken as the target distance.
In a possible implementation manner, the method for determining the target distance according to the area overlapping ratio may include:
when it is determined that the area overlap ratio satisfies a second condition, the area overlap ratio is decreased.
Determining the target distance according to the reduced area overlapping rate;
wherein the second condition comprises: the area overlap ratio is less than a second overlap ratio threshold.
That is, a constraint condition is added to the area overlapping ratio between the first image region and the second image region. Under normal conditions, the human head frame comprises the human face frame, a second overlap rate threshold value is set to be 0.4, if the area overlap rate between the human head frame and the human face frame is smaller than the second overlap rate threshold value, the fact that the human face frame and the human head frame are more likely not to belong to the same object, namely a human body, is shown, the area overlap rate between the human head frame and the human face frame can be set, and the reciprocal of the area overlap rate is infinite.
The set target distance between the human head frame and the human face frame is as follows: when the area overlap ratio between the head frame and the face frame is greater than or equal to the second overlap ratio threshold, it may be determined that the target distance between the head frame and the face frame is the reciprocal of the area overlap ratio, and when the area overlap ratio between the head frame and the face frame is less than the second overlap ratio threshold, it may be determined that the target distance between the head frame and the face frame is infinity (i.e., ∞).
In the method, when the area overlapping rate between the first image area and the second image area satisfies the second condition, that is, the probability that the first image area is matched with the second image area is low, so that the area overlapping rate is reduced, and the target distance between the first image area and the second image area is determined according to the reduced area overlapping rate, so that when the sum of the optimal distances between the first image area and the second image area is determined according to the target distance in subsequent S103, the probability that the matching relationship between the first image area and the second image area is determined to be the matching relationship corresponding to the sum of the optimal distances is reduced, and the probability of mismatching is reduced. In a specific implementation, for each of the x first image regions (corresponding to the human head box) and the y second image regions (corresponding to the human face box), the determined target distances therebetween may be represented in a matrix form:
s103: and determining the sum of the optimal distances between the first image area and the second image area according to the target distance.
The sum of the optimal distances may be used to indicate that the matching degree between the first image region and the second image region is the highest. When the target distance is inversely correlated with the matching degree, the sum of the optimum distances may be a minimum value of the sum of the target distances between the first image region and the second image region.
In a specific implementation, the method of S103 may be performed by a KM algorithm or a max-flow min-cut algorithm. The KM algorithm may be an algorithm for solving a matching problem with the maximum (or minimum) weight of the weighted bipartite graph. The max-flow min-cut algorithm may refer to the maximum amount of traffic that can reach a sink from a source in a network flow equal to the minimum sum of the capacities of the set of edges that can cause a disruption in the network flow if removed from the network. I.e. in any network the value of the maximum flow is equal to the capacity of the minimum cut. When the sum of the optimal distances between the first image area and the second image area is determined by these two algorithms, the target distance between the first image area and the second image area may be taken as the weight between the two image areas.
S104: and determining that the matching relationship between the first image area and the second image area corresponding to the sum of the optimal distances is a target matching relationship between the first image area and the second image area.
And aiming at the matching relation between the first image area and the second image area corresponding to the sum of the optimal distances, the target matching relation between the first image area and the second image area is obtained.
According to the technical scheme, the first image area and the second image area corresponding to a plurality of objects in the image are determined; wherein the first image region and the second image region correspond to different target sites in the object, respectively. And determining a target distance between the first image area and the second image area, wherein the target distance is used for embodying the matching degree between the first image area and the second image area, and the target distance is determined according to any one or more combinations of area overlapping rate and distance. And determining the sum of the optimal distances between the first image area and the second image area according to the target distance, wherein the sum of the optimal distances is used for showing that the matching degree between the first image area and the second image area is the highest. And determining that the matching relationship between the first image area and the second image area corresponding to the sum of the optimal distances is a target matching relationship between the first image area and the second image area. In the method, the target distance between the first image area and the second image area which belong to the same object accords with the relevant condition, so that the matching relation between the first image area and the second image area is accurately determined in a target distance mode.
An embodiment of the present application provides an image region matching method, referring to fig. 7, which shows a flowchart of an image region matching method provided in an embodiment of the present application, and as shown in fig. 7, the method includes:
s701: a first area, a second area and a third area corresponding to a plurality of objects in the image are determined.
The first region, the second region, and the third region respectively correspond to different target portions of the subject, and the description of the target portions is as described above, and is not repeated here.
The method comprises the steps that for a plurality of objects in an image, a first area, a second area and a third area corresponding to each object are included in the image. In a possible implementation manner, the object is a human body, the first region is an image region in a human body frame, the second region is an image region in a human head frame, and the third region is an image region in a human face frame.
Next, an image region matching method provided in the embodiment of the present application will be described by taking a scene in which an object is a human body as an example.
S702: and determining a first matching relationship between the first region and the second region and a second matching relationship between the second region and the third region which are respectively used as target matching relationships.
The target matching relationship may be determined by the method provided in S101-S104, and when the target matching relationship is the first matching relationship, the first image area is the first area, and the second image area is the second area; and when the target matching relationship is the second matching relationship, the first image area is the second area, and the second image area is the third area.
S703: and determining a third matching relationship between the first region and the third region according to the first matching relationship and the second matching relationship. When the target matching relationship is the first matching relationship, referring to fig. 6a, this figure shows a schematic diagram that the human body frame and the human head frame have the first matching relationship provided in the embodiment of the present application, and as shown in fig. 6a, the matching relationships between the human body frames and the human head frames are determined by the KM algorithm or the max flow min cut algorithm.
When the target matching relationship is the second matching relationship, referring to fig. 6b, this figure shows a schematic diagram of the second matching relationship between the human head frame and the human face frame provided in the embodiment of the present application, and as shown in fig. 6b, the matching relationship between the human head frame and the human face frame is determined by the KM algorithm or the max flow min cut algorithm.
For example, according to the matching relationship corresponding to fig. 6a and fig. 6b, the third matching relationship between the human body frame and the human face frame can be determined. Referring to fig. 6c, the figure shows a schematic diagram of a third matching relationship between a human body frame and a human face frame provided in the embodiment of the present application, and as shown in fig. 6c, the matching relationships between a plurality of human body frames and human face frames are determined.
It can be seen from the above technical solution that, when a first region and a third region between a plurality of objects in an image need to be matched, first, the first region, the second region, and the third region of the plurality of objects can be determined, and the first region, the second region, and the third region respectively correspond to different target portions in the objects. Then, a first matching relationship between the first region and the second region and a second matching relationship between the second region and the third region, which are target matching relationships, respectively, may be determined. Thus, a third matching relationship between the first region and the third region is determined based on the first matching relationship and the second matching relationship. In the method, the second region is introduced, and the second region can be used for accurately determining the first matching relationship between the second region and the first region and can also be used for determining the second matching relationship between the second region and the third region, so that the second region is used as the intermediate quantity for matching between the first region and the third region, and the accurate determination of the matching relationship between the first region and the third region is realized.
The matching method provided by the embodiment of the present application is introduced below with reference to specific scenarios.
It is assumed that the method is used for matching human bodies and faces of a plurality of persons in an image. Wherein the image may be a frame of image in a video. The method comprises the following steps: reading a frame of image from a video, and inputting the image into a detection model, wherein the detection model can be used for detecting a human body, a human head and a human face. Then, a target distance (weight) matrix between the human body frame and the human head frame of the multiple persons and a target distance (weight) matrix between the human head frame and the human face frame are constructed. Therefore, a first matching relation between the human body frame and the human head frame and a second matching relation between the human head frame and the human face frame are solved through a KM algorithm or a maximum flow minimum cut algorithm. And further obtaining a third matching relation between the human body frame and the human face frame.
Referring to fig. 8, which shows a schematic diagram of an image region matching apparatus provided in an embodiment of the present application, as shown in fig. 8, the apparatus includes:
a first determining unit 801, configured to determine a first image area and a second image area corresponding to a plurality of objects in an image, respectively; the first image region and the second image region respectively correspond to different target sites in a subject;
a second determining unit 802, configured to determine a target distance between the first image area and the second image area, where the target distance is used to represent a matching degree between the first image area and the second image area; the target distance is determined according to any one or more combinations of area overlapping rate and distance;
a third determining unit 803, configured to determine, according to the target distance, a sum of optimal distances between the first image region and the second image region, where the sum of optimal distances is used to show that a matching degree between the first image region and the second image region is the highest;
a fourth determining unit 804, configured to determine that a matching relationship between the first image region and the second image region corresponding to the sum of the optimal distances is a target matching relationship between the first image region and the second image region.
In a possible implementation manner, the second determining unit 802 is specifically configured to:
determining a distance between a target position of the first image area and a target position of the second image area; the target position is a position determined on the first image area and the second image area according to a preset mode;
and determining the target distance according to the distance.
In a possible implementation manner, the second determining unit 802 is specifically configured to:
increasing the distance when it is determined that the distance satisfies a first condition;
determining the target distance according to the increased distance;
wherein the first condition comprises:
the distance is greater than a distance threshold.
In a possible implementation manner, when the target portion corresponding to the first image area includes the target portion corresponding to the second image area, the target condition further includes:
an area overlapping ratio between the first image region and the second image region is smaller than a first overlapping ratio threshold, the area overlapping ratio being determined according to the first image region, the second image region, and an overlapping region between the first image region and the second image region.
In one possible implementation, the distance is any one of a vertical distance, a euclidean distance, a manhattan distance, and a manhattan distance including a weight.
In a possible implementation manner, the second determining unit 802 is specifically configured to:
when the target part corresponding to the first image area comprises the target part corresponding to the second image area, determining the area overlapping rate between the first image area and the second image area; the area overlapping ratio is determined according to the first image area, the second image area and an overlapping area between the first image area and the second image area;
and determining the target distance according to the area overlapping rate.
In a possible implementation manner, the second determining unit 802 is specifically configured to:
when the area overlapping rate is determined to meet a second condition, reducing the area overlapping rate;
determining the target distance according to the reduced area overlapping rate;
wherein the second condition comprises: the area overlap ratio is less than a second overlap ratio threshold.
According to the technical scheme, the first image area and the second image area corresponding to a plurality of objects in the image are determined; wherein the first image area and the second image area correspond to the determined ones, respectively. And determining a target distance between the first image area and the second image area, wherein the target distance is used for embodying the matching degree between the first image area and the second image area, and the target distance is determined according to any one or more combinations of area overlapping rate and distance. And determining the sum of the optimal distances between the first image area and the second image area according to the target distance, wherein the sum of the optimal distances is used for showing that the matching degree between the first image area and the second image area is the highest. And determining that the matching relationship between the first image area and the second image area corresponding to the sum of the optimal distances is a target matching relationship between the first image area and the second image area. In the method, the target distance between the first image area and the second image area which belong to the same object accords with the relevant condition, so that the matching relation between the first image area and the second image area is accurately determined in a target distance mode.
Referring to fig. 9, which shows a schematic diagram of an image region matching apparatus provided in an embodiment of the present application, as shown in fig. 9, the apparatus includes:
a first determining unit 901, configured to determine a first region, a second region, and a third region corresponding to a plurality of objects in an image, respectively, where the first region, the second region, and the third region correspond to different target portions in the objects, respectively;
a second determining unit 902, configured to determine a first matching relationship between the first region and the second region and a second matching relationship between the second region and the third region, which are target matching relationships, respectively; the target matching relationship is determined by the method of any one of claims 1 to 7, when the target matching relationship is the first matching relationship, the first image region is the first region, and the second image region is the second region; when the target matching relationship is the second matching relationship, the first image area is the second area, and the second image area is the third area;
a third determining unit 903, configured to determine a third matching relationship between the first region and the third region according to the first matching relationship and the second matching relationship.
In a possible implementation manner, the object is a human body, the first region is an image region in a human body frame, the second region is an image region in a human head frame, and the third region is an image region in a human face frame.
It can be seen from the above technical solution that, when a first region and a third region between a plurality of objects in an image need to be matched, first, the first region, the second region, and the third region of the plurality of objects can be determined, and the first region, the second region, and the third region respectively correspond to different target portions in the objects. Then, a first matching relationship between the first region and the second region and a second matching relationship between the second region and the third region, which are target matching relationships, respectively, may be determined. Thus, a third matching relationship between the first region and the third region is determined based on the first matching relationship and the second matching relationship. In the method, the second region is introduced, and the second region can be used for accurately determining the first matching relationship between the second region and the first region and can also be used for determining the second matching relationship between the second region and the third region, so that the second region is used as the intermediate quantity for matching between the first region and the third region, and the accurate determination of the matching relationship between the first region and the third region is realized.
The embodiment of the application provides a device, which can be a data processing device executing the identity recognition method, and the device comprises a processor and a memory:
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to perform the above method according to instructions in the program code.
The embodiment of the application provides a computer readable storage medium for storing a computer program, and the computer program is used for executing the method.
As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the above embodiment methods can be implemented by software plus a necessary general hardware platform. Based on such understanding, the technical solution of the present application may be essentially or partially implemented in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the method according to the embodiments or some parts of the embodiments of the present application.
It should be noted that, in the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (12)
1. An image region matching method, characterized in that the method comprises:
determining a first image area and a second image area which correspond to a plurality of objects in an image respectively; the first image region and the second image region respectively correspond to different target sites in a subject;
determining a target distance between the first image area and the second image area, wherein the target distance is used for embodying the matching degree between the first image area and the second image area; the target distance is determined according to any one or more combinations of area overlapping rate and distance;
determining the sum of the optimal distances between the first image area and the second image area according to the target distance, wherein the sum of the optimal distances is used for showing that the matching degree between the first image area and the second image area is the highest;
and determining that the matching relationship between the first image area and the second image area corresponding to the sum of the optimal distances is a target matching relationship between the first image area and the second image area.
2. The method of claim 1, wherein determining the target distance between the first image region and the second image region comprises:
determining a distance between a target position of the first image area and a target position of the second image area; the target position is a position determined on the first image area and the second image area according to a preset mode;
and determining the target distance according to the distance.
3. The method of claim 2, wherein said determining the target distance from the distance comprises:
increasing the distance when it is determined that the distance satisfies a first condition;
determining the target distance according to the increased distance;
the first condition includes: the distance is greater than a distance threshold.
4. The method of claim 3, wherein when the target region corresponding to the first image region comprises the target region corresponding to the second image region, the first condition further comprises:
an area overlapping ratio between the first image region and the second image region is smaller than a first overlapping ratio threshold, the area overlapping ratio being determined according to the first image region, the second image region, and an overlapping region between the first image region and the second image region.
5. The method of claim 2, wherein the distance is any one of a vertical distance, a euclidean distance, a manhattan distance, and a manhattan distance including a weight.
6. The method of claim 1, wherein when the target portion corresponding to the first image region comprises the target portion corresponding to the second image region, the determining the target distance between the first image region and the second image region comprises:
determining an area overlap ratio between the first image region and the second image region; the area overlapping ratio is determined according to the first image area, the second image area and an overlapping area between the first image area and the second image area;
and determining the target distance according to the area overlapping rate.
7. The method of claim 6, wherein determining the target distance according to the area overlap ratio comprises:
when the area overlapping rate is determined to meet a second condition, reducing the area overlapping rate;
determining the target distance according to the reduced area overlapping rate;
wherein the second condition comprises: the area overlap ratio is less than a second overlap ratio threshold.
8. An image region matching method, characterized in that the method comprises:
determining a first region, a second region and a third region respectively corresponding to a plurality of objects in an image, wherein the first region, the second region and the third region respectively correspond to different target parts in the objects;
determining a first matching relationship between the first region and the second region and a second matching relationship between the second region and the third region, which are target matching relationships, respectively; the target matching relationship is determined by the method of any one of claims 1 to 7, when the target matching relationship is the first matching relationship, the first image region is the first region, and the second image region is the second region; when the target matching relationship is the second matching relationship, the first image area is the second area, and the second image area is the third area;
and determining a third matching relationship between the first region and the third region according to the first matching relationship and the second matching relationship.
9. The method according to claim 8, wherein the object is a human body, the first region is an image region within a frame of the human body, the second region is an image region within a frame of a human head, and the third region is an image region within a frame of a human face.
10. An image region matching apparatus, characterized in that the apparatus comprises:
the device comprises a first determining unit, a second determining unit and a display unit, wherein the first determining unit is used for determining a first image area and a second image area which correspond to a plurality of objects in an image respectively; the first image region and the second image region respectively correspond to different target sites in a subject;
a second determining unit, configured to determine a target distance between the first image region and the second image region, where the target distance is used to represent a matching degree between the first image region and the second image region; the target distance is determined according to any one or more combinations of area overlapping rate and distance;
a third determining unit, configured to determine, according to the target distance, a sum of optimal distances between the first image region and the second image region, where the sum of optimal distances is used to show that a matching degree between the first image region and the second image region is the highest;
a fourth determining unit, configured to determine that a matching relationship between the first image region and the second image region corresponding to the sum of the optimal distances is a target matching relationship between the first image region and the second image region.
11. An image region matching apparatus, characterized in that the apparatus comprises:
a first determining unit, configured to determine a first region, a second region, and a third region corresponding to a plurality of objects in an image, respectively, where the first region, the second region, and the third region correspond to different target portions in the objects, respectively;
a second determination unit configured to determine a first matching relationship between the first region and the second region and a second matching relationship between the second region and the third region, which are target matching relationships, respectively; the target matching relationship is determined by the method of any one of claims 1 to 7, when the target matching relationship is the first matching relationship, the first image region is the first region, and the second image region is the second region; when the target matching relationship is the second matching relationship, the first image area is the second area, and the second image area is the third area;
a third determining unit, configured to determine a third matching relationship between the first region and the third region according to the first matching relationship and the second matching relationship.
12. An apparatus, comprising a processor and a memory:
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to perform the image region matching method of any one of claims 1 to 7 or the image region matching method of claim 8 or 9 according to instructions in the program code.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010435120.XA CN111476214A (en) | 2020-05-21 | 2020-05-21 | Image area matching method and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010435120.XA CN111476214A (en) | 2020-05-21 | 2020-05-21 | Image area matching method and related device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111476214A true CN111476214A (en) | 2020-07-31 |
Family
ID=71764586
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010435120.XA Pending CN111476214A (en) | 2020-05-21 | 2020-05-21 | Image area matching method and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111476214A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114022910A (en) * | 2022-01-10 | 2022-02-08 | 杭州巨岩欣成科技有限公司 | Swimming pool drowning prevention supervision method and device, computer equipment and storage medium |
| WO2022198821A1 (en) * | 2021-03-25 | 2022-09-29 | 深圳市商汤科技有限公司 | Method and apparatus for performing matching between human face and human body, and electronic device, storage medium and program |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105160297A (en) * | 2015-07-27 | 2015-12-16 | 华南理工大学 | Masked man event automatic detection method based on skin color characteristics |
| US20160203525A1 (en) * | 2015-01-12 | 2016-07-14 | Ebay Inc. | Joint-based item recognition |
| CN109190454A (en) * | 2018-07-17 | 2019-01-11 | 北京新唐思创教育科技有限公司 | The method, apparatus, equipment and medium of target person in video for identification |
| CN109948494A (en) * | 2019-03-11 | 2019-06-28 | 深圳市商汤科技有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN110070005A (en) * | 2019-04-02 | 2019-07-30 | 腾讯科技（深圳）有限公司 | Images steganalysis method, apparatus, storage medium and electronic equipment |
| CN110163096A (en) * | 2019-04-16 | 2019-08-23 | 北京奇艺世纪科技有限公司 | Character recognition method, device, electronic equipment and computer-readable medium |
| CN110427905A (en) * | 2019-08-08 | 2019-11-08 | 北京百度网讯科技有限公司 | Pedestrian tracting method, device and terminal |
| CN110674719A (en) * | 2019-09-18 | 2020-01-10 | 北京市商汤科技开发有限公司 | Target object matching method and device, electronic equipment and storage medium |
-
2020
- 2020-05-21 CN CN202010435120.XA patent/CN111476214A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160203525A1 (en) * | 2015-01-12 | 2016-07-14 | Ebay Inc. | Joint-based item recognition |
| CN105160297A (en) * | 2015-07-27 | 2015-12-16 | 华南理工大学 | Masked man event automatic detection method based on skin color characteristics |
| CN109190454A (en) * | 2018-07-17 | 2019-01-11 | 北京新唐思创教育科技有限公司 | The method, apparatus, equipment and medium of target person in video for identification |
| CN109948494A (en) * | 2019-03-11 | 2019-06-28 | 深圳市商汤科技有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN110070005A (en) * | 2019-04-02 | 2019-07-30 | 腾讯科技（深圳）有限公司 | Images steganalysis method, apparatus, storage medium and electronic equipment |
| CN110163096A (en) * | 2019-04-16 | 2019-08-23 | 北京奇艺世纪科技有限公司 | Character recognition method, device, electronic equipment and computer-readable medium |
| CN110427905A (en) * | 2019-08-08 | 2019-11-08 | 北京百度网讯科技有限公司 | Pedestrian tracting method, device and terminal |
| CN110674719A (en) * | 2019-09-18 | 2020-01-10 | 北京市商汤科技开发有限公司 | Target object matching method and device, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 谭园芳: "视频图像中的运动人体检测与跟踪算法研究" * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022198821A1 (en) * | 2021-03-25 | 2022-09-29 | 深圳市商汤科技有限公司 | Method and apparatus for performing matching between human face and human body, and electronic device, storage medium and program |
| CN114022910A (en) * | 2022-01-10 | 2022-02-08 | 杭州巨岩欣成科技有限公司 | Swimming pool drowning prevention supervision method and device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210182334A1 (en) | Visual localization method | |
| US8880097B1 (en) | Systems and methods for statistically associating mobile devices to households | |
| US7848545B2 (en) | Method of and system for image processing and computer program | |
| CN105404860A (en) | Method and device for managing information of lost person | |
| WO2019007253A1 (en) | Image recognition method, apparatus and device, and readable medium | |
| US20200356800A1 (en) | Polygonal region detection | |
| CN111476214A (en) | Image area matching method and related device | |
| CN110889314B (en) | Image processing method, device, electronic equipment, server and system | |
| US20240087368A1 (en) | Companion animal life management system and method therefor | |
| US20210295827A1 (en) | Method and device for evaluating quality of content, electronic equipment, and storage medium | |
| CN111601380B (en) | Position location method, device and equipment based on position fingerprint and storage medium | |
| CN111383246A (en) | Scroll detection method, device and equipment | |
| CN114880513A (en) | Target retrieval method and related device | |
| CN111553302A (en) | Key frame selection method, device, equipment and computer readable storage medium | |
| CN107121661B (en) | Positioning method, device and system and server | |
| WO2024001617A1 (en) | Method and apparatus for identifying behavior of playing with mobile phone | |
| KR20160044858A (en) | System and Method for Searching Missing Family Using Facial Information and Storage Medium of Executing The Program | |
| WO2007077694A1 (en) | Muzzle pattern information creating device, muzzle pattern information checking device, muzzle pattern information creating program, and muzzle pattern information creating method | |
| CN112333182B (en) | File processing method, device, server and storage medium | |
| CN115222621A (en) | Image correction method, electronic device, storage medium, and computer program product | |
| US9542043B2 (en) | Facilitation of error tolerant image tracing optimization | |
| CN114359796A (en) | Target identification method and device and electronic equipment | |
| EP3076370A1 (en) | Method and system for selecting optimum values for parameter set for disparity calculation | |
| CN112861900A (en) | Image system and detection method | |
| CN114491106A (en) | Face recognition method, priority determination method, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |