CN115035186B - Target object marking method and terminal equipment - Google Patents
Target object marking method and terminal equipment Download PDFInfo
- Publication number
- CN115035186B CN115035186B CN202111471242.5A CN202111471242A CN115035186B CN 115035186 B CN115035186 B CN 115035186B CN 202111471242 A CN202111471242 A CN 202111471242A CN 115035186 B CN115035186 B CN 115035186B
- Authority
- CN
- China
- Prior art keywords
- parameter
- area
- indicated
- region
- parameters
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04847—Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Mathematical Analysis (AREA)
- Pure & Applied Mathematics (AREA)
- Computational Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Mathematical Optimization (AREA)
- Human Computer Interaction (AREA)
- Geometry (AREA)
- Algebra (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Telephone Function (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The application provides a target object marking method and terminal equipment, and relates to the technical field of terminals. The target object marking method is characterized in that when the number of times that a certain position of a target object is detected is larger and the area of the frame is larger, semantic information indicating the position is more important. Thus, since determining the cross-over ratio basis includes: the overlapping area of the target region (namely the overlapping region of the first region and any second region) and the region indicated by any parameter in the first parameter set can express the importance of the semantic information of the target region; and the overlapping areas of the first region and any second region and the regions indicated by the rest parameters in the first parameter set can express the importance of the semantic information of the first region and the second region. Therefore, the terminal equipment obtains high reliability of the intersection ratio. Furthermore, the reliability of the target object mark displayed on the first interface by the terminal device is high based on the comparison result of the intersection ratio and the preset threshold.
Description
Technical Field
The present application relates to the field of terminal technologies, and in particular, to a target object marking method and a terminal device.
Background
With the development of terminal technology, terminal devices have become a part of people's work and life. The terminal device is generally integrated with functions of image display, information transceiving, voice playing and the like, and brings convenience to work and life of people.
For the image display function of the terminal device, when the terminal device is displaying an image, a target object in the displayed image may be marked.
Currently, the reliability of the terminal device for marking the target object in the image is low. For example, there may be redundant marks in the image or there may be a problem that the target object is not marked.
Disclosure of Invention
The embodiment of the application provides a target object marking method and terminal equipment, so as to solve the problem that the reliability of the terminal equipment for marking a target object in an image is low.
In a first aspect, an embodiment of the present application provides a target object marking method, including: the terminal equipment displays a first interface, and the first interface comprises the acquired image. The terminal equipment detects a target object of a specified type in the image to obtain a first parameter set. Any parameter in the first parameter set is used for indicating the region of the target object. The terminal equipment determines a first parameter meeting a preset condition from the first parameter set, and determines a second parameter set except the first parameter. The terminal device determines the area of the first region indicated by the first parameter, and respectively determines the area intersection ratio of the second regions indicated by any one second parameter in the second parameter set. Wherein, determining the intersection ratio basis comprises: the target region is an overlapping region of the first region and any second region, and the first region indicated by the first parameter and the second region indicated by any second parameter are respectively overlapped with regions indicated by the rest of parameters in the first parameter set. The terminal equipment marks a target object of a first area indicated by the first parameter on the first interface, and marks a target object of a second area indicated by the second parameter, wherein the intersection ratio of the target object and the first area is smaller than a preset threshold value.
According to the target object marking method provided by the embodiment of the application, when the number of times that a certain position of a target object is detected is larger and the area of the frame selection is larger, the more remarkable the characteristic of the position is, namely, the more important the semantic information indicated by the position is. Thus, since determining the cross-over ratio basis includes: the overlapping area of the target region (i.e. the overlapping area of the first region and any second region) and the region indicated by any parameter in the first parameter set can express the importance of the semantic information of the target region; and the overlapping areas of the first region and any second region with the regions indicated by the remaining parameters in the first parameter set respectively. Thus, the importance of the semantic information of the first region and the second region can be expressed more effectively. Therefore, the terminal equipment obtains high reliability of the intersection ratio. Furthermore, the terminal device displays the target object mark on the first interface based on the comparison result of the intersection ratio and the preset threshold, so that the reliability is high, and the situation that redundant marks exist in the image or some target objects are not marked is reduced.
In one possible embodiment, the intersection-to-union ratio is a ratio of an intersection parameter and a union parameter, wherein determining the intersection parameter includes: the sum of the overlapping areas of the target region and the region indicated by any one of the parameters in the first parameter set, and the determining the basis of the union parameter comprises: and the first region indicated by the first parameter and the second region indicated by any second parameter are respectively the sum of the overlapping areas of the regions indicated by the rest parameters in the first parameter set.
As will be appreciated, the basis in determining the intersection parameters includes: when the sum of the overlapping areas of the target region and the region indicated by any one parameter in the first parameter set is larger than the sum of the overlapping areas of the target region and the region indicated by any one parameter in the first parameter set, the weight of the intersection parameter in the calculation of the intersection ratio can be more indicated, and thus the semantic information importance of the target region can be more accurately indicated. The basis for determining the parameters of the union set comprises the following steps: and when the sum of the overlapping areas of the first region indicated by the first parameter and the second region indicated by any second parameter and the regions indicated by the rest parameters in the first parameter set is respectively greater than the sum of the overlapping areas of the first region indicated by the first parameter and the regions indicated by the rest parameters in the first parameter set, the weight of the parameter of the union set in the calculation of the intersection ratio can be more indicated. Therefore, the importance of the semantic information of the first area and the second area can be accurately indicated.
Further, the intersection-to-parallel ratio satisfies the formula
Wherein cover _ IoU i,j In a cross-over ratio>
In order to be the intersection parameters,
as a union of the parameters, s i Is indicated for the first parameterArea of a region, s j Area of second region, s, indicated for any second parameter k Is the area of the target region, S i∩t Is the sum of the overlapping areas of the first region and the regions indicated by the parameters in the first parameter set except the first parameter and any second parameter, S j∩t Is the sum of the overlapping areas of the second regions and the regions indicated by the parameters in the first parameter set except the first parameter and any second parameter, S k∩t And i, j, t and N are positive integers, and are the sum of the overlapping areas of the target region and the regions indicated by the parameters except the first parameter and any second parameter in the first parameter set.
In a possible implementation manner, the terminal device determines an area of a first region indicated by a first parameter, and an area intersection ratio of a second region indicated by any one of second parameters in a second parameter set respectively includes: the terminal device determines the area of the first region indicated by the first parameter. The terminal device traverses the second parameters in the second parameter set. The terminal device determines the area of the overlapping area of the first area indicated by the first parameter and the second area indicated by the traversed second parameter, and calculates the area of the second area indicated by the traversed second parameter. And the terminal equipment determines the area of the first region indicated by the first parameter and the area of the second region indicated by the traversed second parameter, and compares the area of the first region with the area of the second region indicated by the traversed second parameter. And the terminal equipment judges whether the second parameter set has the remaining second parameters. And if the residual second parameters exist, continuously traversing the second parameters in the second parameter set until the residual second parameters do not exist in the second parameter set.
In a possible embodiment, the area indicated by any one of the first parameter set is a rectangular frame, a circular frame or an oval frame for framing the target object.
Wherein, the rectangular box, the circular box or the oval box can mark the area which any parameter in the first parameter set is used for indication in a striking way.
Further, the area indicated by any parameter in the first parameter set is a rectangular frame for framing the target object, and the first parameter setIs the coordinates of the two end points of the diagonal line of the rectangular frame of the framing target object. The area of the rectangular frame indicated by the first parameter satisfies s i ＝(x i2 -x i1 )(y i2 -y i1 ) Wherein s is i Area of rectangular box indicated for first parameter, (x) i1 ，y i1 ) And (x) i2 ，y i2 ) Two end point coordinates of a diagonal line of the rectangular frame indicated by the first parameter, respectively.
The area of the rectangular frame indicated by the second parameter satisfies s j ＝(x j2 -x j1 )(y j2 -y j1 ) Wherein s is j Area of the rectangular box indicated for the second parameter, (x) j1 ，y j1 ) And (x) j2 ，y j2 ) Two end point coordinates of a diagonal line of the rectangular frame indicated by the second parameter, respectively.
Understandably, by the equation s i ＝(x i2 -x i1 )(y i2 -y i1 ) Calculating the area of the rectangular box indicated by the first parameter, and s j ＝(x j2 -x j1 )(y j2 -y j1 ) And the area of the rectangular frame indicated by the second parameter is calculated, so that the method is convenient and quick, and the calculation resource is saved.
In a possible implementation manner, when the terminal device obtains the first parameter set, the terminal device further obtains a confidence level of each parameter in the first parameter set, where the first parameter that meets the preset condition is: and the parameter with the highest confidence coefficient in the first parameter set or the parameter with the confidence coefficient larger than the preset confidence coefficient.
As can be appreciated, the first parameter is: when the parameter with the highest confidence level in the first parameter set or the parameter with the confidence level greater than the preset confidence level is in the first parameter set, the reliability of the first parameter may be high. Furthermore, the reliability of subsequent marking of the target object based on the first parameter is also high.
In one possible implementation, the target objects of the specified type are people in the first interface, or all types of objects in the first interface.
In a possible implementation manner, the first interface is a photographing preview interface of a camera application, an image display interface of a cropping application, or a video playing interface of a video playing application.
Therefore, the target object marking method provided by the embodiment of the application is wide in application range.
In a second aspect, an embodiment of the present application further provides a terminal device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program, so that the terminal device performs the target object marking method as described in the first aspect or any implementation manner of the first aspect.
In a third aspect, the present application further provides a target object marking apparatus, including: and the display unit is used for displaying a first interface, and the first interface comprises the acquired image. The processing unit is used for detecting a target object of a specified type in the image to obtain a first parameter set, wherein any parameter in the first parameter set is used for indicating the region where the target object is located. And the processing unit is also used for determining a first parameter meeting a preset condition from the first parameter set and determining a second parameter set except the first parameter. The processing unit is further configured to determine an area of a first region indicated by the first parameter, and determine an area intersection ratio of a second region indicated by any one of the second parameters in the second parameter sets, where determining the intersection ratio includes: the target region is an overlapping region of the first region and any second region, and the first region indicated by the first parameter and the second region indicated by any second parameter are respectively overlapped with regions indicated by the rest parameters in the first parameter set. And the display unit is also used for marking the target object of the first area indicated by the first parameter on the first interface and marking the target object of the second area indicated by the second parameter, wherein the intersection ratio of the target object and the first area is less than the preset threshold value.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium storing instructions that, when executed, cause a computer to perform a target object marking method as described in the first aspect or any implementation manner of the first aspect.
In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program and when the computer program is executed, causes a computer to execute the target object marking method as described in the first aspect or any implementation manner of the first aspect.
It should be understood that the second aspect to the fifth aspect of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by the aspects and the corresponding possible implementations are similar and will not be described again.
Drawings
FIG. 1 is a schematic diagram of detection of a person taking a preview interface;
FIG. 2 is a schematic illustration of tagging a person for a capture preview interface;
fig. 3 is a schematic diagram of a hardware system architecture of a terminal device according to an embodiment of the present application;
fig. 4 is a schematic diagram of a software system architecture of a terminal device according to an embodiment of the present application;
fig. 5 is a flowchart of a target object marking method provided in an embodiment of the present application;
FIG. 6 is a schematic diagram of an embodiment of an object detection model detecting an image output parameter in a first interface;
fig. 7 is a schematic diagram of a rectangular box indicated by each parameter in the first parameter set according to an embodiment of the present application;
fig. 8 is a schematic interface diagram illustrating tagging of a person a and a person b in a first interface according to an embodiment of the present application;
FIG. 9 is a detailed flowchart of S504 in FIG. 5;
fig. 10 is a schematic interface diagram illustrating labeling of swans a and b in a first interface according to an embodiment of the present disclosure;
fig. 11 is a schematic interface diagram illustrating marking of target objects of various types in a first interface according to an embodiment of the present application;
fig. 12 is a schematic functional block diagram of a target object labeling apparatus according to an embodiment of the present application;
fig. 13 is a schematic hardware structure diagram of a terminal device according to an embodiment of the present application;
fig. 14 is a schematic structural diagram of a chip according to an embodiment of the present application.
Detailed Description
In the embodiments of the present application, terms such as "first" and "second" are used to distinguish the same or similar items having substantially the same function and action. For example, the first value and the second value are only used to distinguish different values, and the order of the values is not limited. Those skilled in the art will appreciate that the terms "first," "second," etc. do not denote any order or quantity, nor do the terms "first," "second," etc. denote any order or importance.
It is noted that, in the present application, words such as "exemplary" or "for example" are used to mean exemplary, illustrative, or descriptive. Any embodiment or design described herein as "exemplary" or "such as" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "such as" is intended to present concepts related in a concrete fashion.
In this application, "at least one" means one or more, "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a alone, A and B together, and B alone, wherein A and B may be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of the singular or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or multiple.
With the development of terminal technology, terminal devices have become a part of people's work and life. The terminal device is generally integrated with functions of image display, information transceiving, voice playing and the like, and brings convenience to work and life of people. For the image display function of the terminal device, when the terminal device is displaying an image, a target object in the displayed image may be marked.
As shown in fig. 1 (a), the terminal device displays a first interface 101, the first interface 101 including an icon 102 of "camera". As shown in fig. 1 (b), the terminal device may display a second interface 103 (i.e., a shooting preview interface) in response to a user's trigger operation on the icon 102 of "camera". The second interface 103 includes an image acquired by a camera of the terminal device. Further, the terminal device may detect the person a and the person b in the image based on the object detection model. Since there is a certain error in the detection of the object detection model, 4 coordinate parameters may be obtained by detecting the person a and the person b. Wherein, the 4 coordinate parameters can be coordinate parameter 1[ (x 1, y 1), (x 2, y 2) ], coordinate parameter 2[ (x 3, y 3), (x 4, y 4) ], coordinate parameter 3[ (x 5, y 5), (x 6, y 6) ], and coordinate parameter 4[ (x 7, y 7), (x 8, y 8) ]; and confidence 0.6 for coordinate parameter 1, confidence 0.4 for coordinate parameter 2, confidence 0.4 for coordinate parameter 3, and confidence 0.3 for coordinate parameter 4. And then the terminal equipment determines the parameter with the maximum confidence as the coordinate parameter 1. Wherein, the coordinate parameter 1, the coordinate parameter 2, the coordinate parameter 3, and the coordinate parameter 4 are used to indicate two end point coordinates of a diagonal line of a rectangular frame of a person in the framed-selected image.
As shown in (c) in fig. 1, a coordinate parameter 1 indicates a rectangular frame 1, a coordinate parameter 2 indicates a rectangular frame 2, a coordinate parameter 3 indicates a rectangular frame 3, and a coordinate parameter 4 indicates a rectangular frame 4.
The terminal device calculates the area of the rectangular frame 1 according to the coordinate parameter 1, calculates the area of the rectangular frame 2 according to the coordinate parameter 2, calculates the area of the rectangular frame 3 according to the coordinate parameter 3, and calculates the area of the rectangular frame 4 according to the coordinate parameter 4. Thus, the terminal equipment can be calculated according to the formula
And calculating the intersection ratio between the rectangular frame 1 (namely the rectangular frame corresponding to the coordinate parameter 1 with the highest confidence coefficient) and the rectangular frame n. Mu is cross-over ratio, S t Is the area of the overlapping region of the rectangular frame 1 and the rectangular frame n, S 1 Is the area of the rectangular frame 1, S n Is the area of a rectangular frame n, and n is more than or equal to 2 and less than or equal to 4.
It can be seen that, in the above calculation formula, the assigned weight of each position in the overlapping region of the rectangular frame 1 and the rectangular frame n is the same, that is, the importance of the semantic information of each position in the overlapping region of the default rectangular frame 1 and the rectangular frame n is the same. In addition, in calculating S 1 +S n -S t In this case, the weight assigned to each position in the overlapping region of the rectangular frame 1, the rectangular frame n, and the rectangular frame 1 and the rectangular frame n is the same, that is, the importance of the semantic information of each position in the overlapping region of the default rectangular frame 1, the rectangular frame n, and the rectangular frame 1 and the rectangular frame n is the same. In fact, the importance of the semantic information of the rectangular frame 1, the rectangular frame n, and each position of the overlapping area of the rectangular frame 1 and the rectangular frame n is different. That is, the more times a certain region on a person is framed and the larger the framed area, the more important semantic information describing the region becomes.
This may result in a low reliability of the calculated intersection ratio between the rectangular frame 1 and the rectangular frame n. Illustratively, the terminal device is preconfigured with a cross-over ratio threshold of 0.5. Suppose that the calculated intersection ratio between the rectangular frame 1 and the rectangular frame 2 is 0.6, the intersection ratio between the rectangular frame 1 and the rectangular frame 3 is 0.9, and the intersection ratio between the rectangular frame 1 and the rectangular frame 4 is 0.51.
It can be seen that the cross-over ratios obtained by the above calculation are all greater than the cross-over ratio threshold value of 0.5. In this manner, the terminal device deletes coordinate parameter 2, coordinate parameter 3, and coordinate parameter 4. Further, as shown in fig. 2, the terminal device marks a rectangular frame 1 for the person a in the image according to the coordinate parameter 1. It is to be understood that in the above manner, the person b is not marked, and thus, the reliability of the marking of the person in the image is low.
In view of this, the present application provides a target object labeling method, which may obtain a first parameter set when detecting a target object of a specified type in an acquired image. Any parameter in the first parameter set is used for indicating the area where the target object is located. Furthermore, when calculating the intersection ratio of the first region indicated by the first parameter satisfying the preset condition in the first parameter set and the second region indicated by the other second parameters, the sum of the overlapping areas of the region indicated by each parameter and the regions indicated by the other parameters is considered.
It is understood that the more times a certain position of the target object is detected and the larger the outlined area, the more important the semantic information describing the position indication. In this way, when calculating the intersection ratio of the first region indicated by the first parameter satisfying the preset condition in the first parameter set and the second region indicated by the other second parameters, the sum of the overlapping areas of the region indicated by each parameter and the regions indicated by the other parameters is considered. Thus, the terminal device obtains the intersection ratio based on calculation with high reliability. Furthermore, the terminal device displays the target object mark on the first interface based on the comparison result of the intersection ratio and the preset threshold, so that the reliability of the target object mark is high, and the situation that redundant marks possibly exist in the image or the existing target object is not marked is reduced.
It is understood that the terminal equipment may also be referred to as a terminal (terminal), a User Equipment (UE), a Mobile Station (MS), a Mobile Terminal (MT), and so on. The terminal device may be a mobile phone (mobile phone), a smart tv, a wearable device, a tablet computer (Pad), a computer with a wireless transceiving function, a Virtual Reality (VR) terminal device, an Augmented Reality (AR) terminal device, a wireless terminal in industrial control (industrial control), a wireless terminal in self-driving (self-driving), a wireless terminal in remote surgery (remote medical supply), a wireless terminal in smart grid (smart grid), a wireless terminal in transportation safety (transportation safety), a wireless terminal in smart city (smart city), a wireless terminal in smart home (smart home), and so on. The embodiment of the present application does not limit the specific technology and the specific device form adopted by the terminal device.
In order to better understand the embodiments of the present application, the following describes the structure of the terminal device according to the embodiments of the present application. Exemplarily, fig. 3 is a schematic structural diagram of a terminal device provided in an embodiment of the present application.
The terminal device may include a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a sensor module 180, a button 190, an indicator 192, a camera 193, a display screen 194, and the like. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.
It is to be understood that the illustrated structure of the embodiments of the present application does not constitute a specific limitation to the terminal device. In other embodiments of the present application, a terminal device may include more or fewer components than shown, or some components may be combined, or some components may be split, or a different arrangement of components may be used. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
The software system of the terminal device may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture, which is not described herein again. The embodiment of the present application takes an Android system with a layered architecture as an example, and exemplarily illustrates a software structure of the terminal device 100. Fig. 4 is a block diagram of a software structure of a terminal device to which the embodiment of the present application is applied. The layered architecture divides the software system of the terminal device 100 into several layers, each layer having a clear role and division of labor. The layers communicate with each other through a software interface. In some embodiments, the Android system may be divided into five layers, namely an application layer (applications), an application framework layer (application framework), an Android runtime (Android runtime), and a system library, a Hardware Abstraction Layer (HAL), and a kernel layer (kernel).
The following describes the technical solutions of the present application and how to solve the above technical problems with specific embodiments. The following embodiments may be implemented independently or in combination, and details of the same or similar concepts or processes may not be repeated in some embodiments.
The following describes a target object tagging method provided in an embodiment of the present application, taking a terminal device as a mobile phone 100 and a target object of a specified type as a person as an example. As shown in fig. 5, the target object marking method provided by the present application includes:
s501: the cell phone 100 displays a first interface including the captured image.
Illustratively, as also shown in fig. 1, the handset 100 displays a system home interface 101, wherein the system home interface 101 includes an icon 102 for "camera". The cell phone 100 can capture an image with a camera and display a first interface 103 (i.e., a photo preview interface) in response to a user's trigger operation of the "camera" icon 102. The first interface 103 includes a captured image, and the captured image includes a person a and a person b.
S502: the mobile phone 100 detects a person in the image to obtain a first parameter set. Any parameter in the first parameter set is used for indicating the region of the target object.
Therein, the cell phone 100 is provided with a target detection model. The target detection model is obtained by inputting a plurality of character image samples into a target detection network to be trained by the mobile phone 100. The target detection network may be, but not limited to, a YOLO network or an RCNN (regions with a connected neural network, RCNN) network.
As can be appreciated, since the target detection model is trained based on a plurality of human image samples, the mobile phone 100 can detect a human (i.e., a target object of a specified type) in the captured image based on the target detection model. Further, as shown in fig. 6, the object detection model may output a first set of parameters. The number of parameters that person a and person b in the first parameter set may obtain may be greater than 2 due to some error in the detection of the object detection model. Illustratively, the first set of parameters may include parameter 1, parameter 2, parameter 3, and parameter 4. The parameter 1, the parameter 2, the parameter 3, and the parameter 4 may be two end point coordinates of a diagonal line of each rectangular frame for framing a person in the image.
Illustratively, parameter 1 may be { (x) 11 ，y 11 )；(x 12 ，y 12 ) I.e. the coordinates of the two end points of the diagonal of the rectangular frame 1; the parameter 2 may be { (x) 21 ，y 21 )；(x 22 ，y 22 ) I.e. the two end point coordinates of the diagonal of the rectangular frame 2; the parameter 3 may be { (x) 31 ，y 31 )；(x 32 ，y 32 ) I.e. the two end point coordinates of the diagonal of the rectangular box 3; the parameter 4 may be { (x) 41 ，y 41 )；(x 42 ，y 42 ) I.e. the coordinates of the two end points of the diagonal of the rectangular frame 4. The positional relationship among the rectangular frames 1, 2, 3, and 4 may be as shown in fig. 7. Of course, the first set of parameters may also include more parameters, which are only illustrated here.
The first parameter set output by the target detection model may be a feature matrix as follows:
as can be understood, the captured image includes a person a and a person b; and the first set of parameters includes 4 parameters. That is to say that the first parameter set contains 2 redundant parameters. Next, how to select 2 parameters with high reliability from the first parameter set is described with reference to S503-S505, and the person a and the person b are labeled according to the selected 2 parameters with high reliability.
S503: the mobile phone 100 determines a first parameter satisfying a preset condition from the first parameter set, and determines a second parameter set other than the first parameter set.
As also shown in fig. 6, while obtaining the first parameter set, the mobile phone 100 also obtains the confidence conf1 of the parameter 1, the confidence conf2 of the parameter 2, the confidence conf3 of the parameter 3, and the confidence conf4 of the parameter 4. The mobile phone 100 may select a parameter corresponding to the confidence with the largest value from the confidence conf1, the confidence conf2, the confidence conf3, and the confidence conf4 as a first parameter meeting a preset condition.
Illustratively, if the confidence conf1 of the parameter 1 is 0.85, the confidence conf2 of the parameter 2 is 0.75, the confidence conf3 of the parameter 3 is 0.7, and the confidence conf4 of the parameter 4 is 0.7, the parameter 1 is selected as the first parameter, and the confidence conf2, the confidence conf3, and the confidence conf4 are determined as the second parameter in the second parameter set.
In other embodiments, the first parameter satisfying the preset condition may be a parameter greater than a preset confidence level. As such, the number of first parameters may be one or more.
S504: the mobile phone 100 determines the area of the first region indicated by the first parameter, and the intersection ratio of the areas of the second regions indicated by any one of the second parameters in the second parameter sets.
Wherein, the basis for determining the intersection ratio comprises the following steps: the overlapping area of the target area and the area indicated by any parameter in the first parameter set, wherein the target area is the overlapping area of the first area and any second area; and the overlapping areas of the first region and any second region with the regions indicated by the remaining parameters in the first parameter set respectively.
Specifically, the intersection-to-union ratio is a ratio of an intersection parameter and a union parameter, and the determining of the intersection parameter includes: the sum of the overlapping areas of the target region and the region indicated by any one of the first set of parameters. And determining the basis of the union set parameters comprises: and the first region indicated by the first parameter and the second region indicated by any second parameter are respectively the sum of the overlapping areas of the regions indicated by the rest parameters in the first parameter set.
As will be appreciated, the basis in determining the intersection parameters includes: when the sum of the overlapping areas of the target region and the region indicated by any one parameter in the first parameter set is larger than the sum of the overlapping areas of the target region and the region indicated by any one parameter in the first parameter set, the weight of the intersection parameter in the calculation of the intersection ratio can be more indicated, and thus the semantic information importance of the target region can be more accurately indicated. The basis for determining the parameters of the union set comprises the following steps: and when the first region indicated by the first parameter and the second region indicated by any second parameter are respectively added with the overlapping areas of the regions indicated by the rest parameters in the first parameter set, the weights of the parameters of the union set in the calculation of the intersection ratio can be more indicated. Therefore, the importance of the semantic information of the first area and the second area can be accurately indicated.
In some embodiments, the intersection ratio of the area of the first region indicated by the first parameter to the area of the second region indicated by any one of the second parameters in the second parameter set satisfies:
for example, the mobile phone 100 may be based on a formula
And calculating the area of the first region indicated by the first parameter and the intersection ratio of the area of the second region indicated by any second parameter in the second parameter set.
Wherein cover _ IoU i,j In order to obtain the cross-over ratio,
in order to be the intersection parameters,
as a union of the parameters, s i Area of the first region, s, indicated for the first parameter j Area of the second region, s, indicated for a given second parameter k For the area of the target region, 2s is used in the intersection parameter since the target region overlaps the first region and the second region indicated by the specified second parameter once, respectively k ，/>
Indicated for the target area with a second parameter other than the first parameter, the specified second parameter, respectivelySum of the overlapping areas of the second regions. Understandably, be>
I.e. the sum of the overlapping areas of the target region and the region indicated by any one of the first set of parameters.
In addition, S i∩t Is the sum of the overlapping areas of the first region and the second region indicated by the second parameter except the first parameter and the specified second parameter, S j∩t Is the sum of the overlapping areas of the second region and the second regions indicated by the second parameters except the first parameter and the specified second parameter, S k∩t The sum of the overlapping areas of the target region and second regions indicated by second parameters except the first parameter and the specified second parameter is used. Understandably, due to the area s of the first region i Includes a target region s k Thus S i∩t Includes s k∩t (ii) a Similarly, due to the area s of the designated second region j Includes a target region s k Thus S j∩t In also includes s k∩t . It can be seen that k∩t Are superimposed twice, so that one s needs to be subtracted from the union parameter k∩t . As can be appreciated, the first and second electrodes,
i.e. the sum of the area of the first region indicated by the first parameter, the area of the second region indicated by the second parameter, and the overlapping areas of the first region and the specified second region with the regions indicated by the remaining parameters in the first parameter set, respectively, i, j, t, N being positive integers.
It is understood that, as the number of times a person's certain position is detected is larger and the area of overlap is larger, the more prominent the person's characteristics that describe that position, i.e., the more important the semantic information of that position indication, is. Furthermore, according to the overlapping area of the target area (i.e. the overlapping area of the first area and any one of the second areas) and the area indicated by any one of the parameters in the first parameter set, the obtained intersection parameter can express the importance of the semantic information of the target area; and according to the overlapping areas of the first region and any one of the second regions and the regions indicated by the rest of the parameters in the first parameter set, the obtained union set parameters can express the importance of the semantic information of the first region and the semantic information of the second region.
Thus, the obtained intersection set parameter can express the importance of the semantic information of the target region, and the obtained union set parameter can express the importance of the semantic information of the first region and the second region. Thus, the handset 100 has high reliability of the intersection ratio obtained based on the intersection parameter and the union parameter.
S505: the mobile phone 100 marks the character a of the first area indicated by the first parameter and marks the character b of the second area indicated by the second parameter, wherein the intersection ratio of the second parameter and the first area is smaller than a preset threshold value, on the first interface 103.
Specifically, the mobile phone 100 compares the calculated intersection ratio of the first parameter and the second parameter with a preset threshold respectively. When the intersection ratio of the second area indicated by the second parameter and the first area indicated by the first parameter is greater than a preset threshold (such as 0.5 or 0.6), indicating that the overlap degree of the second area indicated by the second parameter and the first area indicated by the first parameter is high, the second area indicated by the second parameter and the first area indicated by the first parameter are both the areas where the same person is located. Again, since the confidence of the first parameter is maximal, i.e. the reliability of the first parameter is higher than the reliability of the second parameter. In this way, the second parameter with the intersection ratio larger than the preset threshold value can be deleted, so as to avoid marking the area where the same person is located twice (i.e. avoid the situation that redundant marks exist).
In addition, when the intersection ratio of the second area indicated by the second parameter and the first area indicated by the first parameter is less than or equal to a preset threshold (such as 0.5 or 0.6), it is described that the overlapping degree of the second area indicated by the second parameter and the first area indicated by the first parameter is low, and it is considered that the second area indicated by the second parameter and the first area indicated by the first parameter are both areas where different persons are located. In this way, the second parameter with the intersection ratio less than or equal to the preset threshold may be retained, so that different persons are marked separately (i.e. avoiding the situation that some persons are not marked).
Based on the above, the calculated intersection ratio has high reliability. In this way, after the mobile phone 100 compares the intersection ratio with the preset threshold, the reliability of the reserved second parameter is also high.
Furthermore, the mobile phone 100 may mark the person a in the area indicated by the first parameter and mark the person b in the area indicated by the second parameter, where the intersection ratio of the person a and the first area is smaller than the preset threshold, on the first interface 103. Exemplarily, as shown in fig. 8, the area indicated by the first parameter is a rectangular frame 1, and then the person a is marked by the rectangular frame 1; if the area indicated by the second parameter with the intersection ratio less than or equal to the preset threshold is the rectangular frame 2, the person b is marked by the rectangular frame 2.
It is to be understood that, since the reliability of the first parameter described above is high (the confidence is high), the reliability of the remaining second parameter is also high. As such, the reliability of the persona marking in the first interface 103 based on the first parameter and the retained second parameter is also high. In this way, the situation that redundant marks may exist in the image or that the existing person is not marked is reduced. Still as shown in fig. 8, the person a and the person b are marked with the rectangle 1 and the rectangle frame 2 in fig. 8, respectively, and there are neither redundant marks nor cases where there are persons that are not marked in fig. 8.
Next, a specific execution flow of S504 described above will be described with reference to fig. 9, taking parameters in the first parameter set as coordinates of two end points of a diagonal line of a rectangular frame for framing a person in an image as an example. As shown in fig. 9, the specific execution flow of S504 includes:
s901: the mobile phone 100 determines the area of the rectangular frame indicated by the first parameter according to the first parameter.
Suppose that the first parameter is { (x) i1 ，y i1 )；(x i2 ，y i2 ) }; the mobile phone 100 can calculate the equation s i ＝(x i2 -x i1 )(y i2 - y i1 ) Determining the area s of the rectangular frame indicated by the first parameter i 。
S902: the handset 100 traverses the second parameter in the second set of parameters.
S903: the mobile phone 100 calculates the area of the overlapping region of the rectangular frame indicated by the first parameter and the rectangular frame indicated by the traversed second parameter, and calculates the area of the rectangular frame indicated by the traversed second parameter.
It is understood that the overlapping area of the rectangular frame indicated by the first parameter and the rectangular frame indicated by the traversed second parameter is the above-mentioned target area. Let the second parameter be { (x) j1 ，y j1 )；(x j2 ，y j2 ) The mobile phone 100 can be according to s j ＝(x j2 -x j1 )(y j2 -y j1 ) Determining the area s of the rectangular frame indicated by the second parameter j 。
In addition, the mobile phone 100 can also use the formula x k1 ＝max(xi 1, x j1 ),x k2 ＝min(x i2 ,x j2 )； y k1 ＝max(y i1 ,y j1 ),y k2 ＝min(y i2 ,y j2 ) Determining coordinates of two end points on a diagonal of the target area as { (x) k1 ，y k1 )；(x k2 ，y k2 ) }. Further, the mobile phone 100 can calculate the equation s k ＝(x k2 -x k1 )(y k2 -y k1 ) And determining the area of the target area. Understandably, by the formula s i ＝(x i2 -x i1 )(y i2 -y i1 ) Calculating the area of the rectangular box indicated by the first parameter, and s j ＝(x j2 -x j1 )(y j2 -y j1 ) The area of the rectangular frame indicated by the second parameter is calculated, so that the method is convenient and quick, and the calculation resource is saved.
S904: the mobile phone 100 calculates the formula
And calculating the intersection ratio of the area of the rectangular frame indicated by the first parameter and the area of the rectangular frame indicated by the traversed second parameter.
S905: the mobile phone 100 determines whether there are any remaining second parameters in the second parameter set, if yes, returns to execute S902, and if no, ends.
In this way, the mobile phone 100 may determine the area of the first region indicated by the first parameter, and determine the intersection ratio of the areas of the second regions indicated by any one of the second parameters in the second parameter sets.
Next, based on the embodiment corresponding to fig. 6, the above-mentioned S901 to S905 will be described by taking the first area indicated by the first parameter as the rectangular frame 1 as an example.
Illustratively, the handset 100 operates according to the equation s 1 ＝(x 12 -x 11 )(y 12 -y 11 ) Determining the area s of the rectangular frame 1 1 . The handset 100 traverses the second parameter in the second set of parameters (parameter 2, parameter 3, and parameter 4). When the mobile phone 100 traverses the parameter 2, the equation s is based 2 ＝(x 22 -x 21 )(y 22 -y 21 ) Calculating the area s of the rectangular frame 2 2 . In addition, the mobile phone 100 can also use the formula x k1 ＝max(x 11 ,x 21 ),x k2 ＝min(x 12 ,x 22 )； y k1 ＝max(y 11 ,y 21 ),y k2 ＝min(y 12 ,y 22 ) Determining coordinates of two end points on a diagonal of the target area as { (x) k1 ，y k1 )；(x k2 ，y k2 ) }. Further, the mobile phone 100 can calculate the equation s k ＝(x k2 -x k1 )(y k2 -y k1 ) Determining the area s of the target region (i.e., the overlapping region of the rectangular frame 1 and the rectangular frame 2) k . Further, the mobile phone 100 calculates the equation
The intersection ratio of the area of the rectangular frame 1 and the area of the rectangular frame 2 is calculated. Further, the handset 100 determines that the remaining second parameters exist in the second parameter set. In this way, the mobile phone 100 may traverse to the parameter 3, and calculate the intersection ratio of the area of the rectangular frame 1 and the area of the rectangular frame 3 based on the same manner as described above. The above-mentioned steps are repeated until the intersection ratio of the area of the rectangular frame 1 and the area of the rectangular frame 4 is calculated.
It is to be understood that the above embodiments are illustrated with the designated type of target object as the character. In addition, the target objects of the designated type may also be animals (such as horses, swans, cats, dogs, etc.), plants (such as flowers, trees), etc., and are not limited herein.
It should be noted that the type of the target object may be customized by the user in the function setting of the "camera" application. For example, as shown in fig. 10 (a), if the user sets the target object of the designated type as a swan, when the mobile phone 100 displays the first interface 103, the swan a and the swan b in the first interface 103 may be detected by using a target detection model trained based on a swan picture as a training sample. Further, as shown in fig. 10 (b), the mobile phone 100 may mark the swans a and b in the first interface 103 based on the same principle as in S502-S505. Similarly, there are neither redundant markers nor some swans that are not marked in fig. 10.
For another example, if the user sets the target objects of the designated type as all types of objects in the first interface 103, each target object in the first interface 103 may be detected by using a target detection model trained based on each different type of object picture as a training sample. As shown in fig. 11, the objects included in the first interface 103 include swans, flowers, and the sun, and the mobile phone 100 can mark swans, flowers, and the sun in the first interface 103 based on the same principle as in S502 to S505. Similarly, there is neither redundant marking nor a situation where there are objects that are not marked in fig. 11.
It should be noted that, in the above-described embodiment, the target object of the specified type in the shooting preview interface of the mobile phone 100 marking "camera" application is taken as an example for explanation. In other embodiments, the mobile phone 100 may further mark a target object in the image display interface of the cropping application, a target object in the video playing interface of the video playing application, and the like based on the same manner as described above, which is not limited herein. Therefore, the target object marking method provided by the embodiment of the application is wide in application range.
In addition, in the above embodiment, the mobile phone 100 performs frame selection marking on a target object of a specified type in an image by using a rectangular frame. In addition, the mobile phone 100 may also perform framing marking on the target object of the specified type in the image by using a mark such as a circular frame, an oval frame, and the like, which is not limited herein. Wherein, the rectangular box, the circular box or the oval box can mark the area which any parameter in the first parameter set is used for indication in a striking way.
Referring to fig. 12, the present application further provides a target object marking apparatus 1200, it should be noted that the basic principle and the technical effects of the target object marking apparatus 1200 provided in the embodiment of the present application are the same as those of the above embodiment, and for the sake of brief description, corresponding contents in the above embodiment may be referred to where not mentioned in the embodiment of the present application. As shown in fig. 12, the target object marking apparatus 1200 provided by the present application includes a display unit 1201 and a processing unit 1202, where the display unit 1201 is configured to display a first interface, and the first interface includes an acquired image. The processing unit 1202 is configured to detect a target object of a specified type in the image, and obtain a first parameter set, where any parameter in the first parameter set is used to indicate a region where the target object is located. The processing unit 1202 is further configured to determine, from the first parameter set, a first parameter that satisfies a preset condition, and determine a second parameter set other than the first parameter. The processing unit 1202 is further configured to determine an area of a first region indicated by the first parameter, and determine an intersection ratio of areas of second regions indicated by any one of the second parameters in the second parameter set, where determining the intersection ratio includes: the target region is an overlapping region of the first region and any second region, and the first region indicated by the first parameter and the second region indicated by any second parameter are respectively overlapped with regions indicated by the rest parameters in the first parameter set. The display unit 1201 is further configured to mark, on the first interface, a target object in a first area indicated by the first parameter, and mark a target object in a second area indicated by a second parameter, where an intersection ratio of the target object and the first area is smaller than a preset threshold.
In an alternative embodiment, the union ratio is the ratio of the intersection parameter and the union parameter. Wherein, the basis for determining the intersection parameter comprises: the sum of the overlapping areas of the target region and the region indicated by any one of the parameters in the first parameter set, and the determining the basis of the union parameter comprises: and the first region indicated by the first parameter and the second region indicated by any second parameter are respectively the sum of the overlapping areas of the regions indicated by the rest parameters in the first parameter set.
In an alternative embodiment, the cross-over ratio satisfies the equation
Wherein cover _ IoU i,j For a cross-over ratio, is selected>
Is the intersection parameter->
As a union of the parameters, s i Area of the first region, s, indicated for the first parameter j Area of second region, s, indicated for any second parameter k Is the area of the target region, S i∩t Is the sum of the overlapping areas of the first region and the regions indicated by the parameters in the first parameter set except the first parameter and any second parameter, S j∩t Is the sum of the overlapping areas of the second regions and the regions indicated by the parameters in the first parameter set except the first parameter and any second parameter, S k∩t The sum of the overlapping areas of the target region and the regions indicated by the parameters in the first parameter set except the first parameter and any second parameter is shown, and i, j, t and N are positive integers.
Further, the processing unit 1202 is specifically configured to determine an area of the first region indicated by the first parameter; traversing second parameters in the second parameter set; determining the area of the overlapping region of the first region indicated by the first parameter and the second region indicated by the traversed second parameter, and calculating the area of the second region indicated by the traversed second parameter; determining the area of a first region indicated by the first parameter, and intersecting and comparing the area of a second region indicated by the traversed second parameter; judging whether the second parameter set has the remaining second parameters; and if the residual second parameters exist, continuously traversing the second parameters in the second parameter set until the residual second parameters do not exist in the second parameter set.
In an alternative embodiment, the area indicated by any one of the first parameter set is a rectangular frame, a circular frame or an oval frame for framing the target object.
In an optional embodiment, the area indicated by any parameter in the first parameter set is a rectangular frame for the framing target object, and any parameter in the first parameter set is two end point coordinates of a diagonal line of the rectangular frame for the framing target object. The area of the rectangular frame indicated by the first parameter satisfies s i ＝(x i2 -x i1 )(y i2 -y i1 ) Wherein s is i Area of rectangular box indicated for first parameter, (x) i1 ，y i1 ) And (x) i2 ，y i2 ) Two end point coordinates of a diagonal line of the rectangular frame indicated by the first parameter, respectively. The area of the rectangular frame indicated by the second parameter satisfies s j ＝(x j2 -x j1 )(y j2 -y j1 ) Wherein s is j Area of rectangular box indicated for the second parameter, (x) j1 ，y j1 ) And (x) j2 ，y j2 ) Two end point coordinates of a diagonal line of the rectangular frame indicated by the second parameter, respectively.
In an optional embodiment, the parameter with the highest confidence level in the first parameter set, or the parameter with the confidence level greater than the preset confidence level, is selected.
In an alternative embodiment, the target object of the specified type is a person in the first interface, or all types of objects in the first interface.
In an optional implementation manner, the first interface is a photographing preview interface of a camera application, an image display interface of a cropping application, or a video playing interface of a video playing application.
Fig. 13 is a schematic diagram of a hardware structure of a terminal device according to an embodiment of the present disclosure, and as shown in fig. 13, the terminal device includes a processor 1301, a communication line 1304, and at least one communication interface (an exemplary case of the communication interface 1303 in fig. 13 is described as an example).
The processor 1301 may be a general processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to control the execution of programs according to the present disclosure.
The communication lines 1304 may include circuitry to communicate information between the above-described components.
Possibly, the terminal device may further comprise a memory 1302.
The memory 1302 may be separate and coupled to the processor via a communication link 1304. Memory 1302 may also be integrated with processor 1301.
The memory 1302 is used for storing computer-executable instructions for executing the present invention, and is controlled by the processor 1301 to execute the instructions. The processor 1301 is configured to execute the computer-executable instructions stored in the memory 1302, so as to implement the target object marking method provided by the embodiment of the present application.
Possibly, the computer-executable instructions in the embodiments of the present application may also be referred to as application program codes, which are not specifically limited in the embodiments of the present application.
In particular implementations, processor 1301 may include one or more CPUs, such as CPU0 and CPU1 in fig. 13, as one embodiment.
In particular implementations, terminal device may include multiple processors, such as processor 1301 and processor 1305 in fig. 13, for example. Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (e.g., computer program instructions).
Exemplarily, fig. 14 is a schematic structural diagram of a chip provided in an embodiment of the present application. Chip 140 includes one or more (including two) processors 1410 and a communication interface 1430.
In some embodiments, memory 1440 stores the following elements: an executable module or a data structure, or a subset thereof, or an expanded set thereof.
In an embodiment of the present application, the memory 1440 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1410. A portion of the memory 1440 may also include non-volatile random access memory (NVRAM).
In the embodiment of the present application, processor 1410 Communication interface 1430 and memory 1440 are coupled together by bus system 1420. The bus system 1420 may include a power bus, a control bus, a status signal bus, and the like, in addition to the data bus. For ease of description, the various buses are labeled in FIG. 14 as bus system 1420.
The method described in the embodiments of the present application may be applied to the processor 1410, or implemented by the processor 1410. Processor 1410 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 1410. The processor 1410 may be a general-purpose processor (e.g., a microprocessor or a conventional processor), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate, transistor logic device or discrete hardware component, and the processor 1410 may implement or execute the methods, steps and logic blocks disclosed in the embodiments of the present application.
The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium mature in the field, such as a random access memory, a read only memory, a programmable read only memory, or a charged erasable programmable memory (EEPROM). The storage medium is located in the memory 1440, and the processor 1410 reads the information in the memory 1440 and performs the steps of the above method in combination with the hardware.
In the above embodiments, the instructions stored by the memory for execution by the processor may be implemented in the form of a computer program product. The computer program product may be written in the memory in advance, or may be downloaded in the form of software and installed in the memory.
The computer program product includes one or more computer instructions. The procedures or functions according to the embodiments of the present application are all or partially generated when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. Computer instructions may be stored in or transmitted from one computer-readable storage medium to another, e.g., from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optics, digital Subscriber Line (DSL), or wireless (e.g., infrared, wireless, microwave, etc.) computer-readable storage media may be any available media that a computer can store or a data storage device including one or more servers, data centers, etc. integrated with available media.
The embodiment of the application also provides a computer readable storage medium. The methods described in the above embodiments may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. Computer-readable media may include computer storage media and communication media, and may include any medium that can communicate a computer program from one place to another. A storage medium may be any target medium that can be accessed by a computer.
Combinations of the above should also be included within the scope of computer-readable media. The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A target object labeling method, the method comprising:
the method comprises the steps that terminal equipment displays a first interface, wherein the first interface comprises an acquired image;
the terminal equipment detects a target object of a specified type in the image to obtain a first parameter set, wherein any parameter in the first parameter set is used for indicating the area where the target object is located;
the terminal equipment determines a first parameter meeting a preset condition from a first parameter set, and determines a second parameter set except the first parameter;
the terminal device determines an area of a first region indicated by the first parameter, and respectively intersects and compares the area of a second region indicated by any one second parameter in the second parameter set, where determining the intersection and comparison includes: the target area is an overlapping area of the first area and any one of the second areas, and the first area indicated by the first parameter and the second area indicated by any one of the second parameters are respectively overlapped with areas indicated by the rest of the first parameters in the first parameter set;
the terminal equipment marks the target object of a first area indicated by the first parameter on the first interface, and marks the target object of a second area indicated by a second parameter, wherein the intersection ratio of the target object and the first area is smaller than a preset threshold value.
2. The method of claim 1, wherein the intersection ratio is a ratio of an intersection parameter and a union parameter, and wherein determining the basis for the intersection parameter comprises: the sum of the overlapping areas of the target region and the region indicated by any one of the first parameter set, and the determining the basis of the union parameter comprises: and the first region indicated by the first parameter and the second region indicated by any one of the second parameters are respectively the sum of the overlapping areas of the regions indicated by the rest parameters in the first parameter set.
3. The method of claim 2, wherein the cross-over ratio satisfies the equation
Wherein cover _ IoU i,j In order to obtain the said cross-over ratio,
for the intersection parameter, is>
Is the parameter of the union, s i Area, s, of the first region indicated for the first parameter j Area, s, of second region indicated for any one of said second parameters k Is the area of the target region, S i∩t The first region is respectively overlapped with the regions indicated by the parameters except the first parameter and any one of the second parameters in the first parameter setSum of stack areas, S j∩t Is the sum of the overlapping areas of the second region and the regions indicated by the parameters except the first parameter and any one of the second parameters in the first parameter set, S k∩t And i, j, t and N are positive integers, wherein the sum of the overlapping areas of the target region and the regions indicated by the parameters except the first parameter and any second parameter in the first parameter set is respectively.
4. The method of claim 1, wherein the terminal device determines an area of a first region indicated by the first parameter, and the area of the first region is respectively intersected and compared with an area of a second region indicated by any one of the second parameter sets, and the method comprises:
the terminal equipment determines the area of a first region indicated by the first parameter;
the terminal device traverses the second parameters in the second parameter set;
the terminal device determines the area of the overlapping area of the first area indicated by the first parameter and the traversed second area indicated by the second parameter, and calculates the area of the traversed second area indicated by the second parameter;
the terminal equipment determines the area of a first region indicated by the first parameter, and the area of a second region indicated by the second parameter is intersected and compared with the area of the second region indicated by the second parameter;
the terminal equipment judges whether the second parameter set has the residual second parameters or not;
if the remaining second parameters exist, continuously traversing the second parameters in the second parameter set until the remaining second parameters do not exist in the second parameter set.
5. The method of claim 1, wherein the area indicated by any one of the first set of parameters is a rectangular box, a circular box, or an oval box for framing the target object.
6. The method according to claim 5, wherein the area indicated by any parameter in the first parameter set is a rectangular frame for framing the target object, and any parameter in the first parameter set is two end point coordinates of a diagonal line of the rectangular frame for framing the target object;
the area of the rectangular frame indicated by the first parameter satisfies s i ＝(x i2 -x i1 )(y i2 -y i1 ) Wherein s is i The area of the rectangular box indicated for the first parameter, (x) i1 ，y i1 ) And (x) i2 ，y i2 ) Two end point coordinates of a diagonal line of the rectangular frame indicated by the first parameter respectively;
the area of the rectangular frame indicated by the second parameter satisfies s j ＝(x j2 -x j1 )(y j2 -y j1 ) Wherein s is j (x) the area of the rectangular box indicated for the second parameter j1 ，y j1 ) And (x) j2 ，y j2 ) Two end point coordinates of a diagonal line of the rectangular frame indicated by the second parameter, respectively.
7. The method according to claim 1, wherein when the terminal device obtains the first parameter set, the terminal device also obtains a confidence level of each parameter in the first parameter set, and the first parameter that satisfies a preset condition is: the parameter with the highest confidence coefficient in the first parameter set, or the parameter with the confidence coefficient larger than a preset confidence coefficient.
8. The method of any of claims 1-7, wherein the target objects of the specified type are people in the first interface or all types of objects in the first interface.
9. The method of any one of claims 1-7, wherein the first interface is a photo preview interface of a camera application, an image display interface of a cropping application, or a video playback interface of a video playback application.
10. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, causes the terminal device to perform the method according to any of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111471242.5A CN115035186B (en) | 2021-12-03 | 2021-12-03 | Target object marking method and terminal equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111471242.5A CN115035186B (en) | 2021-12-03 | 2021-12-03 | Target object marking method and terminal equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115035186A CN115035186A (en) | 2022-09-09 |
| CN115035186B true CN115035186B (en) | 2023-04-11 |
Family
ID=83117767
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111471242.5A Active CN115035186B (en) | 2021-12-03 | 2021-12-03 | Target object marking method and terminal equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115035186B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112052787A (en) * | 2020-09-03 | 2020-12-08 | 腾讯科技（深圳）有限公司 | Target detection method and device based on artificial intelligence and electronic equipment |
| CN112906502A (en) * | 2021-01-29 | 2021-06-04 | 北京百度网讯科技有限公司 | Training method, device and equipment of target detection model and storage medium |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108875577A (en) * | 2018-05-11 | 2018-11-23 | 深圳市易成自动驾驶技术有限公司 | Object detection method, device and computer readable storage medium |
| CN109145931B (en) * | 2018-09-03 | 2019-11-05 | 百度在线网络技术（北京）有限公司 | Object detecting method, device and storage medium |
| CN111178119A (en) * | 2018-11-13 | 2020-05-19 | 北京市商汤科技开发有限公司 | Intersection state detection method and device, electronic equipment and vehicle |
| CN109684920B (en) * | 2018-11-19 | 2020-12-11 | 腾讯科技（深圳）有限公司 | Object key point positioning method, image processing method, device and storage medium |
| CN110503095B (en) * | 2019-08-27 | 2022-06-03 | 中国人民公安大学 | Positioning quality evaluation method, positioning method and device of target detection model |
| CN110659600B (en) * | 2019-09-19 | 2022-04-29 | 北京百度网讯科技有限公司 | Object detection method, device and equipment |
| CN112689842B (en) * | 2020-03-26 | 2022-04-29 | 华为技术有限公司 | Target detection method and device |
| CN111797829A (en) * | 2020-06-24 | 2020-10-20 | 浙江大华技术股份有限公司 | License plate detection method and device, electronic equipment and storage medium |
| CN113705643B (en) * | 2021-08-17 | 2022-10-28 | 荣耀终端有限公司 | Target detection method and device and electronic equipment |
-
2021
- 2021-12-03 CN CN202111471242.5A patent/CN115035186B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112052787A (en) * | 2020-09-03 | 2020-12-08 | 腾讯科技（深圳）有限公司 | Target detection method and device based on artificial intelligence and electronic equipment |
| CN112906502A (en) * | 2021-01-29 | 2021-06-04 | 北京百度网讯科技有限公司 | Training method, device and equipment of target detection model and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115035186A (en) | 2022-09-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11276183B2 (en) | Relocalization method and apparatus in camera pose tracking process, device, and storage medium | |
| WO2021013230A1 (en) | Robot control method, robot, terminal, server, and control system | |
| CN108989678B (en) | Image processing method and mobile terminal | |
| US9135678B2 (en) | Methods and apparatus for interfacing panoramic image stitching with post-processors | |
| CN112232279B (en) | Personnel interval detection method and device | |
| CN110059686B (en) | Character recognition method, device, equipment and readable storage medium | |
| EP3364646A1 (en) | Electronic device and method for displaying 360-degree image in the electronic device | |
| CN111050077B (en) | Shooting method and electronic equipment | |
| JP2018033365A (en) | Information processing device, information processing method, and program | |
| CN112434546A (en) | Face living body detection method and device, equipment and storage medium | |
| CN111897525A (en) | Big data processing method and system | |
| CN103327246B (en) | A kind of multimedia shooting processing method, device and intelligent terminal | |
| CN112749362A (en) | Control creating method, device, equipment and storage medium | |
| JPWO2021222570A5 (en) | ||
| WO2022095640A1 (en) | Method for reconstructing tree-shaped tissue in image, and device and storage medium | |
| CN114782296A (en) | Image fusion method, device and storage medium | |
| CN115035186B (en) | Target object marking method and terminal equipment | |
| CN111131706B (en) | Video picture processing method and electronic equipment | |
| CN108712604B (en) | Panoramic shooting method and mobile terminal | |
| CN111050071A (en) | Photographing method and electronic equipment | |
| CN108132817B (en) | Object management method and device | |
| CN114170366B (en) | Three-dimensional reconstruction method based on dotted line feature fusion and electronic equipment | |
| CN109377460A (en) | A kind of image processing method, image processing apparatus and terminal device | |
| WO2022000856A1 (en) | Speed measurement method and apparatus, electronic device, and storage medium | |
| CN111294518B (en) | Portrait composition limb truncation detection method, device, terminal and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |