CN108876858A - Method and apparatus for handling image - Google Patents

Method and apparatus for handling image Download PDF

Info

Publication number
CN108876858A
CN108876858A CN201810734682.7A CN201810734682A CN108876858A CN 108876858 A CN108876858 A CN 108876858A CN 201810734682 A CN201810734682 A CN 201810734682A CN 108876858 A CN108876858 A CN 108876858A
Authority
CN
China
Prior art keywords
image
key point
location information
shooting image
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201810734682.7A
Other languages
Chinese (zh)
Inventor
徐珍琦
王长虎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing ByteDance Network Technology Co Ltd
Original Assignee
Beijing ByteDance Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing ByteDance Network Technology Co Ltd filed Critical Beijing ByteDance Network Technology Co Ltd
Priority to CN201810734682.7A priority Critical patent/CN108876858A/en
Publication of CN108876858A publication Critical patent/CN108876858A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras
    • G06T7/73Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/75Determining position or orientation of objects or cameras using feature-based methods involving models
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras
    • G06T7/73Determining position or orientation of objects or cameras using feature-based methods

Abstract

The embodiment of the present application discloses the method and apparatus for handling image.One specific embodiment of this method includes:It obtains the shooting image of target object and shows the benchmark image of target object;Image will be shot and be input to training in advance, corresponding with target object critical point detection model, obtain location information set, location information includes position coordinates, and position coordinates indicate the corresponding position in shooting image of the key point of the key point information instruction in key point information set;For the key point information in key point information set, determine that the coordinate of the corresponding position in benchmark image of the key point of key point information instruction as reference coordinate, obtains reference coordinate set;According to obtained location information set and reference coordinate set, the corresponding position in benchmark image of object region in shooting image is determined using Image geometry transform.The embodiment utilizes the transformation between the shooting image and benchmark image of critical point detection model realization target object.

Description

Method and apparatus for handling image
Technical field
The invention relates to field of computer technology, and in particular to the method and apparatus for handling image.
Background technique
In the prior art, the tracking and detection to target object are usually realized using the shooting video of target object.One As, using image detection and identification technology, the corresponding shooting image of key frame in shooting video is detected and is identified, To realize the tracking to target object.
Summary of the invention
The embodiment of the present application proposes the method and apparatus for handling image.
In a first aspect, the embodiment of the present application provides a kind of method for handling image, this method includes:Obtain target The shooting image of object and the benchmark image for showing target object;Image will be shot and be input to train in advance and target pair As corresponding critical point detection model, location information set is obtained, wherein location information includes position coordinates, and position coordinates are used In the corresponding position in shooting image of key point for indicating the key point information instruction in key point information set, key point is Point predetermined, on target object;For the key point information in key point information set, determine that the key point information refers to The coordinate of the corresponding position in benchmark image of the key point shown obtains reference coordinate set as reference coordinate;According to obtaining Location information set and reference coordinate set, utilize Image geometry transform to determine that the object region in shooting image is corresponding Position in benchmark image.
In some embodiments, location information further includes visibility information, wherein visibility information is for indicating key point The key point of key point information instruction in information aggregate is shown in the probability in shooting image.
In some embodiments, according to obtained location information set and reference coordinate set, Image geometry transform is utilized Determine the corresponding position in benchmark image of object region in shooting image, including:From obtained location information set The middle location information chosen the visibility information for including and be greater than preset visibility threshold, obtains sub- location information set;According to The position coordinates that the obtained location information in sub- location information set includes and the position in obtained sub- location information set The reference coordinate of the corresponding key point of information determines that the object region in shooting image is corresponding in benchmark using perspective transform Position in image.
In some embodiments, the above method further includes:Target location in benchmark image is labeled, wherein Target position is the corresponding position in benchmark image of object region shot in image.
In some embodiments, training obtains critical point detection model as follows:Training sample set is obtained, In, training sample includes the shooting image location information set corresponding with shooting image of target object;Utilize machine learning Method, using the shooting image of the target object in the training sample in training sample set as input, by the shooting with input The corresponding position information set cooperation of image is desired output, and training obtains critical point detection model.
In some embodiments, it is seen that property information indicates with the numerical value between zero and one, including zero and one.
Second aspect, the embodiment of the present application provide a kind of for handling the device of image, which includes:It obtains single Member is configured to obtain the shooting image of target object and shows the benchmark image of target object;Processing unit is configured to Image will be shot and be input to training in advance, corresponding with target object critical point detection model, obtain location information set, In, location information includes position coordinates, and position coordinates are used to indicate the pass of the key point information instruction in key point information set The corresponding position in shooting image of key point, key point is predetermined, the point on target object;Determination unit is configured At for the key point information in key point information set, determine that the key point of key point information instruction is corresponding in benchmark image In position coordinate as reference coordinate, obtain reference coordinate set;Above-mentioned determination unit is further configured to basis and obtains The location information set and reference coordinate set arrived determines the object region pair in shooting image using Image geometry transform It should be in the position in benchmark image.
In some embodiments, location information further includes visibility information, wherein visibility information is for indicating key point The key point of key point information instruction in information aggregate is shown in the probability in shooting image.
In some embodiments, above-mentioned determination unit is not configured to further:It is chosen from obtained location information set Including visibility information be greater than preset visibility threshold location information, obtain sub- location information set;According to what is obtained The position coordinates that location information in sub- location information set includes and the location information pair in obtained sub- location information set The reference coordinate for the key point answered determines that the object region in shooting image is corresponding in benchmark image using perspective transform Position.
In some embodiments, above-mentioned apparatus further includes:Unit is marked, the target position in benchmark image is configured to Place is labeled, wherein target position is the corresponding position in benchmark image of object region shot in image.
In some embodiments, training obtains critical point detection model as follows:Training sample set is obtained, In, training sample includes the shooting image location information set corresponding with shooting image of target object;Utilize machine learning Method, using the shooting image of the target object in the training sample in training sample set as input, by the shooting with input The corresponding position information set cooperation of image is desired output, and training obtains critical point detection model.
In some embodiments, it is seen that property information indicates with the numerical value between zero and one, including zero and one.
The third aspect, the embodiment of the present application provide a kind of electronic equipment, which includes:One or more processing Device;Storage device, for storing one or more programs;When one or more programs are executed by one or more processors, make Obtain method of the one or more processors realization as described in implementation any in first aspect.
Fourth aspect, the embodiment of the present application provide a kind of computer-readable medium, are stored thereon with computer program, should The method as described in implementation any in first aspect is realized when computer program is executed by processor.
Method and apparatus provided by the embodiments of the present application for handling image, by the shooting figure for obtaining target object Picture will shoot image and be input to critical point detection model train in advance, corresponding to target object, obtains position information set It closes, wherein location information includes position coordinates, and position coordinates are used to indicate the key point information instruction in key point information set The corresponding position in shooting image of key point, key point is predetermined the point on target object;Believe for key point Key point information in breath set determines the coordinate of the corresponding position in benchmark image of the key point of key point information instruction As reference coordinate, reference coordinate set is obtained;It is several using image according to obtained location information set and reference coordinate set What converts the corresponding position in benchmark image of object region determined in shooting image, utilizes key point to realize Detection model handles the shooting image of target object, corresponding in its shooting image to obtain the key point on target object In position utilize image geometry and according to the corresponding position in the benchmark image comprising target object of each key point Transformation obtains the corresponding position in benchmark image of object region in shooting image.
Detailed description of the invention
By reading a detailed description of non-restrictive embodiments in the light of the attached drawings below, the application's is other Feature, objects and advantages will become more apparent upon:
Fig. 1 is that one embodiment of the application can be applied to exemplary system architecture figure therein;
Fig. 2 is the flow chart of one embodiment of the method for handling image of the application;
Fig. 3 is the flow chart according to another embodiment of the method for handling image of the application;
Fig. 4 is the schematic diagram according to an application scenarios of the method for handling image of the application;
Fig. 5 is the structural schematic diagram according to one embodiment of the device for handling image of the application;
Fig. 6 is adapted for the structural schematic diagram for the computer system for realizing the electronic equipment of the embodiment of the present application.
Specific embodiment
The application is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched The specific embodiment stated is used only for explaining related invention, rather than the restriction to the invention.It also should be noted that in order to Convenient for description, part relevant to related invention is illustrated only in attached drawing.
It should be noted that in the absence of conflict, the features in the embodiments and the embodiments of the present application can phase Mutually combination.The application is described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
Fig. 1 is shown can be using the application for handling the method for image or the example of the device for handling image Property framework 100.
As shown in Figure 1, system architecture 100 may include terminal device 101,102,103, network 104 and server 105. Network 104 between terminal device 101,102,103 and server 105 to provide the medium of communication link.Network 104 can be with Including various connection types, such as wired, wireless communication link or fiber optic cables etc..
Terminal device 101,102,103 is interacted by network 104 with server 105, to receive or send message etc..Terminal Various client applications, such as web browser applications, image processing class application can be installed in equipment 101,102,103 Deng.
Terminal device 101,102,103 can be hardware, be also possible to software.When terminal device 101,102,103 is hard When part, it can be the various electronic equipments for supporting processing image, including but not limited to smart phone, tablet computer, e-book is read Read device, pocket computer on knee and desktop computer etc..When terminal device 101,102,103 is software, can install In above-mentioned cited electronic equipment.Multiple softwares or software module may be implemented into (such as providing distributed clothes in it Business), single software or software module also may be implemented into.It is not specifically limited herein.
Server 105 can be to provide the server of various services, for example, figure of the transmission of terminal device 101,102,103 As the image processing server handled.Image processing server can be handled the image received, and at generation Reason is as a result, further, can also feed back to terminal device for processing result.
It should be noted that the local in server 105 can also directly store image, at this point, server 105 can be straight It connects and extracts the local image stored and handled, at this point it is possible to which terminal device 101,102,103 and network 104 is not present.
It should be noted that the method provided by the embodiment of the present application for handling image is generally held by server 105 Row, correspondingly, the device for handling image is generally positioned in server 105.
It may also be noted that can also be equipped with image processing class application in terminal device 101,102,103, terminal is set Standby 101,102,103 can also be based on image processing class using handling facial image, at this point, the side for handling image Method can also be executed by terminal device 101,102,103, and correspondingly, the device for handling image also can be set to be set in terminal In standby 101,102,103.At this point, server 105 and network 104 can be not present in exemplary system architecture 100.
It should be noted that server can be hardware, it is also possible to software.When server is hardware, may be implemented At the distributed server cluster that multiple servers form, individual server also may be implemented into.It, can when server is software It, can also be with to be implemented as multiple softwares or software module (such as providing multiple softwares of Distributed Services or software module) It is implemented as single software or software module.It is not specifically limited herein.
It should be understood that the number of terminal device, network and server in Fig. 1 is only schematical.According to realization need It wants, can have any number of terminal device, network and server.
With continued reference to Fig. 2, the process of one embodiment of the method for handling image according to the application is shown 200.The method for being used to handle image includes the following steps:
Step 201, it obtains the shooting image of target object and shows the benchmark image of target object.
It in the present embodiment, can be by wired for handling the executing subject (server 105 as shown in Figure 1) of image Or target object wirelessly is obtained from local or other storage equipment (terminal device 101,102,103 as shown in Figure 1) Shooting image and show the benchmark image of target object.
Wherein, object can refer to arbitrary entity.For example, object may include eat, wear, living, going, with etc. it is relevant All kinds of articles.Such as vehicle, court etc..Object also may include personage, animal, plant etc..It should be noted that the object is to show Meaning property, the application is not limited to this, and the entity that can arbitrarily shoot is each fallen in the protection scope of the application.
Target object can be with preassigned any object.Target object is also possible to according to preset condition and pair of determination As.For example, preset condition is object taken by certain capture apparatus in preset time period.So in above-mentioned preset time period Object taken by certain capture apparatus is target object.
It is alternatively possible to first obtain the shooting video of target object, the image for any frame extracted from shooting video is all It can be used as the shooting image of target object.Benchmark image can be the shooting image of preassigned target object, can also be with It is the drawing image of target object, such as the image for the target object drawn out using various image rendering softwares.Reference map Setting-out figure as being also possible to target object.
Optionally, the overall picture of target object can be shown in benchmark image.For example, for football pitch, reference map As the general image in the football pitch that can be shot from the overhead of the geometric center in football pitch, it is also possible to the flat of the football pitch drawn Face dimensional drawing.
Step 202, image will be shot and is input to training in advance, corresponding with target object critical point detection model, obtained To location information set.
In the present embodiment, the shooting image that above-mentioned steps 201 can be obtained is input to training in advance, corresponds to mesh The critical point detection model for marking object, obtains location information set.Wherein, location information can be in key point information set Key point information instruction the corresponding relevant information in position in shooting image of key point.Location information may include position Coordinate.Position coordinates can be used to indicate that the key point of the key point information instruction in key point information set is corresponding in shooting figure Position as in.
Wherein, key point can be predetermined the point on target object.Key point information can be and key point phase Information closing, can indicating a key point.For example, can be carried out from 1 to 10 for 10 key points on target object Number, and number the key point information that can be used as indicating key point.Generally, key point can be stored in advance and key point is believed The corresponding relationship of breath.
In some cases, it can be possible to there is the case where shooting image has only taken Partial key point.For example, shooting image The half of target object is only taken, and target object is not photographed is also preset with key point on the other half.For example, shooting Image has only taken the side of target object, and is also preset with key point on the side of target object not being photographed.
Optionally, the key point on each target object corresponds to a location information, i.e., the key point not being photographed Location information can also be corresponding with.At this point, if there is N number of preset key point on target object, then can be obtained for shooting image To the location information set comprising N number of location information.Specifically, it is not photographed the corresponding position in shooting image of key point Coordinate can be used uniformly special coordinates such as (- 1, -1) to indicate, the key point not taken can also be corresponded to The position in the coordinate system in plane where shooting image is as the corresponding position coordinates of key point not being photographed.
Optionally, the corresponding position of key point that shooting image takes can also be only included in the location information set obtained Confidence breath.At this point, if there is N number of preset key point on target object, wherein shooting image has taken M key point, then It can only include the M corresponding position coordinates of key point being photographed in obtained location information set.At this point, position Confidence breath can also include key point information, it can obtain the key point letter of each key point taken by each shooting image Breath and position coordinates.
In the present embodiment, critical point detection model is usually corresponding with target object, that is, is directed to different target objects, can The corresponding critical point detection model of different target objects is respectively trained.Wherein, critical point detection model can be used for Characterize the correspondence of the information of the shooting image position in shooting image corresponding with the key point on target object of target object Relationship.Specifically, various methods be can use and obtain above-mentioned critical point detection model.
It is alternatively possible to obtain critical point detection model as follows:
The first step, a large amount of shooting images of all angles of available target object.
Second step obtains the corresponding location information set of each shooting image, i.e., preset each key on target object The relevant information of the corresponding position in shooting image of point.When location information includes position coordinates, then for each shooting figure Picture, the corresponding position in shooting image of preset each key point on available target object.
Third step passes through a large amount of shooting image and each shooting image pair to the above-mentioned first step and second step acquisition The statistical analysis for the location information set answered establishes pair for being stored with shooting image location information set corresponding with image is shot Relation table is answered, and using the mapping table of foundation as above-mentioned critical point detection model.
It later, can be by the shooting image and above-mentioned corresponding relationship in a shooting image for getting target object Each shooting image in table is successively compared, will be consistent with shooting image in above-mentioned mapping table or with shooting image Position information set cooperation corresponding to the highest shooting image of similarity is the corresponding position of shooting image of the target object obtained Set information aggregate.
It is alternatively possible to obtain critical point detection model using the method training of machine learning as follows:
The first step obtains training sample set.Wherein, training sample may include shooting image and the shooting of target object The corresponding location information set of image.
Second step, using the method for machine learning, using the shooting image in the training sample in training sample set as The corresponding position information set cooperation of shooting image with input is desired output by input, and training obtains critical point detection model.
In above-mentioned second step, specifically, it can train as follows and obtain critical point detection model:
Firstly, obtaining initial key point detection model.Wherein, initial key point detection model can be it is various types of not Artificial neural network trained or that training is not completed or to a variety of indisciplines or training complete for handling image (such as Image detection, image characteristics extraction etc.) the model that is combined of artificial neural network.For example, initial key point detects Model can be deep learning model, convolutional neural networks for handling image etc..Initial key point detection model can also be with It is the trained various types of for handling the artificial nerve network model of image of existing open source.The inspection of initial key point It surveys model and is also possible to the neural network model constructed using existing neural network API according to actual application demand.
Then, using the shooting image in the training sample in training sample set as above-mentioned initial key point detection model Input, based on initial key point detection model output and preset loss function difference degree, calculated using backpropagation Method etc. adjusts the parameter of each layer of initial key point detection model, until meeting certain default termination condition, it is determined that obtain Critical point detection model.Wherein, default termination condition can be in training process, the position of initial key point detection model output The difference of location information set corresponding to information aggregate and the shooting image of corresponding input is less than certain threshold value.
Step 203, for the key point information in key point information set, the key point of key point information instruction is determined The coordinate of the corresponding position in benchmark image obtains reference coordinate set as reference coordinate.
In the present embodiment, it can determine that the coordinate of the corresponding position in benchmark image of each key point is sat as benchmark Mark, to obtain reference coordinate set.
It is alternatively possible to manually mark out the coordinate of the corresponding position in benchmark image of each key point in advance.At this point, can To obtain each reference coordinate marked in advance, to obtain reference coordinate set.
It is alternatively possible to which benchmark image is input to above-mentioned critical point detection model, the corresponding position of benchmark image is obtained Information aggregate, at this point, the position coordinates that the location information in obtained location information set includes are reference coordinate.
Step 204, it according to obtained location information set and reference coordinate set, is determined and is shot using Image geometry transform The corresponding position in benchmark image of object region in image.
In the present embodiment, it can first be sat according to the position that each position information in obtained location information set includes The reference coordinate of mark and the corresponding key point of each position information, it is (such as affine using existing various Image geometry transform methods Transformation, perspective transform, complex transformation etc.) obtain shooting image and the transformation relation between benchmark image.So as to determination Shoot the corresponding position in benchmark image of object region in image.
Wherein, object region can be the arbitrary image region in preassigned shooting image.Object-image region Domain is also possible to the image-region in the shooting image filtered out according to certain screening conditions.For example, screening conditions can be Average pixel value in image-region is greater than presetted pixel threshold value.In another example screening conditions can be to show specified object Image-region, at this point, the image that object region can include by the external contact zone of the image of object specified in image The image-region that the edge line of region or specified object is included.
For example, in the shooting image of certain match in a court, object region can be to show ginseng Add the image-region of certain sportsman of match, or show the image-region of football.Specifically, it can use existing figure As detection and recognition methods detect the position of certain sportsman or football in shooting image.
Wherein it is possible to determine then position of the object region in shooting image utilizes shooting image and base first Transformation relation between quasi- image obtains position of the object region in benchmark image.Specifically, in object region Shape be rule figure when, can using the geometric center point of object region shooting image in position as target Position of the image-region in shooting image.It, can be by target figure when the shape of object region is irregular figure The average value of position of each pixel for including as region in shooting image is as object region in shooting image Position.
For example, three location informations can be chosen from location information set, and according to these three location information packets The reference coordinate of the position coordinates included and the corresponding key point of these three location informations, determines shooting figure using affine transformation Picture and the spin matrix between benchmark image.Then, it in the position shot in image and is obtained according to object region Spin matrix, calculate the corresponding position in benchmark image of object region.
It should be noted that about the transformation relation solved using various Image geometry transform methods between two images Detailed process is the well-known technique studied and applied extensively at present, and details are not described herein.
The method provided by the above embodiment for handling image of the application, which realizes, utilizes critical point detection model pair The shooting image of target object is handled, to obtain the corresponding position in its shooting image of the key point on target object, And it according to the corresponding position in the benchmark image comprising target object of each key point, is clapped using Image geometry transform Take the photograph the corresponding position in benchmark image of object region in image.
With further reference to Fig. 3, it illustrates the processes 300 of another embodiment of the method for handling image.The use In the process 300 of the method for processing image, include the following steps:
Step 301, it obtains the shooting image of target object and shows the benchmark image of target object.
The specific implementation procedure of this step can refer to the related description of the step 201 in Fig. 2 corresponding embodiment, herein not It repeats again.
Step 302, image will be shot and is input to training in advance, corresponding with target object critical point detection model, obtained To location information set, wherein location information includes position coordinates and visibility information.
In the present embodiment, location information can also include visibility information.Wherein, it is seen that property information can be used to indicate that The key point of key point information instruction in key point information set is shown in the probability in shooting image.
In some optional implementations of the present embodiment, it is seen that property information indicates with the numerical value between zero and one, Including zero and one.Numerical value is bigger, and the probability that can indicate that corresponding key point is shown in shooting image is bigger.
The specific implementation procedure of this step can refer to the related description of the step 202 in Fig. 2 corresponding embodiment, herein not It repeats again.
Step 303, the visibility information for including is chosen from obtained location information set is greater than preset visibility threshold The location information of value obtains sub- location information set.
In the present embodiment, the visibility information that can include by each position information in obtained location information set point It is not compared with preset visibility threshold, then chooses the position letter that the visibility information for including is greater than visibility threshold Breath, obtains sub- location information set.Wherein, it is seen that property threshold value can be according to historical experience or according to critical point detection model The accuracy of the visibility information of output determines.
Step 304, the position coordinates for including according to the location information in obtained sub- location information set and obtained son The reference coordinate of the corresponding key point of location information in location information set determines the mesh in shooting image using perspective transform The corresponding position in benchmark image in logo image region.
In the present embodiment, the position coordinates that can include according to each position information in sub- location information set, and The reference coordinate of the corresponding key point of each position information determines that the target area in shooting image is corresponding in base using perspective transform Position in quasi- image.
It for example, can be first according to the size relation of visibility information, according to sequence from big to small by sub- position Each position information is ranked up in information aggregate, then, therefrom select including visibility information biggish first four position letter Breath.Later, the position coordinates for including according to four location informations selected key point corresponding with this four location informations Reference coordinate, determine shooting image and benchmark image between corresponding perspective transformation matrix.Then, become according to determining perspective The position of matrix and object region in shooting image is changed, it is corresponding in benchmark image to calculate object region Position.
It should be noted that being mesh about the detailed process of the transformation relation between two images is solved using perspective transform The well-known technique of preceding extensive research and application, details are not described herein.
Step 305, the target location in benchmark image is labeled.
In the present embodiment, it can be labeled in the target location in benchmark image.Wherein, target position can be Shoot the corresponding position in benchmark image of object region in image.Wherein, the form of mark can be text, figure The various forms such as case, hyperlink.
With continued reference to the signal that Fig. 4, Fig. 4 are according to the application scenarios of the method for handling image of the present embodiment Figure.In the application scenarios of Fig. 4,8 key points are preset on a football pitch 401.Each key point is successively compiled from 1 to 8 Number.The shooting image 403 of a wherein frame is chosen from the recorded video for the football match held on the football pitch.Separately Outside, a top view 402 in the football pitch is obtained as benchmark image.Image 403 will be shot and be input to training in advance, correspondence Critical point detection model 404 in the football pitch, obtains location information set 405.
Wherein, in location information set 405 include 8 key points respectively correspond shooting image 403 in position can Opinion property information and position coordinates.As shown in the figure, coordinate can be established as coordinate origin using a vertex for shooting image 403 System.Position coordinates can indicate the coordinate under the coordinate system of foundation.Wherein, the corresponding visibility of key point that number is 1 Information is V1, position coordinates C1, and the corresponding visibility information of the key point that number is 2 is V2, position coordinates C2, and number is The corresponding visibility information of 3 key point is V3, position coordinates C3, and the corresponding visibility information of the key point that number is 4 is V4, position coordinates C4, the corresponding visibility information of the key point that number is 5 are V5, position coordinates C5, the pass that number is 6 The corresponding visibility information of key point is V6, and position coordinates C6, the corresponding visibility information of the key point that number is 7 is V7, position Setting coordinate is C7, and the corresponding visibility information of the key point that number is 8 is V8, position coordinates C8.
Later, the corresponding visibility information of each key point can be compared with preset visibility threshold 406 respectively Compared with therefrom selecting corresponding visibility information greater than the key point of visibility threshold 406 includes numbering the key for being 1,2,5,6,7 Point.The corresponding location information of each key point selected is as shown in figure label 407.
Later, these key points that manually number that marks in advance is 1,2,5,6,7 can first be obtained in standard picture The coordinate of position in 402 is respectively C1`, C2`, C5`, C6`, C6`, as shown in figure label 408.It is then possible to according to choosing The position coordinates that the location information 407 of each key point out includes, and the position of each key point for selecting in standard picture Coordinate 408, using perspective transform solve shooting image 403 and standard picture 402 between transformation relation 409.
Next, can be according to the position and above-mentioned determination for shooting the image-region 410 in image 403 where football Transformation relation out solves the corresponding target position 411 in benchmark image 402 of image-region 410.Further, may be used also To mark out the position of the instruction of target position 411 in benchmark image 402.As shown in figure label 412, a band can be used The circle of shade412 come label target position.
It can carry out above-mentioned treatment process similarly, for each frame shooting image in recorded video, so as to The corresponding position in benchmark image of key point shown in each frame shooting image is successively marked in standard picture.In turn, It can determine the corresponding position in benchmark image of image-region where the football in each frame shooting image, and in benchmark It is labeled at corresponding position in image.This section recorded in recorded video can be found out as a result, during the games The position of football is mobile.Similarly, it can also determine that the image-region in each frame image where some sportsman is corresponding in benchmark Position in image is simultaneously labeled, thus obtain this section during the games, the position of running of the sportsman.It, can based on this Further to carry out data analysis to labeled data.
From figure 3, it can be seen that compared with the corresponding embodiment of Fig. 2, the process of the information-pushing method in the present embodiment 300 after obtaining the corresponding position in shooting image of each key point using critical point detection model, further according to each key Point determines the perspective transform relationship between shooting image and benchmark image in the position in benchmark image, so as to determine to clap The corresponding position in benchmark image of object region in image is taken the photograph, and the corresponding position in benchmark image is marked Note, so as to intuitively find out the mapping position of the object region in shooting figure picture in benchmark image, and in this base On plinth, certain data analysis can also be carried out.
With further reference to Fig. 5, as the realization to method shown in above-mentioned each figure, this application provides for handling image One embodiment of device, the Installation practice is corresponding with embodiment of the method shown in Fig. 2, which specifically can be applied to In various electronic equipments.
As shown in figure 5, the device 500 provided in this embodiment for handling image includes obtaining 501, processing unit 502 With determination unit 503.Wherein, acquiring unit 501 is configured to obtain the shooting image of target object and shows target object Benchmark image;Processing unit 502 is configured to shoot image and is input to training in advance, corresponding with target object key Point detection model, obtains location information set, wherein location information includes position coordinates, and position coordinates are for indicating key point The corresponding position in shooting image of the key point of key point information instruction in information aggregate, key point is predetermined, Point on target object;Determination unit 503 is configured to determine the key for the key point information in key point information set The coordinate of the corresponding position in benchmark image of key point of point information instruction obtains reference coordinate set as reference coordinate; Above-mentioned determination unit 503 is further configured to according to obtained location information set and reference coordinate set, several using image What converts the corresponding position in benchmark image of object region determined in shooting image.
In the present embodiment, in the device 500 for handling image:Acquiring unit 501, processing unit 502 and determining list The specific processing of member 503 and its brought technical effect can be respectively with reference to step 201, the steps 202 in Fig. 2 corresponding embodiment With the related description of step 203, details are not described herein.
In some optional implementations of the present embodiment, location information further includes visibility information, wherein visibility Information is used to indicate that the key point of the key point information instruction in key point information set to be shown in the probability in shooting image.
In some optional implementations of the present embodiment, above-mentioned determination unit 503 is not configured to further:From obtaining Location information set in choose include visibility information be greater than preset visibility threshold location information, obtain sub- position Information aggregate;The position coordinates for including according to the location information in obtained sub- location information set and obtained sub- location information The reference coordinate of the corresponding key point of location information in set determines the object-image region in shooting image using perspective transform The corresponding position in benchmark image in domain.
In some optional implementations of the present embodiment, above-mentioned apparatus 500 further includes:Mark unit (does not show in figure Out), the target location being configured in benchmark image is labeled, wherein target position is the target shot in image The corresponding position in benchmark image of image-region.
In some optional implementations of the present embodiment, training obtains critical point detection model as follows: Obtain training sample set, wherein training sample includes the shooting image location information corresponding with shooting image of target object Set;Using the method for machine learning, using the shooting image of the target object in the training sample in training sample set as The corresponding position information set cooperation of shooting image with input is desired output by input, and training obtains critical point detection model.
In some optional implementations of the present embodiment, it is seen that property information indicates with the numerical value between zero and one, Including zero and one.
The device provided by the above embodiment of the application obtains the shooting image of target object, processing by acquiring unit Unit will shoot image and be input to critical point detection model train in advance, corresponding to target object, obtain position information set It closes, wherein location information includes position coordinates, and position coordinates are used to indicate the key point information instruction in key point information set The corresponding position in shooting image of key point, key point is predetermined the point on target object, believes for key point Key point information in breath set, determination unit determine that the key point of key point information instruction corresponds to
The coordinate of position in benchmark image obtains reference coordinate set as reference coordinate, and according to obtained position Information aggregate and reference coordinate set determine that the object region in shooting image is corresponding in benchmark using Image geometry transform Position in image is handled to realize using shooting image of the critical point detection model to target object, to obtain The corresponding position shot at it in image of key point on target object, and target pair is being included according to each key point correspondence It is corresponding in reference map to obtain the object region in shooting image using Image geometry transform for position in the benchmark image of elephant Position as in.
Below with reference to Fig. 6, it illustrates the electronic equipment (end of example as shown in figure 1 for being suitable for being used to realize the embodiment of the present application End equipment or server) computer system 600 structural schematic diagram.Electronic equipment shown in Fig. 6 is only an example, no The function and use scope for coping with the embodiment of the present application bring any restrictions.
As shown in fig. 6, computer system 600 includes central processing unit (CPU) 601, it can be read-only according to being stored in Program in memory (ROM) 602 or be loaded into the program in random access storage device (RAM) 603 from storage section 608 and Execute various movements appropriate and processing.In RAM 603, also it is stored with system 600 and operates required various programs and data. CPU 601, ROM 602 and RAM 603 are connected with each other by bus 604.Input/output (I/O) interface 605 is also connected to always Line 604.
I/O interface 605 is connected to lower component:Importation 606 including keyboard, mouse etc.;It is penetrated including such as cathode The output par, c 607 of spool (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.;Storage section 608 including hard disk etc.; And the communications portion 609 of the network interface card including LAN card, modem etc..Communications portion 609 via such as because The network of spy's net executes communication process.Driver 610 is also connected to I/O interface 605 as needed.Detachable media 611, such as Disk, CD, magneto-optic disk, semiconductor memory etc. are mounted on as needed on driver 610, in order to read from thereon Computer program be mounted into storage section 608 as needed.
Particularly, in accordance with an embodiment of the present disclosure, it may be implemented as computer above with reference to the process of flow chart description Software program.For example, embodiment of the disclosure includes a kind of computer program product comprising be carried on computer-readable medium On computer program, which includes the program code for method shown in execution flow chart.In such reality It applies in example, which can be downloaded and installed from network by communications portion 609, and/or from detachable media 611 are mounted.When the computer program is executed by central processing unit (CPU) 601, limited in execution the present processes Above-mentioned function.
It should be noted that the computer-readable medium of the application can be computer-readable signal media or computer Readable storage medium storing program for executing either the two any combination.Computer readable storage medium for example can be --- but it is unlimited In system, device or the device of --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor, or any above combination.It calculates The more specific example of machine readable storage medium storing program for executing can include but is not limited to:It is electrical connection with one or more conducting wires, portable Formula computer disk, hard disk, random access storage device (RAM), read-only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device or The above-mentioned any appropriate combination of person.In this application, computer readable storage medium can be it is any include or storage program Tangible medium, which can be commanded execution system, device or device use or in connection.And in this Shen Please in, computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal, In carry computer-readable program code.The data-signal of this propagation can take various forms, including but not limited to Electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be computer-readable Any computer-readable medium other than storage medium, the computer-readable medium can send, propagate or transmit for by Instruction execution system, device or device use or program in connection.The journey for including on computer-readable medium Sequence code can transmit with any suitable medium, including but not limited to:Wirelessly, electric wire, optical cable, RF etc. or above-mentioned Any appropriate combination.
Flow chart and block diagram in attached drawing are illustrated according to the system of the various embodiments of the application, method and computer journey The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation A part of one module, program segment or code of table, a part of the module, program segment or code include one or more use The executable instruction of the logic function as defined in realizing.It should also be noted that in some implementations as replacements, being marked in box The function of note can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are actually It can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it to infuse Meaning, the combination of each box in block diagram and or flow chart and the box in block diagram and or flow chart can be with holding The dedicated hardware based system of functions or operations as defined in row is realized, or can use specialized hardware and computer instruction Combination realize.
Being described in unit involved in the embodiment of the present application can be realized by way of software, can also be by hard The mode of part is realized.Described unit also can be set in the processor, for example, can be described as:A kind of processor, packet Include acquiring unit, processing unit and determination unit.Wherein, the title of these units is not constituted under certain conditions to the unit The restriction of itself, for example, acquiring unit is also described as " obtaining the shooting image of target object and showing target object Benchmark image unit ".
As on the other hand, present invention also provides a kind of computer-readable medium, which be can be Included in device described in above-described embodiment;It is also possible to individualism, and without in the supplying device.Above-mentioned calculating Machine readable medium carries one or more program, when said one or multiple programs are executed by the device, so that should Device:It obtains the shooting image of target object and shows the benchmark image of target object;Shooting image is input to preparatory instruction Critical point detection model experienced, corresponding with target object, obtains location information set, wherein location information includes that position is sat Mark, position coordinates are used to indicate that the key point of the key point information instruction in key point information set to be corresponding in shooting image Position, key point is predetermined, the point on target object;For the key point information in key point information set, determine The coordinate of the corresponding position in benchmark image of the key point of key point information instruction obtains reference coordinate as reference coordinate Set;According to obtained location information set, determine that the object region in shooting image is corresponding using Image geometry transform Position in benchmark image.
Above description is only the preferred embodiment of the application and the explanation to institute's application technology principle.Those skilled in the art Member is it should be appreciated that invention scope involved in the application, however it is not limited to technology made of the specific combination of above-mentioned technical characteristic Scheme, while should also cover in the case where not departing from foregoing invention design, it is carried out by above-mentioned technical characteristic or its equivalent feature Any combination and the other technical solutions formed.Such as features described above has similar function with (but being not limited to) disclosed herein Can technical characteristic replaced mutually and the technical solution that is formed.

Claims (14)

1. a kind of method for handling image, including:
It obtains the shooting image of target object and shows the benchmark image of target object;
The shooting image is input to training in advance, corresponding with target object critical point detection model, is obtained in place Set information aggregate, wherein location information includes position coordinates, and position coordinates are used to indicate the key point in key point information set The corresponding position in the shooting image of the key point of information instruction, key point are predetermined, on the target object Point;
For the key point information in key point information set, determine that the key point of key point information instruction is corresponding in the base The coordinate of position in quasi- image obtains reference coordinate set as reference coordinate;
According to obtained location information set and reference coordinate set, determined in the shooting image using Image geometry transform The corresponding position in the benchmark image of object region.
2. according to the method described in claim 1, wherein, location information further includes visibility information, wherein visibility information is used In the probability for indicating that the key point of the key point information instruction in key point information set is shown in the shooting image.
3. according to the method described in claim 2, wherein, the location information set and reference coordinate set that the basis obtains, The corresponding position in the benchmark image of object region in the shooting image, packet are determined using Image geometry transform It includes:
The location information that the visibility information for including is greater than preset visibility threshold is chosen from obtained location information set, Obtain sub- location information set;
The position coordinates for including according to the location information in obtained sub- location information set and obtained sub- location information set In the corresponding key point of location information reference coordinate, utilize perspective transform determine shooting image in object region pair It should be in the position in the benchmark image.
4. according to the method described in claim 1, wherein, the method also includes:
Target location in the benchmark image is labeled, wherein target position is the target in the shooting image The corresponding position in the benchmark image of image-region.
5. method described in one of -4 according to claim 1, wherein training obtains critical point detection model as follows:
Obtain training sample set, wherein training sample includes the shooting image position corresponding with shooting image of target object Information aggregate;
Using the method for machine learning, the shooting image of the target object in the training sample in the training sample set is made It is desired output by the corresponding position information set cooperation of shooting image with input, training obtains critical point detection mould for input Type.
6. according to the method described in claim 2, wherein, it is seen that property information indicates with the numerical value between zero and one, including zero With one.
7. it is a kind of for handling the device of image, including:
Acquiring unit is configured to obtain the shooting image of target object and shows the benchmark image of target object;
Processing unit is configured to the shooting image being input to training in advance, corresponding with target object key Point detection model, obtains location information set, wherein location information includes position coordinates, and position coordinates are for indicating key point The corresponding position in the shooting image of the key point of key point information instruction in information aggregate, key point is predetermined , point on the target object;
Determination unit is configured to determine key point information instruction for the key point information in key point information set The coordinate of the corresponding position in the benchmark image of key point obtains reference coordinate set as reference coordinate;
The determination unit is further configured to utilize image according to obtained location information set and reference coordinate set Geometric transformation determines the corresponding position in the benchmark image of object region in the shooting image.
8. device according to claim 7, wherein location information further includes visibility information, wherein visibility information is used In the probability for indicating that the key point of the key point information instruction in key point information set is shown in the shooting image.
9. device according to claim 8, wherein the determination unit is not configured to further:
The location information that the visibility information for including is greater than preset visibility threshold is chosen from obtained location information set, Obtain sub- location information set;
The position coordinates for including according to the location information in obtained sub- location information set and obtained sub- location information set In the corresponding key point of location information reference coordinate, utilize perspective transform determine shooting image in object region pair It should be in the position in the benchmark image.
10. device according to claim 7, wherein described device further includes:
Unit is marked, the target location being configured in the benchmark image is labeled, wherein target position is described Shoot the corresponding position in the benchmark image of object region in image.
11. the device according to one of claim 7-10, wherein critical point detection model is trained as follows It arrives:
Obtain training sample set, wherein training sample includes the shooting image position corresponding with shooting image of target object Information aggregate;
Using the method for machine learning, the shooting image of the target object in the training sample in the training sample set is made It is desired output by the corresponding position information set cooperation of shooting image with input, training obtains critical point detection mould for input Type.
12. device according to claim 8, wherein visibility information indicates with the numerical value between zero and one, including zero With one.
13. a kind of electronic equipment, including:
One or more processors;
Storage device is stored thereon with one or more programs;
When one or more of programs are executed by one or more of processors, so that one or more of processors are real Now such as method as claimed in any one of claims 1 to 6.
14. a kind of computer-readable medium, is stored thereon with computer program, wherein the realization when program is executed by processor Such as method as claimed in any one of claims 1 to 6.
CN201810734682.7A 2018-07-06 2018-07-06 Method and apparatus for handling image Pending CN108876858A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810734682.7A CN108876858A (en) 2018-07-06 2018-07-06 Method and apparatus for handling image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810734682.7A CN108876858A (en) 2018-07-06 2018-07-06 Method and apparatus for handling image

Publications (1)

Publication Number Publication Date
CN108876858A true CN108876858A (en) 2018-11-23

Family

ID=64299700

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810734682.7A Pending CN108876858A (en) 2018-07-06 2018-07-06 Method and apparatus for handling image

Country Status (1)

Country Link
CN (1) CN108876858A (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110111241A (en) * 2019-04-30 2019-08-09 北京字节跳动网络技术有限公司 Method and apparatus for generating dynamic image
CN110210526A (en) * 2019-05-14 2019-09-06 广州虎牙信息科技有限公司 Predict method, apparatus, equipment and the storage medium of the key point of measurand
CN110223220A (en) * 2019-06-14 2019-09-10 北京百度网讯科技有限公司 A kind of method and apparatus handling image
CN110335313A (en) * 2019-06-17 2019-10-15 腾讯科技(深圳)有限公司 Audio collecting device localization method and device, method for distinguishing speek person and system
CN110674815A (en) * 2019-09-29 2020-01-10 四川长虹电器股份有限公司 Invoice image distortion correction method based on deep learning key point detection
CN111241887A (en) * 2018-11-29 2020-06-05 北京市商汤科技开发有限公司 Target object key point identification method and device, electronic equipment and storage medium
CN111582153A (en) * 2020-05-07 2020-08-25 北京百度网讯科技有限公司 Method and device for determining document orientation
CN113129375A (en) * 2021-04-21 2021-07-16 阿波罗智联(北京)科技有限公司 Data processing method, device, equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103390282A (en) * 2013-07-30 2013-11-13 百度在线网络技术(北京)有限公司 Image tagging method and device
CN104778712A (en) * 2015-04-27 2015-07-15 厦门美图之家科技有限公司 Method and system for pasting image to human face based on affine transformation
CN107590807A (en) * 2017-09-29 2018-01-16 百度在线网络技术(北京)有限公司 Method and apparatus for detection image quality
CN107609506A (en) * 2017-09-08 2018-01-19 百度在线网络技术(北京)有限公司 Method and apparatus for generating image
WO2018099556A1 (en) * 2016-11-30 2018-06-07 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Image processing device and method for producing in real-time a digital composite image from a sequence of digital images of an interior of a hollow structure

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103390282A (en) * 2013-07-30 2013-11-13 百度在线网络技术(北京)有限公司 Image tagging method and device
CN104778712A (en) * 2015-04-27 2015-07-15 厦门美图之家科技有限公司 Method and system for pasting image to human face based on affine transformation
WO2018099556A1 (en) * 2016-11-30 2018-06-07 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Image processing device and method for producing in real-time a digital composite image from a sequence of digital images of an interior of a hollow structure
CN107609506A (en) * 2017-09-08 2018-01-19 百度在线网络技术(北京)有限公司 Method and apparatus for generating image
CN107590807A (en) * 2017-09-29 2018-01-16 百度在线网络技术(北京)有限公司 Method and apparatus for detection image quality

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111241887A (en) * 2018-11-29 2020-06-05 北京市商汤科技开发有限公司 Target object key point identification method and device, electronic equipment and storage medium
CN111241887B (en) * 2018-11-29 2024-04-16 北京市商汤科技开发有限公司 Target object key point identification method and device, electronic equipment and storage medium
CN110111241B (en) * 2019-04-30 2023-10-03 北京字节跳动网络技术有限公司 Method and apparatus for generating dynamic image
CN110111241A (en) * 2019-04-30 2019-08-09 北京字节跳动网络技术有限公司 Method and apparatus for generating dynamic image
CN110210526A (en) * 2019-05-14 2019-09-06 广州虎牙信息科技有限公司 Predict method, apparatus, equipment and the storage medium of the key point of measurand
CN110223220A (en) * 2019-06-14 2019-09-10 北京百度网讯科技有限公司 A kind of method and apparatus handling image
CN110223220B (en) * 2019-06-14 2023-03-31 北京百度网讯科技有限公司 Method and device for processing image
CN110335313A (en) * 2019-06-17 2019-10-15 腾讯科技(深圳)有限公司 Audio collecting device localization method and device, method for distinguishing speek person and system
WO2020253616A1 (en) * 2019-06-17 2020-12-24 腾讯科技(深圳)有限公司 Audio collection device positioning method and apparatus, and speaker recognition method and system
US11915447B2 (en) 2019-06-17 2024-02-27 Tencent Technology (Shenzhen) Company Limited Audio acquisition device positioning method and apparatus, and speaker recognition method and system
CN110335313B (en) * 2019-06-17 2022-12-09 腾讯科技(深圳)有限公司 Audio acquisition equipment positioning method and device and speaker identification method and system
CN110674815A (en) * 2019-09-29 2020-01-10 四川长虹电器股份有限公司 Invoice image distortion correction method based on deep learning key point detection
CN111582153B (en) * 2020-05-07 2023-06-30 北京百度网讯科技有限公司 Method and device for determining orientation of document
CN111582153A (en) * 2020-05-07 2020-08-25 北京百度网讯科技有限公司 Method and device for determining document orientation
CN113129375B (en) * 2021-04-21 2023-12-01 阿波罗智联(北京)科技有限公司 Data processing method, device, equipment and storage medium
CN113129375A (en) * 2021-04-21 2021-07-16 阿波罗智联(北京)科技有限公司 Data processing method, device, equipment and storage medium

Similar Documents

Publication Publication Date Title
CN108876858A (en) Method and apparatus for handling image
CN108986169A (en) Method and apparatus for handling image
CN108898185A (en) Method and apparatus for generating image recognition model
CN108898186A (en) Method and apparatus for extracting image
CN108470328A (en) Method and apparatus for handling image
CN107911753A (en) Method and apparatus for adding digital watermarking in video
CN108830235A (en) Method and apparatus for generating information
CN109086719A (en) Method and apparatus for output data
CN108345387A (en) Method and apparatus for output information
CN109389640A (en) Image processing method and device
CN109034069A (en) Method and apparatus for generating information
CN108510454A (en) Method and apparatus for generating depth image
CN109472264A (en) Method and apparatus for generating object detection model
CN108960110A (en) Method and apparatus for generating information
CN108337505A (en) Information acquisition method and device
CN108491823A (en) Method and apparatus for generating eye recognition model
CN108062544A (en) For the method and apparatus of face In vivo detection
CN108509921A (en) Method and apparatus for generating information
CN109255767A (en) Image processing method and device
CN108133197A (en) For generating the method and apparatus of information
CN108182457A (en) For generating the method and apparatus of information
CN110110666A (en) Object detection method and device
CN109241934A (en) Method and apparatus for generating information
CN108446658A (en) The method and apparatus of facial image for identification
CN109389096A (en) Detection method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination