CN108897899A - The localization method and its device of the target area of a kind of pair of video flowing - Google Patents
The localization method and its device of the target area of a kind of pair of video flowing Download PDFInfo
- Publication number
- CN108897899A CN108897899A CN201810964684.5A CN201810964684A CN108897899A CN 108897899 A CN108897899 A CN 108897899A CN 201810964684 A CN201810964684 A CN 201810964684A CN 108897899 A CN108897899 A CN 108897899A
- Authority
- CN
- China
- Prior art keywords
- frame
- video flowing
- target area
- locating
- tagging
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/48—Matching video sequences
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The present invention provides the localization method of the target area of a kind of pair of video flowing and its devices, wherein the method includes：Video flowing is obtained, and extracts key frame；Locating frame is identified using identification model；Construct frame tagging；Localization region is determined according to default characteristic information；It determines contour line and profile line coordinates corresponding with the contour line, and profile line coordinates is stored in frame tagging.The present invention realize for comprising target area video flowing in key frame intelligent positioning, so as to determine the timestamp in video flowing including locating frame according to label, and obtain the contour line and profile line coordinates of the locating frame in video flowing, so as to further be carried out according to updated label easily to the editor of video flowing, location efficiency is high, time short speed is fast, brings conveniently for the work of supervisor.
Description
Technical field
The present invention relates to technical field of image processing, determine more specifically to the target area of a kind of pair of video flowing
Position method and device thereof.
Background technique
Video flowing refers to the transmission of video data, for example, it can be passed through net as a stable and continuous stream
Network processing.Because of flowing, client browser or plug-in unit can show multi-medium data before entire file is transmitted.Example
Such as, video flowing, the data, or the data played online etc. that can be transmitted by net cast.
With the rapid development of the relevant industry of present video playing, brings to the supervision of market and business and increasingly compel
The needs cut, especially wherein live streaming, play video, video flowing in readable storage medium storing program for executing etc. video playing mode online,
Content that is supervising for the needs being directed to or needing to mark emphatically, proposes higher technical requirement.
Currently, the existing supervision of content for being directed to need to supervise or need to position, need manually for
Target video stream carry out frame by frame check or F.F. is checked, need to take considerable time, human cost and physics cost, work numerous
Trivial inefficiency can not accomplish to position and monitor in real time, to prison especially for the monitoring of the partial content in live streaming industry
The work of pipe personnel brings huge inconvenience.
Summary of the invention
In view of this, localization method and its device that the present invention provides the target area of a kind of pair of video flowing are existing to solve
The deficiency of technology.
To solve the above problems, the present invention provides the localization method of the target area of a kind of pair of video flowing, including：
Video flowing is obtained, and extracts the key frame arranged in the video flowing according to timestamp；
Identified using identification model trained in advance include target positioning object the key frame, as positioning
Frame；
Construct frame tagging corresponding with the locating frame of each frame；It include corresponding with the locating frame in the frame tagging
Timestamp；
The positioning of each frame is determined according to the default characteristic information of target area corresponding with target positioning object
Localization region in frame；
Determine the localization region contour line and profile line coordinates corresponding with the contour line, and by the contour line
Coordinate is stored in the frame tagging, in order to be edited according to the frame tagging to the target area in each locating frame.
Preferably, described " to be determined according to the default characteristic information of target area corresponding with target positioning object
Localization region in the locating frame of each frame " includes：
According to the default characteristic information of the target area, feature vector is generated；
It will be in the locating frame of target area and next frame in the locating frame of each frame according to described eigenvector
Target area is matched, choose in the locating frame of the next frame with the highest region of target area matching degree, as
Localization region.
Preferably, described " determine the localization region contour line and profile line coordinates corresponding with the contour line, and
The profile line coordinates is stored in the frame tagging, in order to according to the frame tagging to the target area in each locating frame
Edited " include：
Extract the localization region of each locating frame；
Interception includes the minimum screenshot of the localization region；
Binary conversion treatment is carried out to the minimum screenshot, obtains binaryzation screenshot；
Edge detection is carried out to the binaryzation screenshot, obtains edge wheel profile, and obtain according to the edge wheel profile
Profile line coordinates, and the profile line coordinates is stored in the frame tagging, in order to according to the frame tagging to each positioning
It is edited target area in frame.
Preferably, described " identified using identification model trained in advance include target positioning object the key
After frame, as locating frame ", further include：
The quantity for confirming the target positioning object for including in the locating frame, as number of targets；
It is described " each frame to be determined according to the default characteristic information of target area corresponding with target positioning object
Localization region in locating frame " includes：
The default characteristic information is compared with the locating frame of each frame；
If exist in the locating frame with the matched region of the default characteristic information, and matched with the characteristic information
Region number be equal to the number of targets, then using the region as localization region.
Preferably, described " quantity for the target positioning object for including in the locating frame being confirmed, as number of targets "
Later, further include：
If the number of targets is greater than goal-selling threshold value, determine that there are abnormal conditions in the video flowing, and confirm
The quantity of the locating frame of existing abnormal conditions, stops the transmission of the video flowing, prompt information is generated, in order to prompt correlation
Personnel check；
If the number of targets is not more than the targets threshold, the number of targets is saved, in order to construct and each frame
The corresponding frame tagging of locating frame.
Preferably, the default characteristic information includes that color characteristic and Pixel Dimensions corresponding with the color characteristic are special
Sign；
After described " obtain video flowing, and extract the key frame arranged in the video flowing according to timestamp ", further include：
Prescreening is carried out to the key frame, if wherein special comprising the color with the default characteristic information of the target area
The frame that the Pixel Dimensions feature of the color characteristic of seeking peace matches, then as screening frame；
The frame other than the screening frame in the key frame is deleted, retains screening frame therein, as the key after prescreening
Frame, in order to identify the key frame for including target positioning object using identification model trained in advance, as positioning
Frame.
In addition, to solve the above problems, being wrapped the present invention also provides the positioning device of the target area of a kind of pair of video flowing
It includes：Extraction module, identification module, building module, locating module and determining module；
The extraction module for obtaining video flowing, and extracts the key frame arranged in the video flowing according to timestamp；
The identification module includes described in target positioning object for being identified using identification model trained in advance
Key frame, as locating frame；
The building module, for constructing frame tagging corresponding with the locating frame of each frame；Include in the frame tagging
Timestamp corresponding with the locating frame；
The locating module, for the default characteristic information according to target area corresponding with target positioning object
Determine the localization region in the locating frame of each frame；
The determining module, contour line and contour line corresponding with the contour line for determining the localization region are sat
Mark, and the profile line coordinates is stored in the frame tagging, in order to according to the frame tagging to the mesh in each locating frame
It is edited in mark region.
In addition, to solve the above problems, the present invention also provides a kind of user terminal, including memory and processor, institute
It states memory and runs the target to video flowing for storing the finder to the target area of video flowing, the processor
The finder in region is so that the user terminal executes the localization method of the target area as described above to video flowing.
In addition, to solve the above problems, the present invention also provides a kind of computer readable storage medium, it is described computer-readable
The finder to the target area of video flowing, the finder of the target area to video flowing are stored on storage medium
The localization method of the target area as described above to video flowing is realized when being executed by processor.
The localization method and its device of the target area of a kind of pair of video flowing provided by the invention.Wherein, the present invention is mentioned
The method of confession includes：Video flowing is obtained, and extracts the key frame arranged in the video flowing according to timestamp；Utilize preparatory training
Identification model identify include target positioning object the key frame, as locating frame；The positioning of building and each frame
The corresponding frame tagging of frame；It include timestamp corresponding with the locating frame in the frame tagging；It is positioned according to the target
The default characteristic information of the corresponding target area of object determines the localization region in the locating frame of each frame；Determine the positioning
The contour line in region and profile line coordinates corresponding with the contour line, and the profile line coordinates is stored in the frame tagging
In, in order to be edited according to the frame tagging to the target area in each locating frame.The present invention, which passes through, utilizes identification mould
Type identifies key frame, to find out locating frame therein, and establishes the mark comprising timestamp corresponding with locating frame
Label, and then determine localization region therein, contour line and profile line coordinates are determined further according to the localization region, then update the mark
Label, in order to further be edited to the video flowing, thus realize for comprising target area video flowing in pass
The intelligent positioning of key frame so as to determine the timestamp in video flowing including locating frame according to label, and obtains in video flowing
Locating frame contour line and profile line coordinates, so as to further according to updated label carry out easily to video
The editor of stream, location efficiency is high, time short speed is fast, brings conveniently for the work of supervisor.
Detailed description of the invention
Fig. 1 is the hardware running environment that the present invention is related to the localization method example scheme of the target area of video flowing
Structural schematic diagram；
Fig. 2 is flow diagram of the present invention to the localization method first embodiment of the target area of video flowing；
Fig. 3 is flow diagram of the present invention to the localization method second embodiment of the target area of video flowing；
Fig. 4 is flow diagram of the present invention to the localization method 3rd embodiment of the target area of video flowing；
Fig. 5 is flow diagram of the present invention to the localization method fourth embodiment of the target area of video flowing；
Fig. 6 is flow diagram of the present invention to the 5th embodiment of localization method of the target area of video flowing；
Fig. 7 is flow diagram of the present invention to the localization method sixth embodiment of the target area of video flowing；
Fig. 8 is the functional block diagram of the present invention to the positioning device of the target area of video flowing.
The embodiments will be further described with reference to the accompanying drawings for the realization, the function and the advantages of the object of the present invention.
Specific embodiment
The embodiment of the present invention is described below in detail, in which the same or similar labels are throughly indicated same or like
Element or element with the same or similar functions.
In addition, term " first ", " second " are used for descriptive purposes only and cannot be understood as indicating or suggesting relative importance
Or implicitly indicate the quantity of indicated technical characteristic.Define " first " as a result, the feature of " second " can be expressed or
Implicitly include one or more of the features.In the description of the present invention, the meaning of " plurality " is two or more,
Unless otherwise specifically defined.
In the present invention unless specifically defined or limited otherwise, term " installation ", " connected ", " connection ", " fixation " etc.
Term shall be understood in a broad sense, for example, it may be being fixedly connected, may be a detachable connection, or integral；It can be mechanical connect
It connects, is also possible to be electrically connected；It can be directly connected, can also can be in two elements indirectly connected through an intermediary
The interaction relationship of the connection in portion or two elements.It for the ordinary skill in the art, can be according to specific feelings
Condition understands the concrete meaning of above-mentioned term in the present invention.
It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not intended to limit the present invention.
As shown in Figure 1, Fig. 1 is the structural schematic diagram of the hardware running environment for the terminal that the embodiment of the present invention is related to.
As shown in Figure 1, the terminal may include：Processor 1001, such as CPU, network interface 1004, user interface
1003, memory 1005, communication bus 1002.Wherein, communication bus 1002 is for realizing the connection communication between these components.
User interface 1003 may include display screen, input unit such as keyboard, remote controler, and optional user interface 1003 can also include
Standard wireline interface and wireless interface.Network interface 1004 optionally may include standard wireline interface and wireless interface (such as
WI-FI interface).Memory 1005 can be high speed RAM memory, be also possible to stable memory, such as magnetic disk storage.
Memory 1005 optionally can also be the storage device independently of aforementioned processor 1001.
Optionally, terminal can also include RF (Radio Frequency, radio frequency) circuit, sensor, voicefrequency circuit,
WiFi module etc..In addition, mobile terminal can also configure gyroscope, barometer, hygrometer, thermometer, infrared sensor etc.
Other sensors, details are not described herein.
It will be understood by those skilled in the art that the restriction of the not structure paired terminal of terminal shown in Fig. 1, may include ratio
More or fewer components are illustrated, certain components or different component layouts are perhaps combined.
As shown in Figure 1, as may include operating system, number in a kind of memory 1005 of computer readable storage medium
According to the finder of interface control program, network attachment procedure and the target area to video flowing.
The localization method and its device of the target area of a kind of pair of video flowing provided by the invention.Wherein, the method is real
Showed for comprising target area video flowing in key frame intelligent positioning, so as to be determined in video flowing according to label
Include the timestamp of locating frame, and obtain the contour line and profile line coordinates of the locating frame in video flowing, so as into one
Step is carried out according to updated label easily to the editor of video flowing, and location efficiency is high, time short speed is fast, is superintendent
The work of member is brought conveniently.
Embodiment 1：
Referring to Fig. 2, first embodiment of the invention provides the localization method of the target area of a kind of pair of video flowing, including：
Step S10 obtains video flowing, and extracts the key frame arranged in the video flowing according to timestamp；
The video flowing of the present embodiment, the real-time transmission that can be directed in live streaming industry is supervised, for wherein occurring
The video of the behavior for relating to Huang or being designed into other violations be monitored, in addition it is also possible to for in transmission of video or playing
In content carry out real-time edition, need the content that positions to position to therein, so to orient the content of profile into
The further editor of row.
It is above-mentioned, carry out acquisition video flowing, can according to video flowing acquisition instruction, to specified time point therein or when
Between the video flowing of section obtained, or all video flowings are obtained.
It is above-mentioned, video decomposition is carried out to accessed video flowing, obtains multiple key frames, the quantity of key frame can be according to
According to key frame acquired in preset time, for example, mono- key frame of 0.2S.
Timestamp is time tag corresponding with key frame, and each key frame is equipped with a timestamp, can be chased after by timestamp
It traces back time point of corresponding key frame, to carry out the editor to the frame according to time point.
Step S20, identified using identification model trained in advance include target positioning object the key frame, make
For locating frame；
Artificial neural network (Artificial Neural Network, i.e. ANN), it is artificial since being the 1980s
The research hotspot that smart field rises.It is abstracted human brain neuroid from information processing angle, and it is simple to establish certain
Model is formed different networks by different connection types.Neural network or class are also often directly referred to as in engineering and academia
Neural network.Neural network is a kind of operational model, is constituted by being coupled to each other between a large amount of node (or neuron).Each
A kind of specific output function of node on behalf, referred to as excitation function (activation function).Company between every two node
It connects and all represents a weighted value for passing through the connection signal, referred to as weight, this is equivalent to the memory of artificial neural network.
The output of network then according to the connection type of network, the difference of weighted value and excitation function and it is different.And network itself is usually all
Certain algorithm of nature or function are approached, it is also possible to the expression to a kind of logic strategy.
It is above-mentioned, by being trained study to preset artificial neural network, thus the identification model trained in advance,
And then key frame obtained is identified frame by frame, it has determined whether to position object comprising target, if including mesh
Position object is demarcated, then extracts the frame as locating frame.
Target positions object, for the object positioned in the present embodiment, for example, carrying out control process to live streaming
In, set the privacy places of human body as target and position object, then through this embodiment provided in method target is positioned
Object is positioned.
Step S30 constructs frame tagging corresponding with the locating frame of each frame；Include in the frame tagging and the positioning
The corresponding timestamp of frame；
It is above-mentioned, to each locating frame, establish frame tagging, wherein it include corresponding timestamp in frame tagging, in addition,
Also it can wrap containing for example, target positions the information such as the size of object.In the present embodiment, by being set for each locating frame
Framing label improves editorial efficiency so as to edit according to frame tagging to the frame positioned, is staff
It provides convenience.
Step S40 is determined each according to the default characteristic information of target area corresponding with target positioning object
Localization region in the locating frame of frame；
Above-mentioned, default characteristic information positions the characteristic information of object for the preset target positioned,
In may include the target positioning pixel size of object, color, texture, contrast, adjacent pixels feature etc. characteristic information.
It is above-mentioned, target area, for region corresponding with target positioning object.
It is above-mentioned, it determines the localization region in each locating frame according to default characteristic information by image recognition, that is, determines
The size of localization region, area etc. information.
Step S50, determine the localization region contour line and profile line coordinates corresponding with the contour line, and by institute
It states profile line coordinates to be stored in the frame tagging, in order to carry out the target area in each locating frame according to the frame tagging
Editor.
Above-mentioned, contour line is the outer peripheral line of minimum of identified localization region by image recognition, which includes
Identifiable localization region.Contour line can be determined during determination by image recognition, such as edge detection method
The difference of pixel or color and vein, so that it is determined that its contour line, in intuitive vision visualized operation system, true
After fixed wheel profile, flash for prompting can be carried out, so that editorial staff be prompted further to be worked, such as is carried out pair according to contour line
Target area in contour line carries out stamp or mapping operations.
Above-mentioned, profile line coordinates is the contour line of digitization, can be the coordinate of each pixel shared by contour line, lead to
It crosses and gets profile line coordinates, can further be operated according to the contour line of digitization, to improve image processing speed.
Above-mentioned, profile line coordinates is stored in frame tagging, can be according to contour line coordinate pair by the editor to frame tagging
Localization region is edited, and in the prior art, directly carries out the editor to localization region to patterned image, compares it
Under, it is operated by digitized coordinate, such as stamp or textures, increases the accuracy of graphics edition, improve figure
Shape editorial efficiency.
It is above-mentioned, can also binary information conversion be carried out to frame tagging, since system bottom is binary data, lead to
It crosses and frame tagging is converted directly into binary message, improve the efficiency for reading and editing for figure in system.
The present embodiment to find out locating frame therein, and is built by being identified using identification model to key frame
The label comprising timestamp corresponding with locating frame is found, and then determines localization region therein, is determined further according to the localization region
Contour line and profile line coordinates, then update the label, in order to further be edited to the video flowing, realize for comprising
Target area video flowing in key frame intelligent positioning, include locating frame in video flowing so as to be determined according to label
Timestamp, and obtain the contour line and profile line coordinates of the locating frame in video flowing, can be further according to updated
Label carries out easily to the editor of video flowing, and location efficiency is high, time short speed is fast, is the work side of bringing of supervisor
Just.
Embodiment 2：
Referring to Fig. 3, second embodiment of the invention provides the localization method of the target area of a kind of pair of video flowing, based on above-mentioned
First embodiment shown in Fig. 2, the step S40, " according to the default of target area corresponding with target positioning object
Characteristic information determines the localization region in the locating frame of each frame " include：
Step S41 generates feature vector according to the default characteristic information of the target area；
In the present embodiment, extracted by HOG (Histograms of Oriented Gradients) feature extracting method
This needs the textural characteristics in the region (target area) of stamp, by preset characteristic information, such as color characteristic and textural characteristics,
Generate first eigenvector.
It may include color characteristic and image texture characteristic for example, default characteristic information, it specifically will be in the target area
Each pixel RGB ratio value (i.e. rgb value) dimension by 3 dimension, be converted to 11 dimensions, in other words, at present it is each
The RGB ratio value of a pixel can only characterize three kinds of colors of red, green, blue, and the dimension of RGB ratio value is converted by 3 dimensions
After 11 dimensions, the color that the RGB ratio value of each pixel characterizes is refine to 11 kinds, as black, blue, yellow,
Grey, pink colour, red, white etc., so that when extracting needs the color characteristic in stamp region, it can be adaptively according to needing stamp
The actual color distribution situation in region selected.Simultaneously by the thought of dimensionality reduction, adaptively by each pixel
RGB ratio value drops to 2 dimensions by 11 dimensions, to extract the significant color characteristic needed in stamp region.
Step S42, according to described eigenvector by the locating frame of each frame target area and next frame determine
Position frame in target area matched, choose in the locating frame of the next frame with the highest area of target area matching degree
Domain, as localization region.
It is above-mentioned, in the present embodiment, according to feature vector, by each locating frame target area and next frame adjacent thereto
Locating frame in target area matched, i.e., two target areas carry out the matching based on feature vector, so that it is determined that go out
The highest region of matching degree in the locating frame of next frame, thus using the region as localization region.In the present embodiment, pass through
Using feature vector, the target area in current locating frame is compared and is matched with the target area of next frame, thus
The highest region of matching degree in next frame is found out as localization region, so as to according to the timestamp in positioning label, foundation
The sequence of timestamp, the positioning area being sequentially determined in the locating frame of each current locating frame and next frame adjacent thereto
Domain improves location efficiency, improves the accuracy of positioning.
Embodiment 3：
Referring to Fig. 4, third embodiment of the invention provides the localization method of the target area of a kind of pair of video flowing, based on above-mentioned
First embodiment shown in Fig. 2, the step S50 " determine the contour line of the localization region and corresponding with the contour line
Profile line coordinates, and the profile line coordinates is stored in the frame tagging, in order to according to the frame tagging to each positioning
Edited target area in frame " include：
Step S51 extracts the localization region of each locating frame；
Step S52, interception include the minimum screenshot of the localization region；
It is above-mentioned, after extracting the localization region in locating frame, shot operation is carried out to it, interception includes localization region
Minimum screenshot, wherein including by localization region determined by image recognition in minimum screenshot.
Step S53 carries out binary conversion treatment to the minimum screenshot, obtains binaryzation screenshot；
It is above-mentioned, it should be noted that binaryzation (English：Thresholding) be image segmentation a kind of simplest side
Method.Binaryzation can be greyscale image transitions at bianry image.The pixel grey scale for being greater than some threshold grey scale value is set as gray scale
The pixel grey scale for being less than this value is set as gray scale minimum, to realize binaryzation by maximum.It is chosen not according to threshold value
Together, the algorithm of binaryzation is divided into fixed threshold and adaptive threshold.More commonly used binarization method then has：Two-peak method, P parameter
Method, iterative method and OTSU method etc..The binaryzation of image exactly sets 0 or 255 for the gray value of the pixel on image,
Exactly whole image is showed and significantly there was only black and white visual effect.
It is above-mentioned, by the binaryzation of image, image is made to be converted to the bicolorable graph that each pixel only has black and white two kinds of colors
Picture obtains binaryzation screenshot.
Step S54 carries out edge detection to the binaryzation screenshot, obtains edge wheel profile, and according to the edge wheel
Profile obtains profile line coordinates, and the profile line coordinates is stored in the frame tagging, in order to according to the frame tagging pair
It is edited target area in each locating frame.
It is above-mentioned, edge detection is carried out to binaryzation screenshot, so that it is determined that more accurate contour line and obtaining contour line out
Coordinate updates frame tagging, in order to further progress stamp or textures editor.In the present embodiment, by being carried out for image
Interception includes the minimum screenshot of localization region, is carried out in transmission process in data flow, the occupied space of substantial amounts is huge
The locating frame that key frame is determined occupies a large amount of system resource, carries out real-time or timing processing, meeting to above-mentioned locating frame
The resource of system is caused to handle overload, if numerous, the to be processed locating frame substantial amounts of data cause to drag to a certain extent
The case where slow processing speed even causes system crash.The present embodiment passes through the minimum screenshot that interception includes localization region, into
One step is positioned and is handled to minimum screenshot, greatly reduce as data it is numerous caused by system resource a large amount of occupancy,
The size of data for greatly reducing data interaction, improves data processing speed, improves efficiency, reduces resource occupation
And waste.Binary conversion treatment is carried out to image, to convert the image into the image of black and white two kinds of colors, is removed therein big
The interference for measuring non-identifying element further improves the accuracy of image recognition, improves the efficiency of identification and detection.
Embodiment 4：
Referring to Fig. 5, fourth embodiment of the invention provides the localization method of the target area of a kind of pair of video flowing, based on above-mentioned
First embodiment shown in Fig. 2, the step S20 " identify to include target positioning object using identification model trained in advance
After the key frame of body, as locating frame ", further include：
Step S60 confirms the quantity for the target positioning object for including in the locating frame, as number of targets；
The step S40 " is determined according to the default characteristic information of target area corresponding with target positioning object
Localization region in the locating frame of each frame " includes：
The default characteristic information is compared step S43 with the locating frame of each frame；
Step S44, if in the locating frame exist with the matched region of the default characteristic information, and with the feature
The number in the region of information matches is equal to the number of targets, then using the region as localization region.
It is above-mentioned, to acquired key frame carry out using wherein include at identification model identification target positioning object
Frame determines in all locating frames after locating frame, includes the quantity of target positioning object, as number of targets.The mesh
Mark number, can be used as one of the index of accuracy of authentication image positioning, by by the comparison of default characteristic information and locating frame from
And further confirm that the number for being included in each locating frame, and then the number is compared with number of targets, if phase
Deng then can be using the region as localization region.By further by the comparison of number of targets, so that it is determined that the target for being included
The number for positioning object, improves the accuracy of image recognition.
Embodiment 5：
Referring to Fig. 6, fifth embodiment of the invention provides the localization method of the target area of a kind of pair of video flowing, based on above-mentioned
Fourth embodiment shown in fig. 5, the step S60 " confirm the number for the target positioning object for including in the locating frame
After amount, as number of targets ", further include：
Step S70 determines in the video flowing if the number of targets is greater than goal-selling threshold value there are abnormal conditions,
And confirm the quantity for the locating frame of abnormal conditions occur, stop the transmission of the video flowing, generates prompt information, in order to
Prompt related personnel checks；
Step S80, if the number of targets be not more than the targets threshold, save the number of targets, in order to construct with
The corresponding frame tagging of the locating frame of each frame.
It is above-mentioned, in certain occasion or application scenarios, to the image in the data flow of video or live streaming, need to carry out into
The monitoring of one step then can determine that when the trees for the target positioning object that it occurs reach certain amount to there are abnormal feelings
Condition.For example, setting target positioning object in live streaming as the privacy places of human body, passing through the target for including in confirmation locating frame
The quantity for positioning object, as number of targets, goal-selling threshold value is 1, if the quantity of number of targets is 1, can pass through technology
Carry out to its stamp or mapping operations, if there is quantity be greater than 1, then can determine that abnormal to occur, which has
Yellow suspicion is related to, produces prompt information, and the transmission of video flowing can be stopped, relevant staff is prompted to check.This implementation
In example, by setting targets threshold, and then obtained number of targets is compared with the threshold value, if number of targets is greater than threshold
Value, then generate prompt information, so as to be in real time monitored video flowing, intelligent pair in transmission process in video flowing
The case where target positioning object beyond threshold value wherein occurred, is prompted, and the abnormal conditions that early warning occurs are staff
The monitoring and editor of video flowing are provided convenience.
Embodiment 6：
Referring to Fig. 7, sixth embodiment of the invention provides the localization method of the target area of a kind of pair of video flowing, based on above-mentioned
First embodiment shown in Fig. 2, the default characteristic information include color characteristic and pixel ruler corresponding with the color characteristic
Very little feature；
The step S10, " obtain video flowing, and extract in the video flowing according to timestamp arrange key frame " it
Afterwards, further include：
Step S90 carries out prescreening to the key frame, if wherein comprising the default characteristic information with the target area
Color characteristic and the color characteristic the frame that matches of Pixel Dimensions feature, then as screening frame；
Step S100 deletes the frame other than the screening frame in the key frame, retains screening frame therein, as prescreening
Key frame afterwards, in order to identify the key frame for including target positioning object using identification model trained in advance,
As locating frame.
Above-mentioned, after getting the key frame in video flowing and video flowing, since data volume is huge, picture quality is higher
Data flow causes key frame to occupy a large amount of storage resource and system processing resources, in the present embodiment, to all key frames into
Prescreening of row gets rid of useless frame to filter out the frame therein further identified.Screening is based on pre-
If characteristic information, include target positioning object color characteristic and Pixel Dimensions feature corresponding with the color characteristic, i.e.,
First confirm that include the key frame of color and size dimension for being adapted or matching with target positioning object.Delete it
Useless frame other than middle screening frame retains screening frame, as key frame, does not need largely to carry out further to get rid of
The image of positioning and identification, greatly reduces the workload of image recognition, improves work efficiency, and is framing and editor's work
It provides convenience.
In addition, the present invention also provides the positioning devices of the target area of a kind of pair of video flowing with reference to Fig. 8, including：Extract mould
Block 10, identification module 20, building module 30, locating module 40 and determining module 50；
The extraction module 10 for obtaining video flowing, and extracts the key arranged in the video flowing according to timestamp
Frame；
The identification module 20 includes the institute of target positioning object for utilizing identification model trained in advance to identify
Key frame is stated, as locating frame；
The building module 30, for constructing frame tagging corresponding with the locating frame of each frame；Include in the frame tagging
There is timestamp corresponding with the locating frame；
The locating module 40, for being believed according to the default feature of target area corresponding with target positioning object
Breath determines the localization region in the locating frame of each frame；
The determining module 50, for determining the contour line and contour line corresponding with the contour line of the localization region
Coordinate, and the profile line coordinates is stored in the frame tagging, in order to according to the frame tagging in each locating frame
It is edited target area.
In addition, the present invention also provides a kind of user terminal, including memory and processor, the memory is for storing
To the finder of the target area of video flowing, the processor run the finder of the target area to video flowing with
The user terminal is set to execute the localization method of the target area as described above to video flowing.
In addition, being stored on the computer readable storage medium the present invention also provides a kind of computer readable storage medium
There is the finder to the target area of video flowing, when the finder of the target area to video flowing is executed by processor
Realize the localization method of the target area as described above to video flowing.
The serial number of the above embodiments of the invention is only for description, does not represent the advantages or disadvantages of the embodiments.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side
Method can be realized by means of software and necessary general hardware platform, naturally it is also possible to by hardware, but in many cases
The former is more preferably embodiment.Based on this understanding, technical solution of the present invention substantially in other words does the prior art
The part contributed out can be embodied in the form of software products, which is stored in one as described above
In storage medium (such as ROM/RAM, magnetic disk, CD), including some instructions are used so that terminal device (it can be mobile phone,
Computer, server or network equipment etc.) execute method described in each embodiment of the present invention.
The above is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair
Equivalent structure or equivalent flow shift made by bright specification and accompanying drawing content is applied directly or indirectly in other relevant skills
Art field, is included within the scope of the present invention.
Claims (9)
1. the localization method of the target area of a kind of pair of video flowing, which is characterized in that including：
Video flowing is obtained, and extracts the key frame arranged in the video flowing according to timestamp；
Identified using identification model trained in advance include target positioning object the key frame, as locating frame；
Construct frame tagging corresponding with the locating frame of each frame；It include the time corresponding with the locating frame in the frame tagging
Stamp；
It is determined in the locating frame of each frame according to the default characteristic information of target area corresponding with target positioning object
Localization region；
Determine the localization region contour line and profile line coordinates corresponding with the contour line, and by the profile line coordinates
It is stored in the frame tagging, in order to be edited according to the frame tagging to the target area in each locating frame.
2. as described in claim 1 to the localization method of the target area of video flowing, which is characterized in that it is described " according to it is described
The default characteristic information of the corresponding target area of target positioning object determines the localization region in the locating frame of each frame " packet
It includes：
According to the default characteristic information of the target area, feature vector is generated；
According to described eigenvector by the target in the locating frame of target area and next frame in the locating frame of each frame
Region is matched, choose in the locating frame of the next frame with the highest region of target area matching degree, as positioning
Region.
3. as described in claim 1 to the localization method of the target area of video flowing, which is characterized in that described " it is described fixed to determine
The contour line in position region and profile line coordinates corresponding with the contour line, and the profile line coordinates is stored in the frame tagging
In, in order to be edited according to the frame tagging to the target area in each locating frame " include：
Extract the localization region of each locating frame；
Interception includes the minimum screenshot of the localization region；
Binary conversion treatment is carried out to the minimum screenshot, obtains binaryzation screenshot；
Edge detection is carried out to the binaryzation screenshot, obtains edge wheel profile, and profile is obtained according to the edge wheel profile
Line coordinates, and the profile line coordinates is stored in the frame tagging, in order to according to the frame tagging in each locating frame
Target area edited.
4. as described in claim 1 to the localization method of the target area of video flowing, which is characterized in that described " to utilize instruction in advance
Experienced identification model identify include target positioning object the key frame, as locating frame " after, further include：
The quantity for confirming the target positioning object for including in the locating frame, as number of targets；
It is described " positioning of each frame to be determined according to the default characteristic information of target area corresponding with target positioning object
Localization region in frame " includes：
The default characteristic information is compared with the locating frame of each frame；
If in the locating frame exist with the matched region of the default characteristic information, and with the matched area of the characteristic information
The number in domain is equal to the number of targets, then using the region as localization region.
5. as claimed in claim 4 to the localization method of the target area of video flowing, which is characterized in that described " confirmation is described fixed
After the quantity for the target positioning object for including in the frame of position, as number of targets ", further include：
If the number of targets is greater than goal-selling threshold value, determine that there are abnormal conditions in the video flowing, and confirms that appearance is different
The quantity of the locating frame of reason condition stops the transmission of the video flowing, prompt information is generated, in order to prompt related personnel
It is checked；
If the number of targets is not more than the targets threshold, the number of targets is saved, in order to construct the positioning with each frame
The corresponding frame tagging of frame.
6. as described in claim 1 to the localization method of the target area of video flowing, which is characterized in that
The default characteristic information includes color characteristic and Pixel Dimensions feature corresponding with the color characteristic；
After described " obtain video flowing, and extract the key frame arranged in the video flowing according to timestamp ", further include：
Prescreening is carried out to the key frame, if wherein comprising with the color characteristic of the default characteristic information of the target area and
The frame that the Pixel Dimensions feature of the color characteristic matches, then as screening frame；
The frame other than the screening frame in the key frame is deleted, screening frame therein is retained, as the key frame after prescreening, with
Convenient for identifying the key frame for including target positioning object using identification model trained in advance, as locating frame.
7. the positioning device of the target area of a kind of pair of video flowing, which is characterized in that including：Extraction module, identification module, building
Module, locating module and determining module；
The extraction module for obtaining video flowing, and extracts the key frame arranged in the video flowing according to timestamp；
The identification module includes the key of target positioning object for utilizing identification model trained in advance to identify
Frame, as locating frame；
The building module, for constructing frame tagging corresponding with the locating frame of each frame；Include in the frame tagging and institute
State the corresponding timestamp of locating frame；
The locating module, for being determined according to the default characteristic information of target area corresponding with target positioning object
Localization region in the locating frame of each frame；
The determining module, for determine the localization region contour line and profile line coordinates corresponding with the contour line,
And the profile line coordinates is stored in the frame tagging, in order to according to the frame tagging to the target area in each locating frame
It is edited in domain.
8. a kind of user terminal, which is characterized in that including memory and processor, the memory is for storing to video flowing
Target area finder, the processor runs the finder of the target area to video flowing so that the use
Family terminal is executed as described in any one of claim 1-6 to the localization method of the target area of video flowing.
9. a kind of computer readable storage medium, which is characterized in that be stored on the computer readable storage medium to video
It realizes when the finder of the finder of the target area of stream, the target area to video flowing is executed by processor as weighed
Benefit requires described in any one of 1-6 to the localization method of the target area of video flowing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810964684.5A CN108897899A (en) | 2018-08-23 | 2018-08-23 | The localization method and its device of the target area of a kind of pair of video flowing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810964684.5A CN108897899A (en) | 2018-08-23 | 2018-08-23 | The localization method and its device of the target area of a kind of pair of video flowing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108897899A true CN108897899A (en) | 2018-11-27 |
Family
ID=64358263
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810964684.5A Pending CN108897899A (en) | 2018-08-23 | 2018-08-23 | The localization method and its device of the target area of a kind of pair of video flowing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108897899A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109816700A (en) * | 2019-01-11 | 2019-05-28 | 佰路得信息技术(上海)有限公司 | A kind of information statistical method based on target identification |
| CN110445954A (en) * | 2019-07-26 | 2019-11-12 | 腾讯科技（深圳）有限公司 | Image-pickup method, device and electronic equipment |
| CN111368140A (en) * | 2020-02-19 | 2020-07-03 | 新华智云科技有限公司 | Video tag generation method and system |
| WO2023195912A3 (en) * | 2022-04-07 | 2023-11-30 | 脸萌有限公司 | Processing method and apparatus, terminal device, and medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102207966A (en) * | 2011-06-01 | 2011-10-05 | 华南理工大学 | Video content quick retrieving method based on object tag |
| CN102831595A (en) * | 2012-06-20 | 2012-12-19 | 中国农业大学 | Marker detection method for image recognition of target points in natural environments |
| CN105681627A (en) * | 2016-03-03 | 2016-06-15 | 联想(北京)有限公司 | Image shooting method and electronic equipment |
| CN107066931A (en) * | 2017-01-12 | 2017-08-18 | 张家港全智电子科技有限公司 | A kind of target trajectory tracking based on monitor video |
| CN107274373A (en) * | 2017-06-30 | 2017-10-20 | 武汉斗鱼网络科技有限公司 | Live middle code printing method and device |
| CN107590443A (en) * | 2017-08-23 | 2018-01-16 | 上海交通大学 | Limiter stage live video automatic testing method and system based on the study of depth residual error |
-
2018
- 2018-08-23 CN CN201810964684.5A patent/CN108897899A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102207966A (en) * | 2011-06-01 | 2011-10-05 | 华南理工大学 | Video content quick retrieving method based on object tag |
| CN102831595A (en) * | 2012-06-20 | 2012-12-19 | 中国农业大学 | Marker detection method for image recognition of target points in natural environments |
| CN105681627A (en) * | 2016-03-03 | 2016-06-15 | 联想(北京)有限公司 | Image shooting method and electronic equipment |
| CN107066931A (en) * | 2017-01-12 | 2017-08-18 | 张家港全智电子科技有限公司 | A kind of target trajectory tracking based on monitor video |
| CN107274373A (en) * | 2017-06-30 | 2017-10-20 | 武汉斗鱼网络科技有限公司 | Live middle code printing method and device |
| CN107590443A (en) * | 2017-08-23 | 2018-01-16 | 上海交通大学 | Limiter stage live video automatic testing method and system based on the study of depth residual error |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109816700A (en) * | 2019-01-11 | 2019-05-28 | 佰路得信息技术(上海)有限公司 | A kind of information statistical method based on target identification |
| CN109816700B (en) * | 2019-01-11 | 2023-02-24 | 佰路得信息技术(上海)有限公司 | Information statistical method based on target identification |
| CN110445954A (en) * | 2019-07-26 | 2019-11-12 | 腾讯科技（深圳）有限公司 | Image-pickup method, device and electronic equipment |
| CN111368140A (en) * | 2020-02-19 | 2020-07-03 | 新华智云科技有限公司 | Video tag generation method and system |
| CN111368140B (en) * | 2020-02-19 | 2023-07-04 | 新华智云科技有限公司 | Video tag generation method and system |
| WO2023195912A3 (en) * | 2022-04-07 | 2023-11-30 | 脸萌有限公司 | Processing method and apparatus, terminal device, and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108897899A (en) | The localization method and its device of the target area of a kind of pair of video flowing | |
| CN112232293A (en) | Image processing model training method, image processing method and related equipment | |
| WO2022041830A1 (en) | Pedestrian re-identification method and device | |
| CN108900895A (en) | The screen method and its device of the target area of a kind of pair of video flowing | |
| CN109544862A (en) | Behavior recognition method and device based on smart home, storage medium and equipment | |
| CN112750162B (en) | Target identification positioning method and device | |
| CN108564012A (en) | A kind of pedestrian's analytic method based on characteristics of human body's distribution | |
| CN110222629A (en) | Bale No. recognition methods and Bale No. identifying system under a kind of steel scene | |
| CN113762314B (en) | Firework detection method and device | |
| CN107506738A (en) | Feature extracting method, image-recognizing method, device and electronic equipment | |
| JP7429756B2 (en) | Image processing method, device, electronic device, storage medium and computer program | |
| EP4080479A2 (en) | Method for identifying traffic light, device, cloud control platform and vehicle-road coordination system | |
| CN114842548A (en) | Violation identification method and device, storage medium and terminal | |
| CN108664906B (en) | Method for detecting content in fire scene based on convolutional network | |
| CN114419493A (en) | Image annotation method and device, electronic equipment and storage medium | |
| CN106960188B (en) | Weather image classification method and device | |
| CN108197563B (en) | Method and device for acquiring information | |
| CN111859370B (en) | Method, apparatus, electronic device and computer readable storage medium for identifying service | |
| CN112052833B (en) | Object density monitoring system, method, video analysis server and storage medium | |
| CN115546680A (en) | Moving target detection method and device, storage medium and electronic equipment | |
| CN111476086A (en) | Smoke and fire identification method and device, computer equipment and readable storage medium | |
| CN109146893B (en) | Oil light area segmentation method and device and mobile terminal | |
| CN113449559A (en) | Table identification method and device, computer equipment and storage medium | |
| Yan et al. | [Retracted] Defect Point Location Method of Civil Bridge Based on Internet of Things Wireless Communication | |
| CN118314527B (en) | Hoisting safety monitoring method, hoisting safety monitoring device, terminal and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220629 Address after: Room 368, 302, 211 Fute North Road, China (Shanghai) pilot Free Trade Zone, Pudong New Area, Shanghai Applicant after: Shanghai Yuepu Investment Center (L.P.) Address before: 518000 floor 33, Yantian modern industry service center, 3018 Shayan Road, Shatoujiao street, Yantian District, Shenzhen City, Guangdong Province Applicant before: SHENZHEN MALONG TECHNOLOGY Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181127 |
|
| RJ01 | Rejection of invention patent application after publication |