CN110147750B - Image searching method and system based on motion acceleration and electronic equipment - Google Patents
Image searching method and system based on motion acceleration and electronic equipment Download PDFInfo
- Publication number
- CN110147750B CN110147750B CN201910393254.7A CN201910393254A CN110147750B CN 110147750 B CN110147750 B CN 110147750B CN 201910393254 A CN201910393254 A CN 201910393254A CN 110147750 B CN110147750 B CN 110147750B
- Authority
- CN
- China
- Prior art keywords
- target
- frame
- tracked
- search range
- acceleration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/53—Querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
- G06V20/47—Detecting features for summarising video content
Abstract
The application relates to an image searching method and system based on motion acceleration and electronic equipment. The method comprises the following steps: step a: calculating the acceleration of the target to be tracked in the current frame image according to the displacement of the first two frames of images; step b: determining a search range rectangular frame of the target to be tracked in the current frame image according to the acceleration calculation result; step c: and extracting a candidate frame of the target to be tracked in the current frame image along the diagonal line of the search range rectangular frame through an RPN (resilient packet network), and performing feature analysis on the candidate frame to obtain the position of the target to be tracked in the current frame image. According to the method, a limited search range rectangular frame is determined by an acceleration calculation mode, the candidate frame of the tracking target is selected along the diagonal line of the search range rectangular frame, a smaller search range is determined, global search is not needed, the search range is greatly reduced, the calculation amount is reduced, and the calculation speed is improved.
Description
Technical Field
The present application relates to image search technologies, and in particular, to an image search method and system based on motion acceleration, and an electronic device.
Background
With the development of artificial intelligence technology, more and more leading-edge knowledge is falling to the ground, wherein the technology of tracking objects (targets) in videos is widely concerned by colleges and universities and business industries. Currently, for tracking a target in a video, a technical solution generally adopted is to mark a target position to be tracked in a first frame of the video, and then perform global search in each next frame to find the target to be tracked in the next frame. This is usually done in several ways:
firstly, a global image is searched in a sliding window mode [ Girshick R B, Donahue J, Darrell T, et al. Rich features Hierarchies for Accurate Object Detection and Semantic Segmentation [ J ]. computer vision and pattern recognition,2014:580-587 ], the efficiency of the searching mode is relatively low, and the deformation of an Object in the motion process cannot be overcome.
Secondly, a Region Proposing Network (RPN) is adopted to carry out [ Ren S, He K, Girshick R B, et al. fast R-CNN: Towards Real-Time Object Detection with Region proposing Network [ J ]. IEEE Transactions on Pattern Analysis and Machine Analysis, 2017,39(6):1137 and 1149 ].
As described above, the existing image search techniques are all global search, which results in long search time and relatively much computational redundancy.
Disclosure of Invention
The application provides an image searching method, an image searching system and electronic equipment based on motion acceleration, and aims to solve at least one of the technical problems in the prior art to a certain extent.
In order to solve the above problems, the present application provides the following technical solutions:
an image searching method based on motion acceleration comprises the following steps:
step a: calculating the acceleration of the target to be tracked in the current frame image according to the displacement of the first two frames of images;
step b: determining a search range rectangular frame of the target to be tracked in the current frame image according to the acceleration calculation result;
step c: and extracting a candidate frame of the target to be tracked in the current frame image along the diagonal line of the search range rectangular frame through an RPN (resilient packet network), and performing feature analysis on the candidate frame to obtain the position of the target to be tracked in the current frame image.
The technical scheme adopted by the embodiment of the application further comprises the following steps: in step a, the acceleration is in vector units, and has both a speed and a direction, and the acceleration calculation formula is as follows:
the technical scheme adopted by the embodiment of the application further comprises the following steps: in the step b, the determining, according to the acceleration calculation result, a search range rectangular frame of the target to be tracked in the current frame image specifically includes: defining the center position of the target to be tracked in the (i +1) th frame image as the intersection point of the diagonal lines of the rectangular frame of the search range
Respectively representing the horizontal and vertical coordinates of the central position of the target to be tracked, and defining the initial search origin of the next frame, namely the i +2 frame, as follows:
the starting point of the search range rectangular box of the i +2 frame is
The length and width of the search range rectangular frame are respectively as follows:
widthi+2＝2*widthi+1，heighti+2＝2*heighti+2。
the technical scheme adopted by the embodiment of the application further comprises the following steps: in the step c, the extracting, by the RPN network, the candidate frame of the target to be tracked in the current frame image along the diagonal line of the search range rectangular frame specifically includes: and obtaining three points on the diagonal lines of the search range rectangular frame according to preset spacing distances, and then performing rescaling according to three set length-width scale ratios to obtain nine candidate frames.
The technical scheme adopted by the embodiment of the application further comprises the following steps: in the step a, the first two frames of images are continuous two frames of images, two frames of images at discrete intervals, or two frames of images at any time.
Another technical scheme adopted by the embodiment of the application is as follows: an image search system based on motion acceleration, comprising:
an acceleration calculation module: the system is used for calculating the acceleration of the target to be tracked in the current frame image according to the displacement of the first two frames of images;
a search range calculation module: the search range rectangular frame used for determining the target to be tracked in the current frame image according to the acceleration calculation result;
a candidate frame extraction module: the candidate frame of the target to be tracked in the current frame image is extracted along the diagonal line of the search range rectangular frame through an RPN network;
a target retrieval module: and the candidate frame is subjected to feature analysis to obtain the position of the target to be tracked in the current frame image.
The technical scheme adopted by the embodiment of the application further comprises the following steps: the acceleration is a vector unit, and has both speed and direction, and the acceleration calculation formula is as follows:
the technical scheme adopted by the embodiment of the application further comprises the following steps: the search range calculation module determines a search range rectangular frame of the target to be tracked in the current frame image according to the acceleration calculation result, and the search range rectangular frame specifically comprises: defining the center position of the target to be tracked in the (i +1) th frame image as the intersection point of the diagonal lines of the rectangular frame of the search range
Respectively representing the horizontal and vertical coordinates of the central position of the target to be tracked, and defining the initial search origin of the next frame, namely the i +2 frame, as follows:
the starting point of the search range rectangular box of the i +2 frame is
The length and width of the search range rectangular frame are respectively as follows:
widthi+2＝2*widthi+1，heighti+2＝2*heighti+2。
the technical scheme adopted by the embodiment of the application further comprises the following steps: the candidate frame extracting module extracts a candidate frame of the target to be tracked in the current frame image along the diagonal line of the search range rectangular frame through the RPN network, and specifically comprises the following steps: and obtaining three points on the diagonal lines of the search range rectangular frame according to preset spacing distances, and then performing rescaling according to three set length-width scale ratios to obtain nine candidate frames.
The technical scheme adopted by the embodiment of the application further comprises the following steps: the first two frames of images are continuous two frames of images, two frames of images at discrete intervals or two frames of images at any time.
The embodiment of the application adopts another technical scheme that: an electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the one processor to cause the at least one processor to perform the following operations of the above-described motion acceleration-based image search method:
step a: calculating the acceleration of the target to be tracked in the current frame image according to the displacement of the first two frames of images;
step b: determining a search range rectangular frame of the target to be tracked in the current frame image according to the acceleration calculation result;
step c: and extracting a candidate frame of the target to be tracked in the current frame image along the diagonal line of the search range rectangular frame through an RPN (resilient packet network), and performing feature analysis on the candidate frame to obtain the position of the target to be tracked in the current frame image.
Compared with the prior art, the embodiment of the application has the advantages that: compared with the prior art, the image searching method and system based on the motion acceleration and the electronic equipment determine a smaller searching range by utilizing a limited searching range rectangular frame determined by an acceleration calculation mode and selecting a candidate frame for tracking the target along three points on the diagonal line of the searching range rectangular frame, so that the searching range is greatly reduced, the calculated amount is reduced and the calculating speed is improved.
Drawings
FIG. 1 is a flowchart of an image search method based on motion acceleration according to an embodiment of the present application;
FIG. 2(a) is a target frame of an i-th frame of a target to be tracked, and FIG. 2(b) is a target frame of an (i +1) -th frame of the target to be tracked;
FIG. 3 is a schematic view of an acceleration calculation method under the same shooting picture (with the camera fixed);
FIG. 4 is a rectangular block diagram of the i +2 frame search range;
fig. 5 is a schematic diagram of a generation rule of an RPN network according to an embodiment of the present application;
FIG. 6 is a schematic structural diagram of an image search system based on motion acceleration according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a hardware device of an image search method based on motion acceleration according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
Please refer to fig. 1, which is a flowchart illustrating an image searching method based on motion acceleration according to an embodiment of the present application. The image searching method based on the motion acceleration comprises the following steps:
step 100: marking the position of a target to be tracked in a first frame image of the beginning of a video;
step 200: calculating the acceleration and possible direction of the target to be tracked in the current frame image according to the displacement of the previous two frames of images;
in step 200, as shown in fig. 2, fig. 2(a) is a target frame of the target to be tracked in the ith frame, and fig. 2(b) is a target frame of the target to be tracked in the (i +1) th frame. The target tracking process in the figure is abstracted into a mathematical form, namely, the target tracking process can be represented as shown in figure 3, and is an acceleration calculation mode under the same shooting picture (the camera is fixed). In the embodiment of the application, the acceleration has the same property as the acceleration in physics, and is vector unit, namely, the speed and the direction. The acceleration calculation formula is specifically as follows:
it can be understood that the present application is not limited to determining the acceleration of the third frame according to the displacement of the two consecutive previous frames of images, and the acceleration of the current frame can be calculated by using the displacement of the target in the frames at discrete intervals or two frames of images at any time.
Step 300: determining a search range rectangular frame of the target to be tracked in the current frame image according to the acceleration and direction calculation result;
in step 300, the calculation method of the search range rectangular frame specifically includes: defining the center position of the target to be tracked in the (i +1) th frame as the intersection point of the diagonal lines of the rectangular frame of the search range
Respectively representing the horizontal and vertical coordinates of the central position of the target to be tracked, and defining the initial search origin of the next frame, namely the i +2 frame, as follows:
the formulas (2) and (3) determine the lower left origin position of the initial search of the i +2 frame, and the starting point of the rectangular frame of the search range of the i +2 frame is
The length and width of the rectangular frame of the search range are respectively as follows:
widthi+2＝2*widthi+1，heighti+2＝2*heighti+2 (4)
the above process is drawn as shown in fig. 4, which is a rectangular frame diagram of the i +2 frame search range. As shown in the figure, the actual detection range of the i +2 frame is changed from the whole picture of the traditional algorithm to a rectangular frame in the upper right picture (for clear display, the picture size of the i +2 frame is enlarged, the size of the whole picture is practically unchanged all the time, and only the rectangular frame in the search range is changed), so that the calculation amount of target search is reduced.
It will be appreciated that because the center and four points of the congruent quadrilateral can each determine a unique rectangular box, the starting search origin for the next frame can be determined whether by the origin of coordinates or by the four boundary vertices of the previous frame.
Step 400: three points are taken along the diagonal line of the rectangular frame of the search range according to a set interval distance through an RPN, and 9 candidate frames of the target to be tracked in the current frame image are extracted according to three length-width ratios respectively;
in step 400, the conventional detection method is to perform operations of keeping the original size, scaling the original image by 0.5, and enlarging the original image by 2 times on each selected center position of the image in the whole image, and then perform changes with aspect ratios of 1:1, 1:2, and 2:1 on the sizes of the three images. So each center point position can find 3 × 3 candidate boxes for selection. In order to save calculation power and increase speed, the method does not adopt the candidate frames with three scales, namely, the operations of keeping the original size of the original picture unchanged, scaling the original picture by 0.5 and enlarging the original picture by 2 times are not carried out, but three points on the diagonal line of the rectangular frame of the search range are adopted to select the candidate frames.
Step 500: and performing characteristic analysis on the 9 candidate frames to obtain the position of the target to be tracked in the current frame image.
Specifically, please refer to fig. 5, which is a schematic diagram illustrating candidate box generation rules according to an embodiment of the present application. Nine blocks in fig. 5 are candidate blocks of the generated target to be tracked. The nine frames are obtained by uniformly amplifying the candidate frames with fixed sizes by 1.25 times and then extracting the candidate frames according to three different aspect ratios. D in fig. 5 is the diagonal diameter length of the rectangular frame in the search range, three points are obtained on the diagonal according to the interval distances of 0.25, 0.5 and 0.75, and then the scaling is performed again according to the three aspect ratios of 1:1, 1:2 and 2:1, so as to obtain nine candidate frames. The existing RPN network detects and compares 9N candidate frames with N central points on all pictures, but the method only needs to detect the 9 candidate frames after determining a search range rectangular frame, so that the retrieval range is greatly reduced. It can be understood that parameters such as the spacing distance of the points on the diagonal line and the scaling aspect ratio can be set according to actual operation.
In addition, if the candidate frame exceeds the search range rectangular frame after being zoomed, the pixel value or the characteristic value of the position of the previous frame of picture is reserved at the step until the selected candidate frame can accurately capture all the search range rectangular frames.
Please refer to fig. 6, which is a schematic structural diagram of an image search system based on motion acceleration according to an embodiment of the present application. The image searching system based on the motion acceleration comprises a position marking module, an acceleration calculating module, a searching range calculating module, a candidate frame extracting module and a target retrieving module.
A position marking module: the tracking device is used for marking the position of a target to be tracked in a first frame image of the beginning of a video;
an acceleration calculation module: the system is used for calculating the acceleration and the possible direction of the target to be tracked in the current frame image according to the displacement of the previous two frames of images; specifically, as shown in fig. 2, fig. 2(a) is a target frame of the target to be tracked in the ith frame, and fig. 2(b) is a target frame of the target to be tracked in the (i +1) th frame. The target tracking process in the figure is abstracted into a mathematical form, namely, the target tracking process can be represented as shown in figure 3, and is an acceleration calculation mode under the same shooting picture (the camera is fixed). In the embodiment of the application, the acceleration has the same property as the acceleration in physics, and is vector unit, namely, the speed and the direction. The acceleration calculation formula is specifically as follows:
it can be understood that the present application is not limited to determining the acceleration of the third frame according to the displacement of the first two frames of the image, and the acceleration can be calculated by using the displacement of the target in the frames at discrete intervals or the two frames of the image at any time.
A search range calculation module: the search range rectangular frame is used for determining a search range rectangular frame of the target to be tracked in the current frame image according to the acceleration and direction calculation result; the calculation mode of the search range rectangular frame is specifically as follows: defining the center position of the target to be tracked in the (i +1) th frame as the intersection point of the diagonal lines of the rectangular frame of the search range
Respectively representing the horizontal and vertical coordinates of the central position of the target to be tracked, and defining the initial search origin of the next frame, namely the i +2 frame, as follows:
the formulas (2) and (3) determine the lower left origin position of the initial search of the i +2 frame, and the starting point of the rectangular frame of the search range of the i +2 frame is
The length and width of the rectangular frame are respectively as follows:
widthi+2＝2*widthi+1，heighti+2＝2*heighti+2 (4)
the above process is drawn as shown in fig. 4, which is a rectangular frame diagram of the i +2 frame search range. As shown in the figure, the actual detection range of the i +2 frame is changed from the whole picture of the traditional algorithm to a search range rectangular frame in the upper right picture (for clear display, the picture size of the i +2 frame is enlarged, the size of the whole picture is practically unchanged all the time, and only the search range rectangular frame is changed), so that the calculation amount of target search is reduced.
It will be appreciated that because the center and four points of the congruent quadrilateral can each determine a unique rectangular box, the starting search origin for the next frame can be determined whether by the origin of coordinates or by the four boundary vertices of the previous frame.
A candidate frame extraction module: the method comprises the steps of obtaining three points along a diagonal line of a rectangular frame of a search range according to a set interval distance through an RPN network, and extracting 9 candidate frames of a target to be tracked in a current frame image according to three length-width ratios; in order to save calculation power and increase speed, the method and the device do not adopt the candidate frames with three scales, namely, the operations of keeping the original size of the original picture unchanged, scaling the original picture by 0.5 and enlarging the original picture by 2 times are not carried out, but three points on the diagonal line of the rectangular frame of the search range are adopted to select the candidate frames.
Specifically, please refer to fig. 5, which is a schematic diagram illustrating a generation rule of an RPN network according to an embodiment of the present application. Nine frames in the figure are generated candidate frames of the target to be tracked. The nine frames are obtained by uniformly amplifying the candidate frames with fixed sizes by 1.25 times and then extracting the candidate frames according to three different aspect ratios. D in fig. 4 is the diagonal diameter length of the rectangular frame in the search range, three points are obtained on the diagonal according to the interval distances of 0.25, 0.5 and 0.75, and then the scaling is performed again according to the three aspect ratios of 1:1, 1:2 and 2:1, so as to obtain nine candidate frames. The existing RPN network detects and compares 9N candidate frames with N central points on all pictures, but the method only needs to detect the 9 candidate frames after determining a search range rectangular frame, so that the retrieval range is greatly reduced. It can be understood that parameters such as the spacing distance of the points on the diagonal line and the scaling aspect ratio can be set according to actual operation.
In addition, if the candidate frame exceeds the search range rectangular frame after being zoomed, the pixel value or the characteristic value of the position of the previous frame of picture is reserved at the step until the selected candidate frame can accurately capture all the search range rectangular frames.
A target retrieval module: and the method is used for carrying out feature analysis on the 9 candidate frames to obtain the position of the target to be tracked in the current frame image.
Fig. 7 is a schematic structural diagram of a hardware device of an image search method based on motion acceleration according to an embodiment of the present application. As shown in fig. 7, the apparatus includes one or more processors and memory. Taking a processor as an example, the apparatus may further include: an input system and an output system.
The processor, memory, input system, and output system may be connected by a bus or other means, as exemplified by the bus connection in fig. 7.
The memory, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules. The processor executes various functional applications and data processing of the electronic device, i.e., implements the processing method of the above-described method embodiment, by executing the non-transitory software program, instructions and modules stored in the memory.
The memory may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data and the like. Further, the memory may include high speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory optionally includes memory located remotely from the processor, and these remote memories may be connected to the processing system over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input system may receive input numeric or character information and generate a signal input. The output system may include a display device such as a display screen.
The one or more modules are stored in the memory and, when executed by the one or more processors, perform the following for any of the above method embodiments:
step a: calculating the acceleration of the target to be tracked in the current frame image according to the displacement of the first two frames of images;
step b: determining a search range rectangular frame of the target to be tracked in the current frame image according to the acceleration calculation result;
step c: and extracting a candidate frame of the target to be tracked in the current frame image along the diagonal line of the search range rectangular frame through an RPN (resilient packet network), and performing feature analysis on the candidate frame to obtain the position of the target to be tracked in the current frame image.
The product can execute the method provided by the embodiment of the application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in this embodiment, reference may be made to the methods provided in the embodiments of the present application.
Embodiments of the present application provide a non-transitory (non-volatile) computer storage medium having stored thereon computer-executable instructions that may perform the following operations:
step a: calculating the acceleration of the target to be tracked in the current frame image according to the displacement of the first two frames of images;
step b: determining a search range rectangular frame of the target to be tracked in the current frame image according to the acceleration calculation result;
step c: and extracting a candidate frame of the target to be tracked in the current frame image along the diagonal line of the search range rectangular frame through an RPN (resilient packet network), and performing feature analysis on the candidate frame to obtain the position of the target to be tracked in the current frame image.
Embodiments of the present application provide a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions that, when executed by a computer, cause the computer to perform the following:
step a: calculating the acceleration of the target to be tracked in the current frame image according to the displacement of the first two frames of images;
step b: determining a search range rectangular frame of the target to be tracked in the current frame image according to the acceleration calculation result;
step c: and extracting a candidate frame of the target to be tracked in the current frame image along the diagonal line of the search range rectangular frame through an RPN (resilient packet network), and performing feature analysis on the candidate frame to obtain the position of the target to be tracked in the current frame image.
Compared with the prior art, the image searching method and system based on the motion acceleration and the electronic equipment determine a smaller searching range by utilizing a limited searching range rectangular frame determined by an acceleration calculation mode and selecting a candidate frame for tracking the target along three points on the diagonal line of the searching range rectangular frame, so that the searching range is greatly reduced, the calculated amount is reduced and the calculating speed is improved.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (7)
1. An image searching method based on motion acceleration is characterized by comprising the following steps:
step a: calculating the acceleration of the target to be tracked in the current frame image according to the displacement of the first two frames of images;
step b: determining a search range rectangular frame of the target to be tracked in the current frame image according to the acceleration calculation result;
step c: extracting a candidate frame of the target to be tracked in the current frame image along the diagonal line of the search range rectangular frame through an RPN (resilient packet network), and performing feature analysis on the candidate frame to obtain the position of the target to be tracked in the current frame image;
in step a, the acceleration is a vector unit, and has both magnitude and direction, and the acceleration expression is:
in the step c, the extracting, by the RPN network, the candidate frame of the target to be tracked in the current frame image along the diagonal line of the search range rectangular frame specifically includes: obtaining three points on diagonal lines of the search range rectangular frame according to preset spacing distances, and then performing secondary scaling according to three set length-width scale ratios to obtain nine candidate frames; the three points are respectively located at 0.25 times, 0.5 times and 0.75 times of the length of the diagonal line; the three length and width dimensions are respectively as follows: 1:1, 1:2 and 2: 1.
2. The image searching method based on motion acceleration according to claim 1, wherein in the step b, the determining a search range rectangular frame of the target to be tracked in the current frame image according to the acceleration calculation result specifically comprises: centering the target to be tracked in the i +1 th frame imageDefining the center position as the intersection point of the diagonal lines of the rectangular frame of the search range
Respectively representing the horizontal and vertical coordinates of the central position of the target to be tracked, and defining the initial search origin of the next frame, namely the i +2 frame, as follows:
the starting point of the search range rectangular box of the i +2 frame is
The length and width of the search range rectangular frame are respectively as follows:
widthi+2＝2*widthi+1,heighti+2＝2*heighti+1。
3. the image searching method based on motion acceleration according to any one of claims 1 to 2, wherein in the step a, the first two frames of images are continuous two frames of images or two frames of images at discrete intervals.
4. An image search system based on motion acceleration, comprising:
an acceleration calculation module: the system is used for calculating the acceleration of the target to be tracked in the current frame image according to the displacement of the first two frames of images;
a search range calculation module: the search range rectangular frame used for determining the target to be tracked in the current frame image according to the acceleration calculation result;
a candidate frame extraction module: the candidate frame of the target to be tracked in the current frame image is extracted along the diagonal line of the search range rectangular frame through an RPN network;
a target retrieval module: the candidate frame is subjected to feature analysis to obtain the position of the target to be tracked in the current frame image;
the acceleration is a vector unit and has both magnitude and direction, and the acceleration expression is as follows:
the candidate frame extracting module extracts a candidate frame of the target to be tracked in the current frame image along the diagonal line of the search range rectangular frame through the RPN network, and specifically comprises the following steps: obtaining three points on diagonal lines of the search range rectangular frame according to preset spacing distances, and then performing secondary scaling according to three set length-width scale ratios to obtain nine candidate frames; the three points are respectively located at 0.25 times, 0.5 times and 0.75 times of the length of the diagonal line; the three length and width dimensions are respectively as follows: 1:1, 1:2 and 2: 1.
5. The image search system based on motion acceleration according to claim 4, wherein the determining, by the search range calculation module, the search range rectangular frame of the target to be tracked in the current frame image according to the acceleration calculation result specifically includes: defining the center position of the target to be tracked in the (i +1) th frame image as the intersection point of the diagonal lines of the rectangular frame of the search range
Respectively representing the horizontal and vertical coordinates of the central position of the target to be tracked, and defining the initial search origin of the next frame, namely the i +2 frame, as follows:
the starting point of the search range rectangular box of the i +2 frame is
The length and width of the search range rectangular frame are respectively as follows:
widthi+2＝2*widthi+1,heighti+2＝2*heighti+1。
6. the image search system based on motion acceleration according to any one of claims 4 to 5, characterized in that the first two frames of images are continuous two frames of images or two frames of images at discrete intervals.
7. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the following operations of the motion acceleration-based image search method of any one of items 1 to 3 above:
step a: calculating the acceleration of the target to be tracked in the current frame image according to the displacement of the first two frames of images;
step b: determining a search range rectangular frame of the target to be tracked in the current frame image according to the acceleration calculation result;
step c: extracting a candidate frame of the target to be tracked in the current frame image along the diagonal line of the search range rectangular frame through an RPN (resilient packet network), and performing feature analysis on the candidate frame to obtain the position of the target to be tracked in the current frame image;
in step a, the acceleration is a vector unit, and has both magnitude and direction, and the acceleration expression is:
in the step c, the extracting, by the RPN network, the candidate frame of the target to be tracked in the current frame image along the diagonal line of the search range rectangular frame specifically includes: obtaining three points on diagonal lines of the search range rectangular frame according to preset spacing distances, and then performing secondary scaling according to three set length-width scale ratios to obtain nine candidate frames; the three points are respectively located at 0.25 times, 0.5 times and 0.75 times of the length of the diagonal line; the three length and width dimensions are respectively as follows: 1:1, 1:2 and 2: 1.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910393254.7A CN110147750B (en) | 2019-05-13 | 2019-05-13 | Image searching method and system based on motion acceleration and electronic equipment |
| PCT/CN2019/130538 WO2020228353A1 (en) | 2019-05-13 | 2019-12-31 | Motion acceleration-based image search method, system, and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910393254.7A CN110147750B (en) | 2019-05-13 | 2019-05-13 | Image searching method and system based on motion acceleration and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110147750A CN110147750A (en) | 2019-08-20 |
| CN110147750B true CN110147750B (en) | 2021-08-24 |
Family
ID=67595163
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910393254.7A Active CN110147750B (en) | 2019-05-13 | 2019-05-13 | Image searching method and system based on motion acceleration and electronic equipment |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110147750B (en) |
| WO (1) | WO2020228353A1 (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110147750B (en) * | 2019-05-13 | 2021-08-24 | 深圳先进技术研究院 | Image searching method and system based on motion acceleration and electronic equipment |
| CN112800811B (en) * | 2019-11-13 | 2023-10-13 | 深圳市优必选科技股份有限公司 | Color block tracking method and device and terminal equipment |
| CN111008305B (en) | 2019-11-29 | 2023-06-23 | 百度在线网络技术（北京）有限公司 | Visual search method and device and electronic equipment |
| CN113177918B (en) * | 2021-04-28 | 2022-04-19 | 上海大学 | Intelligent and accurate inspection method and system for electric power tower by unmanned aerial vehicle |
| CN116205914B (en) * | 2023-04-28 | 2023-07-21 | 山东中胜涂料有限公司 | Waterproof coating production intelligent monitoring system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102915552A (en) * | 2012-09-18 | 2013-02-06 | 中国科学院计算技术研究所 | Controllable flame animation generation method and system |
| CN106403976A (en) * | 2016-08-30 | 2017-02-15 | 哈尔滨航天恒星数据系统科技有限公司 | Dijkstra optimal traffic path planning method and system based on impedance matching |
| CN106604035A (en) * | 2017-01-22 | 2017-04-26 | 北京君泊网络科技有限责任公司 | Motion estimation method for video encoding and compression |
| CN106877237A (en) * | 2017-03-16 | 2017-06-20 | 天津大学 | A kind of method of insulator missing in detection transmission line of electricity based on Aerial Images |
| CN107180435A (en) * | 2017-05-09 | 2017-09-19 | 杭州电子科技大学 | A kind of human body target tracking method suitable for depth image |
| CN109635821A (en) * | 2018-12-04 | 2019-04-16 | 北京字节跳动网络技术有限公司 | Feature extracting method, device, equipment and the readable medium of image-region |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104933064B (en) * | 2014-03-19 | 2018-02-23 | 株式会社理光 | The method and apparatus for predicting the kinematic parameter of destination object |
| CN106373143A (en) * | 2015-07-22 | 2017-02-01 | 中兴通讯股份有限公司 | Adaptive method and system |
| CN105678808A (en) * | 2016-01-08 | 2016-06-15 | 浙江宇视科技有限公司 | Moving object tracking method and device |
| CN107346538A (en) * | 2016-05-06 | 2017-11-14 | 株式会社理光 | Method for tracing object and equipment |
| CN109559330B (en) * | 2017-09-25 | 2021-09-10 | 北京金山云网络技术有限公司 | Visual tracking method and device for moving target, electronic equipment and storage medium |
| CN108363998A (en) * | 2018-03-21 | 2018-08-03 | 北京迈格威科技有限公司 | A kind of detection method of object, device, system and electronic equipment |
| CN110147750B (en) * | 2019-05-13 | 2021-08-24 | 深圳先进技术研究院 | Image searching method and system based on motion acceleration and electronic equipment |
-
2019
- 2019-05-13 CN CN201910393254.7A patent/CN110147750B/en active Active
- 2019-12-31 WO PCT/CN2019/130538 patent/WO2020228353A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102915552A (en) * | 2012-09-18 | 2013-02-06 | 中国科学院计算技术研究所 | Controllable flame animation generation method and system |
| CN106403976A (en) * | 2016-08-30 | 2017-02-15 | 哈尔滨航天恒星数据系统科技有限公司 | Dijkstra optimal traffic path planning method and system based on impedance matching |
| CN106604035A (en) * | 2017-01-22 | 2017-04-26 | 北京君泊网络科技有限责任公司 | Motion estimation method for video encoding and compression |
| CN106877237A (en) * | 2017-03-16 | 2017-06-20 | 天津大学 | A kind of method of insulator missing in detection transmission line of electricity based on Aerial Images |
| CN107180435A (en) * | 2017-05-09 | 2017-09-19 | 杭州电子科技大学 | A kind of human body target tracking method suitable for depth image |
| CN109635821A (en) * | 2018-12-04 | 2019-04-16 | 北京字节跳动网络技术有限公司 | Feature extracting method, device, equipment and the readable medium of image-region |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110147750A (en) | 2019-08-20 |
| WO2020228353A1 (en) | 2020-11-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110147750B (en) | Image searching method and system based on motion acceleration and electronic equipment | |
| US11062123B2 (en) | Method, terminal, and storage medium for tracking facial critical area | |
| CN108257146B (en) | Motion trail display method and device | |
| CN111047626B (en) | Target tracking method, device, electronic equipment and storage medium | |
| CN108986152B (en) | Foreign matter detection method and device based on difference image | |
| Tau et al. | Dense correspondences across scenes and scales | |
| JP7273129B2 (en) | Lane detection method, device, electronic device, storage medium and vehicle | |
| CN110866497B (en) | Robot positioning and mapping method and device based on dotted line feature fusion | |
| CN111640089A (en) | Defect detection method and device based on feature map center point | |
| CN111310759B (en) | Target detection inhibition optimization method and device for dual-mode cooperation | |
| CN112967341A (en) | Indoor visual positioning method, system, equipment and storage medium based on live-action image | |
| US20220172376A1 (en) | Target Tracking Method and Device, and Electronic Apparatus | |
| CN103440667A (en) | Automatic device for stably tracing moving targets under shielding states | |
| CN113850136A (en) | Yolov5 and BCNN-based vehicle orientation identification method and system | |
| CN113112542A (en) | Visual positioning method and device, electronic equipment and storage medium | |
| CN111177811A (en) | Automatic fire point location layout method applied to cloud platform | |
| CN103744903A (en) | Sketch based scene image retrieval method | |
| Ni et al. | An improved kernelized correlation filter based visual tracking method | |
| CN110956131B (en) | Single-target tracking method, device and system | |
| CN111192312A (en) | Depth image acquisition method, device, equipment and medium based on deep learning | |
| CN105608423A (en) | Video matching method and device | |
| CN114429631B (en) | Three-dimensional object detection method, device, equipment and storage medium | |
| CN113361371B (en) | Road extraction method, device, equipment and storage medium | |
| CN115063473A (en) | Object height detection method and device, computer equipment and storage medium | |
| CN114648556A (en) | Visual tracking method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |