WO2020258499A1 - 物体检测方法、装置、计算机设备和存储介质 - Google Patents

物体检测方法、装置、计算机设备和存储介质 Download PDF

Info

Publication number
WO2020258499A1
WO2020258499A1 PCT/CN2019/103195 CN2019103195W WO2020258499A1 WO 2020258499 A1 WO2020258499 A1 WO 2020258499A1 CN 2019103195 W CN2019103195 W CN 2019103195W WO 2020258499 A1 WO2020258499 A1 WO 2020258499A1
Authority
WO
WIPO (PCT)
Prior art keywords
detection
frame
network
offset
feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/103195
Other languages
English (en)
French (fr)
Inventor
叶明�
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Publication of WO2020258499A1 publication Critical patent/WO2020258499A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames

Definitions

  • This application relates to an object detection method, device, computer equipment and storage medium.
  • Object detection is one of the classic problems in computer vision. Its task is to mark the position of the object with a box and give the object category. Therefore, object detection is widely used in many aspects such as face recognition, security monitoring and dynamic tracking.
  • an object detection method, device, computer equipment, and storage medium are provided.
  • An object detection method including:
  • the server receives a video to be detected and an object detection instruction sent by the terminal, where the video to be detected includes video frames;
  • the server determines a detection mode according to the object detection instruction, and the detection mode includes fast detection and precision detection;
  • the server inputs each video frame into a pre-trained single-frame network or feature offset network for object detection according to the key frame flag of each video frame , To perform object detection on the video frame by using the single-frame network or the feature offset network to obtain a detection result;
  • the server inputs each of the video frames into the single-frame network and the feature offset network, so as to use the single-frame network to perform Object detection, and using the feature offset network to perform multiple object detection on each of the video frames according to the playback sequence of each of the video frames;
  • the detection result includes the detection feature map and the feature point offset value;
  • the server offsets the feature points in the corresponding detection feature map according to the feature point offset value to obtain the offset feature map, and determines object information according to the detection feature map and the coordinate frame in the offset feature map .
  • An object detection device including:
  • a receiving module configured to receive a video to be detected and an object detection instruction sent by a terminal, where the video to be detected includes video frames;
  • a determining module configured to determine a detection mode according to the object detection instruction, the detection mode including rapid detection and precision detection;
  • the detection module is used to input each video frame into a pre-trained single-frame network or feature bias for object detection according to the key frame flag of each video frame when the detection method is the fast detection
  • a mobile network to perform object detection on the video frame using the single-frame network or feature offset network to obtain a detection result
  • the detection module is further configured to input each of the video frames into the single-frame network and the feature offset network when the detection method is the accuracy detection, so as to use the single-frame network to perform the Frame object detection, and using the feature offset network to perform multiple object detection on each video frame according to the playback sequence of each video frame; the detection result includes a detection feature map and a feature point offset value; and
  • the offset module is configured to input each of the video frames into a pre-trained single-frame network and feature offset network for object detection according to the feature point offset value, and according to the detected feature map and offset feature
  • the coordinate box in the figure determines the object information.
  • a computer device including a memory and one or more processors, the memory stores computer readable instructions, when the computer readable instructions are executed by the processor, the one or more processors execute The following steps:
  • the detection mode including fast detection and precision detection
  • the detection method is the fast detection
  • each of the video frames is input into the single frame network and the feature offset network respectively, so as to use the single frame network to perform object detection on each of the video frames, And using the feature offset network to perform multiple object detection on each of the video frames according to the playback sequence of each of the video frames; the detection result includes the detection feature map and the feature point offset value; and
  • the feature points in the corresponding detection feature map are offset according to the feature point offset value to obtain the offset feature map, and the object information is determined according to the detection feature map and the coordinate frame in the offset feature map.
  • One or more non-volatile storage media storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors perform the following steps:
  • the detection mode including fast detection and precision detection
  • the detection method is the fast detection
  • each of the video frames is input into the single frame network and the feature offset network respectively, so as to use the single frame network to perform object detection on each of the video frames, And using the feature offset network to perform multiple object detection on each of the video frames according to the playback sequence of each of the video frames; the detection result includes the detection feature map and the feature point offset value; and
  • the feature points in the corresponding detection feature map are offset according to the feature point offset value to obtain the offset feature map, and the object information is determined according to the detection feature map and the coordinate frame in the offset feature map.
  • Fig. 1 is an application scenario diagram of an object detection method according to one or more embodiments.
  • Fig. 2 is a schematic flowchart of an object detection method according to one or more embodiments.
  • FIG. 3 is a schematic flow diagram of the steps of detecting each video frame by using a preset single-frame network and a feature offset network to obtain a detection result based on a detection method in one or more embodiments.
  • Fig. 4 is a schematic flow diagram of a step of detecting each video frame based on a detection method using a preset single-frame network and a feature offset network in another embodiment to obtain a detection result.
  • Fig. 5 is a block diagram of an object detection device according to one or more embodiments.
  • Figure 6 is a block diagram of a computer device according to one or more embodiments.
  • the object detection method provided in this application can be applied to the application environment as shown in FIG. 1.
  • the application environment involves a terminal 102 and a server 104, and the terminal 102 communicates with the server 104 through a network.
  • the terminal 102 may independently implement the above object detection method.
  • the terminal 102 may also send the video to be detected and the object detection instruction to the server 104, and the server 104 implements the object detection method.
  • the server 104 receives the video to be detected and the object detection instruction sent by the terminal 102, and the video to be detected includes video frames.
  • the server 104 determines the detection mode according to the object detection instruction, and the detection mode includes fast detection and precision detection.
  • the server 104 When the detection method is rapid detection, the server 104 inputs each video frame into a pre-trained single-frame network or feature offset network for object detection according to the key frame flag of each video frame, so as to use the single-frame network or feature offset network.
  • the mobile network performs object detection on the video frame to obtain the detection result; when the detection method is precision detection, the server 104 inputs each video frame into the single-frame network and the feature offset network respectively, so as to use the single-frame network to perform object detection on each video frame , And use the feature offset network to perform multiple object detection on each video frame according to the playback sequence of each video frame; the detection result includes the detection feature map and the feature point offset value; the server 104 detects the corresponding object according to the feature point offset value
  • the feature points in the feature map are offset to obtain the offset feature map, and the object information is determined according to the detection feature map and the coordinate frame in the offset feature map.
  • the terminal 102 may be, but is not limited to, various personal computers, notebook computers, smart phones
  • an object detection method is provided. Taking the method applied to the server in FIG. 1 as an example for description, the method includes the following steps:
  • Step S202 Receive a video to be detected and an object detection instruction sent by the terminal, and the video to be detected includes video frames.
  • the video to be detected refers to the video that needs object detection, and the video frame is the video image used to compose the video.
  • the video to be detected includes but is not limited to surveillance video of surveillance equipment.
  • the object detection instruction refers to an instruction used to request the server to perform object detection.
  • the user terminal when the user has an object detection requirement, the user terminal is operated to issue an object detection instruction to the server, and the object detection instruction to be detected is sent to the server along with the object detection instruction.
  • Step S204 Determine a detection mode according to the object detection instruction, the detection mode includes fast detection and precision detection.
  • the detection methods include fast detection and precision detection, and the method of object detection is determined by different object detection.
  • the terminal is operated to initiate an object detection task and select a corresponding detection method, and the terminal generates a corresponding object detection instruction according to the user's operation. That is to say, if the requirement of the object detection task is to obtain the detection result quickly, the user will select the fast detection method synchronously on the terminal page when the user initiates the object detection task on the operating terminal.
  • the terminal receives the user’s selection of the fast detection method,
  • the selected fast detection method generates a fast object detection command, and the generated object detection command is a fast object detection command.
  • the server When the server receives the fast object detection command, it can respond to the fast object detection command to determine that the detection mode is fast detection, and then perform fast object detection based on the fast detection mode. Similarly, when the user needs high-precision object detection results, the accuracy detection mode is selected synchronously when the object detection instruction is issued, and the generated object detection instruction is the accuracy object detection instruction.
  • the server receives the precision object detection instruction, it can respond to the precision object detection instruction to perform high-precision object detection.
  • the terminal when the user operates the terminal to initiate an object detection task and select a corresponding detection method, the terminal can directly generate an object detection instruction, and send the method selected by the user to the server along with the object detection instruction, that is, the object detection instruction includes the detection method.
  • the server determines the detection mode included in the object detection command by judging the object detection instruction, thereby determining the detection mode of the object detection.
  • Step S206 When the detection method is fast detection, input each video frame into a pre-trained single-frame network or feature offset network for object detection according to the key frame flag of each video frame to use the single-frame network or feature The offset network performs object detection on the video frame and obtains the detection result.
  • Step S208 When the detection method is precision detection, each video frame is input into the single-frame network and the feature offset network to use the single-frame network to perform object detection on each video frame, and the feature offset network is used according to the The playback sequence performs multiple object detection on each video frame; the detection result includes the detection feature map and the feature point offset value.
  • the single-frame network and the feature migration network are neural networks that are pre-trained based on video samples and used for object detection.
  • Video samples refer to video frames of labeled objects.
  • the single-frame network is a traditional frame-by-frame detection algorithm network, which can be any existing one, such as a faster rcnn detection network, a yolov3 detection network, etc.
  • Feature offset network is a target detection network similar to optical flow network, using flownet as the network infrastructure. Compared with the traditional optical flow network, the output layer of the feature offset network is changed from calculating the offset value of each pixel in the image to only calculating the offset value of each pixel in the feature map.
  • the detection result obtained by the single-frame network is the detection feature map
  • the detection result obtained by the feature offset network is the feature point offset value. Based on different detection methods, each video frame is input to the pre-trained single-frame network and feature offset network for object detection.
  • each video frame input to the single-frame network and the feature offset network is different.
  • the detection method is fast detection, it is necessary to divide all video frames into key frames and non-key frames according to the key frame flag bits corresponding to each video frame.
  • Video frames belonging to key frames are detected using a single frame network, and video frames belonging to non-key frames are detected using a feature offset network.
  • the detection method is precision detection, all video frames need to be detected by the single-frame network and the feature offset network at the same time, that is, all video frames need to be input to the feature offset network in addition to the single-frame network for detection.
  • Mobile network for detection for detection.
  • the feature offset network will perform multiple object detections on each video frame according to the playback order of each video frame, that is, once according to the positive playback order of each video frame, and once according to the reverse playback order of each video frame.
  • Step S210 Offset the feature points in the corresponding detection feature map according to the feature point offset value to obtain the offset feature map, and determine the object information according to the detection feature map and the coordinate frame in the offset feature map.
  • the detection feature map is the feature map output by the single-frame network detecting the video frame
  • the feature point offset value is the feature offset network detecting the video frame
  • the feature points in the video frame obtained are relative to the previous video frame
  • the offset value of the feature points in that is, the degree of change between the corresponding feature points. That is, the degree of object shift in the video, that is, the degree of movement change of the object, can be obtained through the offset value of the feature points between adjacent video frames.
  • Object information refers to the category to which the detected object belongs.
  • the feature map output by the network model includes the coordinate frame of the detected object, and the object information can be directly determined according to the coordinate frame. That is, when the video frame belongs to the key frame, that is, the single-frame network is used to detect the video frame, and the detection feature map corresponding to the video frame is obtained, then the coordinate box on the detection feature map output by the single-frame detection network is directly Determine object information.
  • the feature offset network is used to detect the video frame, and what is obtained is the degree of change of the feature point in the video frame relative to the previous frame, that is, the feature point offset value . Then further obtain the offset feature map of the video frame according to the feature point offset value corresponding to the video frame and the corresponding detection result of the previous video frame.
  • the offset feature map is the same as the detection feature map, and also includes an object coordinate frame, and the object information of the object in the video frame can be directly determined according to the offset feature map.
  • the detection result corresponding to the previous video frame is the detection feature map or the offset feature map corresponding to the previous video frame.
  • the detection feature map corresponding to the key frame can be directly obtained using the single-frame network, while the detection feature map corresponding to the non-key frame is obtained by using the feature offset network to obtain the feature point offset value, according to the detection feature map corresponding to the previous video frame and
  • the offset feature map of the video frame obtained by calculating the feature point offset value of the video frame. That is, when the detection feature map and the feature point offset value are obtained, the offset feature map corresponding to the video frame belonging to the non-key frame is calculated according to the detection feature map and the feature point offset value.
  • the object information in the video frame is directly determined according to the coordinate boxes in the detection feature map and the offset feature map.
  • the step of obtaining the offset feature map of this video frame according to the detection feature map of the previous video frame and the feature point offset value of this video frame specifically includes: based on the feature point offset value, the corresponding previous frame video frame The detection feature map or the offset feature map for offset. If the corresponding previous video frame is a key frame, it is a detection feature map, and the corresponding previous video frame is a non-key frame, it is an offset feature map.
  • the feature map obtained after the offset is the offset feature map of the current video frame, and the offset feature map is the detection feature map of the current video frame. That is to say, when the feature point offset value is shifted by 2 units to the right, the corresponding feature point in the detection feature map is shifted by 2 units to the right.
  • the detection method after receiving the video to be detected and the object detection instruction, the detection method is determined according to the object detection instruction, thereby ensuring that an accurate detection method is used according to user needs. Then based on the detection method, and using the preset single-frame network and feature offset network to detect each video frame in the video to be detected, the information of the object is determined according to the detected feature map and feature point offset value, so as to ensure the above
  • the method can not only ensure the accuracy of detection through a single-frame network, but also use the feature offset network to obtain feature point offset values that can indicate the degree of change of each video frame. It not only uses the inter-frame information but also improves the detection speed.
  • each video frame is input into a pre-trained single-frame network or feature offset network for object detection according to the key frame flag of each video frame, so as to use the single-frame
  • the network or feature offset network performs object detection on the video frame to obtain the detection result, specifically including: sequentially taking each video frame in the video to be detected as the current frame, and obtaining the key frame flag.
  • the detection network corresponding to the current frame is determined from the single frame network and the feature offset network.
  • Use the detection network corresponding to the current frame to perform object detection on the current frame to obtain the detection result corresponding to the current frame. Change the key frame flag according to the detection result, and return to the step of obtaining the key frame flag until the video frame obtains the corresponding detection result.
  • each video frame in the video to be detected as the current frame in turn, that is, when determining the current frame from each video frame, it needs to be determined in sequence according to the order of the video frames. For example, if the video frames in the video to be detected are video frames in the playback order 1. For video frame 2 and video frame 3, first use video frame 1 as the current frame, and then use video frame 2 as the current frame after obtaining the detection result of video frame 1, and then set video frame 3 after obtaining the detection result of video frame 2. As the current frame.
  • the key frame flag bit refers to a flag used to indicate whether the currently acquired video frame is a key frame, that is, a flag used to indicate whether the current frame is a key frame.
  • the key frame flag includes true (correct) and false (error). If the key frame flag is true, it means that the currently acquired video frame is a key frame. If the key frame flag is false, it means that the currently acquired video frame is Non-key frame.
  • the current key frame flag bit is acquired. Based on the key frame flag bit, the corresponding detection network is selected for the current frame, and the detection network is either a single frame network or a feature offset network. If the obtained key frame flag is true, it means that the current frame is a key frame, and the detection network corresponding to the key frame is a single frame network, and the single frame network is selected as the detection network of the current frame. If the key frame flag is false, it means that the current frame is a non-key frame, and the detection network corresponding to the non-key frame is a feature offset network, and the feature offset network is selected as the detection network of the current frame.
  • the detection network corresponding to the current frame is a single-frame network, and the single-frame network is used to detect the current frame.
  • the detection network corresponding to the current frame is a feature offset network, and the feature offset network is used to detect the current frame.
  • the key frame flag bit is changed according to the detection result obtained by the detection, and the key frame flag bit is re-obtained as the key frame flag bit of the next current frame until all video frames have corresponding detection results.
  • each video frame in the video to be detected is sequentially used as the current frame, that is to say, the video frame used as the current frame for the first time is the first video frame in the video to be detected.
  • the first video frame as the first current frame is the start of this object detection, so the key frame flag corresponding to the first video frame is the preset initial value.
  • the detection of the feature offset network requires the current frame and the corresponding previous video frame to perform offset detection, and the first video frame does not have a corresponding previous video frame. Therefore, the initial value of the key frame flag is true, that is, the first video frame in the video to be detected must be regarded as the key frame.
  • each video frame is detected using a preset single-frame network and a feature offset network to obtain specific detection results. It includes the following steps:
  • Step S302 Determine the current frame.
  • Step S304 Obtain the key frame flag bit.
  • Step S306 based on the acquired key frame flag bit, determine the detection network corresponding to the current frame from the single frame network and the feature offset network.
  • Step S308 Use the detection network corresponding to the current frame to detect the current frame to obtain a detection result corresponding to the current frame.
  • Step S310 Change the key frame flag according to the detection result, and return to step S302, determine the current frame from each video frame, until the video frame obtains the corresponding detection result.
  • video frame 1 is the current frame, and the current key frame flag bit is obtained. If the key frame flag is true, it means that video frame 1 is a key frame. Then, the detection network corresponding to video frame 1 is a single-frame network, and the single-frame network is selected as the detection network of video frame 1. The single-frame network is used to detect video frame 1, and the detection result corresponding to video frame 1 is obtained. And, according to the detection result of video frame 1, the key frame flag is changed. Then, the video frame 2 is determined as the current frame in order, and the current key frame flag bit is obtained, and the key frame flag bit has been changed according to the detection result of video frame 1.
  • the key frame flag has not changed, it is still true, or it may have become false. If the key frame flag is changed to false after the detection result of video frame 1, it means that video frame 2 is a non-key frame. Then, the detection network corresponding to video frame 2 is a feature offset network, and the feature offset network is selected as the detection network of video frame 2. The feature offset network is used to detect video frame 2, and the detection result corresponding to video frame 2 is obtained. Similarly, change the key frame flag again according to the detection result of video frame 2. Finally, it is determined that video frame 3 is the current frame, and the specific detection process is the same as that of video frame 1 and video frame 2, and will not be repeated here.
  • a single frame network is used to detect key frames. While using the single-frame network to ensure the detection accuracy, the feature offset network is used to perform offset detection on non-key frames. It only needs to quickly detect and determine the degree of object change between consecutive frames to obtain the corresponding feature map, not only using the frame Time information also improves the detection speed.
  • determining the detection network corresponding to the current frame from the single-frame network and the feature offset network based on the obtained key frame flags includes: when determining that the current frame is a key frame according to the key frame flags, The single-frame network serves as the detection network corresponding to the current frame. When it is determined that the current frame is a non-key frame according to the key frame flag bit, the characteristic offset network is used as the detection network corresponding to the current frame.
  • the acquired key frame flag bit is true, it means that the current frame is a key frame, and the detection network corresponding to the key frame is a single frame network, and the single frame network is selected as the detection network of the current frame. If the key frame flag bit is false, it means that the current frame is a non-key frame, and the detection network corresponding to the non-key frame is a feature offset network, and the feature offset network is selected as the detection network of the current frame.
  • the step of using the detection network corresponding to the current frame to perform object detection on the current frame to obtain the detection result corresponding to the current frame specifically includes: if the detection network corresponding to the current frame is a single frame network, then using a single frame network Perform single-frame object detection on the current frame, and the obtained detection feature map is used as the detection result corresponding to the current frame. If the detection network corresponding to the current frame is a feature offset network, the feature offset network is used to detect the feature point offset value of the current frame and the previous video frame corresponding to the current frame, and the obtained feature point offset value is used as the current The detection result corresponding to the frame.
  • the detection network corresponding to the video frame is a single-frame network
  • the video frame can be directly input into the single-frame network, and the single-frame network is used to directly perform single-frame detection on the video frame, and the detection result obtained is this
  • the detection feature map of the video frame when the detection network corresponding to the video frame is a feature offset network, the offset value of the video frame needs to be detected. Therefore, in addition to inputting the video frame to the feature offset network, there is also an upper part corresponding to the video frame.
  • a video frame is input to the feature offset network for offset detection. That is, the detection result obtained by using the feature offset network is the offset value of the feature point in the video frame relative to the feature point in the previous video frame.
  • the step of changing the key frame flag bit according to the detection result specifically includes:
  • the object offset average value is calculated based on the feature point offset value, and the object offset average value is accumulated to the preset total object offset value. If the total value of the object offset is not less than the threshold, the total value of the object offset is cleared and the key frame flag bit is changed.
  • the key frame flag is directly changed. For example, if the key frame flag is true, change it to false.
  • the average object offset is calculated based on the feature point offset value, and then the average object offset is added to the total object offset, and finally whether to change is determined according to the total object offset Flag bit. That is, when the total object offset is greater than or equal to the preset threshold value, the key frame flag bit is changed, and the total object offset is cleared at the same time. Conversely, when the total value of the object offset is less than the preset threshold, the key frame flag bit is not changed nor cleared. The initial value of the preset total object offset value is 0. By accumulating the calculated average object offset time and time again, the total object offset value will be cleared to 0 again when it reaches the threshold, and the object offset Once the total value is cleared, it means that the key frame flag needs to be changed.
  • calculating the average value of the object offset specifically includes: according to the feature point offset value of the video frame, offset the feature map corresponding to the previous video frame to obtain the offset feature map corresponding to the video frame.
  • the feature map in the last video frame includes a coordinate frame, that is, the offset feature map obtained according to the offset value of the feature point also has a corresponding offset coordinate frame. Therefore, determine the coordinate frame in the offset feature map, obtain the offset value of the feature points corresponding to all the feature points in the coordinate frame, and do the average calculation based on the offset value of the special diagnosis point corresponding to the feature points in the coordinate frame to obtain the average value Is the mean value of object offset.
  • the feature map corresponding to the previous video frame is the detection feature map.
  • the feature map corresponding to the previous video frame is the offset feature map.
  • the feature point offset value in the detection result includes a feature point positive offset value and a feature point inverse offset value.
  • Step S402 Perform single-frame object detection on each video frame using a single-frame network to obtain a detection feature map corresponding to each video frame.
  • the detection method is precision detection, it means that high-precision object detection is required. There is no need to divide each video frame into key frames and non-key frames like fast detection, and then input them to the corresponding network for detection.
  • each video frame in the video to be detected is input into a single-frame network for detection, and a detection feature map corresponding to each video frame is obtained.
  • the video frame to be detected includes video frame 1, video frame 2, and video frame 3, then video frame 1, video frame 2, and video frame 3 are all input into a single frame network, and the detection characteristics corresponding to video frame 1 are obtained respectively Figure, the detection feature map corresponding to video frame 2 and the detection feature map corresponding to video frame 3.
  • step S404 the feature point positive sequence offset value of each video frame is detected using the feature offset network to obtain the feature point positive offset value corresponding to each video frame.
  • Step S406 Use the feature offset network to detect the feature point reverse offset value of each video frame to obtain the feature point inverse offset value corresponding to each video frame.
  • the positive offset value of feature points refers to the offset of the feature points in the video frame relative to the feature points in the previous video frame
  • the feature point inverse offset value refers to the feature points in the video frame relative to the next video frame.
  • Positive sequence offset detection means that the feature offset network receives positive sequence video frames
  • reverse sequence offset detection means that the feature offset network receives reverse sequence video frames.
  • the positive sequence refers to the positive playback sequence of the video to be detected, such as video frame 1, video frame 2, and video frame 3.
  • the reverse order refers to the reverse playback order of the video to be detected, such as video frame 3, video frame 2, and video frame 1.
  • each video frame in the video to be detected is input into the feature offset network for detection.
  • each video frame in the video to be detected is also Input the reverse order to the feature migration network for reverse order migration detection.
  • the feature point offset value of each video frame relative to the previous video frame can be obtained, that is, the feature point positive offset value.
  • the feature point offset value of each video frame relative to the next video frame can be obtained, that is, the feature point inverse offset value.
  • the feature point inverse offset value Take video frame 1, video frame 2, and video frame 3 as examples , That is, according to the video frame 2 and the video frame 3, the inverse offset value of the feature point of the video frame 2 relative to the video frame 3 is obtained.
  • three different object detections are performed for each video frame in the video to be detected.
  • the detection speed is reduced by performing three detections, the context information of consecutive video frames is fully utilized and more information is passed.
  • Performing detection to improve the accuracy of object detection, especially in the case of blurred motion, using precision detection can improve the accuracy of detection.
  • the corresponding detection is performed according to the feature point offset value.
  • the feature points in the feature map are offset to obtain the offset feature map, and the step of determining the object information according to the detection feature map and the coordinate box in the offset feature map includes: according to the positive offset value of the feature point corresponding to the video frame Offset the feature points in the detection feature map of the previous video frame corresponding to the video frame to generate a positive offset feature map corresponding to the video frame.
  • the feature point in the detection feature map of the next video frame corresponding to the video frame is offset to generate the de-offset feature map corresponding to the video frame.
  • the detection feature map, the positive offset feature map, and the reverse offset feature map of the video frame are feature-fused to generate the final feature map, and the object information is determined according to the coordinate box in the final feature map.
  • the corresponding detection feature map of the previous video frame is offset to obtain the positive offset feature map corresponding to the video frame.
  • the detection feature map of video frame 1 is offset based on the positive offset value of the feature point of video frame 2, and the resulting feature map is the positive offset feature map corresponding to video frame 2.
  • the detection feature map of the corresponding next video frame is offset to obtain the de-offset feature map corresponding to the video frame.
  • the detection feature map of video frame 3 is offset based on the feature point de-offset value of video frame 2, and the obtained feature map is the de-offset feature map corresponding to video frame 2.
  • feature fusion is performed on the three feature maps corresponding to the video frame, that is, the detection feature map, the positive offset feature map, and the reverse offset feature map corresponding to the video frame are feature fused to generate the final feature map.
  • Feature fusion includes but is not limited to averaging or weighted average methods. For example, the average value is the addition of three feature maps and divided by 3, and the weighted average is to be multiplied and added according to the preset weights. Since the first video frame in the video to be detected does not correspond to the previous video frame, and the last video frame does not correspond to the next video frame, the first video frame cannot be based on the previous video frame.
  • the detection feature map generates no positive offset feature map, that is, the first video frame has no positive offset feature map.
  • the last video frame cannot generate a de-offset feature map based on the detection feature map of the next video frame, that is, the last video frame does not have a de-offset feature map.
  • the key frame flag bit is set according to a preset equal interval rule.
  • the equal interval rule refers to a rule that includes the division of key frames. For example, if the equal interval rule specifies that a video frame is taken every 2 frames as Key frame, the two separated video frames are non-key frames. In other words, there are two non-key frames and one key frame in every 3 frames. On the basis that the first frame of the video frame to be detected must be a key frame, it can be simply expressed as "key frame, non-key frame, non-key frame, key frame.". Similarly, when the key frames and non-key frames are determined according to the equal interval rule, the same key frames are input to the single frame network for single frame detection, and the detection feature map is obtained.
  • the non-key frames and the corresponding previous video frames are also input to the feature offset network for offset detection, so as to obtain the feature point offset values of the non-key frames. That is to say, when the video to be detected is received, each video frame in the video to be detected is divided into key frames in advance according to the preset equal interval rule, to determine whether each video frame is a key frame or a non-key frame, and then directly call the corresponding The network is detected without setting the initial value of the key frame flag, and there is no need to change the key flag.
  • the method of using equal interval loops can be applied to a relatively static video scene, and the detection speed can be further improved.
  • an object detection device which includes: a receiving module 502, a determining module 504, a detecting module 506, and an offset module 508,
  • the receiving module 502 is configured to receive a video to be detected and an object detection instruction sent by the terminal, and the video to be detected includes video frames.
  • the determining module 504 is configured to determine a detection mode according to the object detection instruction, and the detection mode includes fast detection and precision detection.
  • the detection module 506 is used to input each video frame into a pre-trained single frame network or feature offset network for object detection according to the key frame flag of each video frame when the detection method is fast detection, so as to use the single frame
  • the network or feature offset network performs object detection on the video frame to obtain the detection result.
  • the detection module 506 is also used to input each video frame into a single frame network and a feature offset network when the detection method is precision detection, so as to use the single frame network to perform object detection on each video frame, and use the feature offset network according to each
  • the playback sequence of the video frames performs multiple object detection on each video frame; the detection results include the detection feature map and the feature point offset value.
  • the offset module 508 is used to offset the feature points in the corresponding detection feature map according to the feature point offset value to obtain the offset feature map, and determine the object information according to the detection feature map and the coordinate frame in the offset feature map .
  • the detection module 506 is further configured to sequentially use each video frame in the video to be detected as the current frame to obtain key frame flag bits; based on the obtained key frame flag bits, from the single frame network and the feature offset network Determine the detection network corresponding to the current frame; use the detection network corresponding to the current frame to perform object detection on the current frame to obtain the detection result corresponding to the current frame; change the key frame flag according to the detection result, and return to the step of obtaining the key frame flag until each The corresponding detection results are obtained for the video frames.
  • the detection module 506 is further configured to use the single-frame network as the detection network corresponding to the current frame when it is determined that the current frame is a key frame according to the key frame flag bit; when the current frame is determined to be non-key frame according to the key frame flag bit When frame, the characteristic offset network is regarded as the detection network corresponding to the current frame.
  • the detection module 506 is further configured to, if the detection network corresponding to the current frame is a single-frame network, use the single-frame network to perform single-frame object detection on the current frame, and the obtained detection feature map is used as the detection corresponding to the current frame Results; if the detection network corresponding to the current frame is a feature offset network, the feature offset network is used to detect the feature point offset value of the current frame and the previous frame of the video frame corresponding to the current frame, and the feature point offset value obtained As the detection result corresponding to the current frame.
  • the detection module 506 is further configured to change the key frame flag bit when the detection result is the detection feature map, and return to the step of obtaining the key frame flag bit until each video frame obtains the corresponding detection result; or
  • the average object offset is calculated based on the feature point offset value, and the average object offset is added to the preset total object offset; if the total object offset is not less than the gate Limit value, clear the total value of the object offset, change the key frame flag, and return to the step of obtaining the key frame flag until each video frame obtains the corresponding detection result.
  • the detection module 506 is further configured to perform single-frame object detection on each video frame using a single-frame network to obtain a detection feature map corresponding to each video frame; use a feature offset network to perform feature point correction on each video frame.
  • the detection of the sequence offset value obtains the positive offset value of the feature points corresponding to each video frame; the feature offset network is used to detect the feature point reverse offset value of each video frame to obtain the feature point inverse offset corresponding to each video frame value.
  • the offset module 508 is further configured to offset the feature points in the detection feature map of the previous video frame corresponding to the video frame according to the positive offset value of the feature point corresponding to the video frame to generate a video frame Corresponding positive offset feature map; according to the inverse offset value of the feature point corresponding to the video frame, offset the feature points in the detection feature map of the next video frame corresponding to the video frame to generate the de-offset corresponding to the video frame Feature map; feature fusion of the detection feature map, positive offset feature map, and reverse offset feature map of the video frame to generate the final feature map; determine the object information according to the coordinate box in the final feature map.
  • Each module in the above-mentioned object detection device can be implemented in whole or in part by software, hardware and a combination thereof.
  • the foregoing modules may be embedded in the form of hardware or independent of the processor in the computer device, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the foregoing modules.
  • a computer device is provided.
  • the computer device may be a server, and its internal structure diagram may be as shown in FIG. 6.
  • the computer equipment includes a processor, a memory, a network interface and a database connected through a system bus.
  • the processor of the computer device is used to provide calculation and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium stores an operating system, computer readable instructions, and a database.
  • the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the non-volatile storage medium.
  • the computer equipment database is used to store data.
  • the network interface of the computer device is used to communicate with an external terminal through a network connection.
  • the computer-readable instructions are executed by the processor to realize an object detection method.
  • FIG. 6 is only a block diagram of part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied.
  • the specific computer device may Including more or fewer parts than shown in the figure, or combining some parts, or having a different arrangement of parts.
  • a computer device includes a memory and one or more processors.
  • the memory stores computer readable instructions.
  • the steps of the object detection method provided in any embodiment of the present application are implemented.
  • One or more non-volatile storage media storing computer-readable instructions.
  • the computer-readable instructions When executed by one or more processors, the one or more processors implement the object provided in any embodiment of the present application Steps of detection method.
  • Non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory may include random access memory (RAM) or external cache memory.
  • RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Channel (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
  • SRAM static RAM
  • DRAM dynamic RAM
  • SDRAM synchronous DRAM
  • DDRSDRAM double data rate SDRAM
  • ESDRAM enhanced SDRAM
  • SLDRAM synchronous chain Channel
  • memory bus Radbus direct RAM
  • RDRAM direct memory bus dynamic RAM
  • RDRAM memory bus dynamic RAM

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

一种物体检测方法，包括：接收终端发送的待检测视频和物体检测指令，待检测视频中包括视频帧；根据物体检测指令的类型确定检测方式，检测方式包括快速检测和精度检测；当检测方式为快速检测时，根据各视频帧的关键帧标志位将各视频帧输入预先训练好的用于物体检测的单帧网络或特征偏移网络，以利用单帧网络或特征偏移网络对视频帧进行物体检测，得到检测结果；当检测方式为精度检测时，分别将各视频帧输入单帧网络和特征偏移网络，以利用单帧网络对各视频帧进行物体检测，以及利用特征偏移网络根据各视频帧的播放顺序对各视频帧进行多次的物体检测；检测结果包括检测特征图和特征点偏移值；根据特征点偏移值将对应的检测特征图中的特征点进行偏移，得到偏移特征图，并根据检测特征图和偏移特征图中的坐标框确定物体信息。

Description

物体检测方法、装置、计算机设备和存储介质
相关申请的交叉引用
本申请要求于2019年06月25日提交中国专利局，申请号为2019105547489，申请名称为“物体检测方法、装置、计算机设备和存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及一种物体检测方法、装置、计算机设备和存储介质。
背景技术
物体检测是计算机视觉中的经典问题之一，其任务是用框标出物体的位置并给出物体的类别。因此，物体检测在人脸识别、安全监控以及动态追踪等很多方面都有广泛的应用。
然而，发明人意识到，传统进行物体检测的算法大多采用逐帧检测网络直接对视频帧依次进行检测。虽然在一定程度上保证了检测的精度性，但是在高分辨情况下难以达到实时，导致检测速度缓慢。
发明内容
根据本申请公开的各种实施例，提供一种物体检测方法、装置、计算机设备和存储介质。
一种物体检测方法，包括：
服务器接收终端发送的待检测视频和物体检测指令，所述待检测视频中包括视频帧；
所述服务器根据所述物体检测指令确定检测方式，所述检测方式包括快速检测和精度检测；
当所述检测方式为所述快速检测时，所述服务器根据各所述视频帧的关键帧标志位将各所述视频帧输入预先训练好的用于物体检测的单帧网络或特征偏移网络，以利用所述单帧网络或特征偏移网络对所述视频帧进行物体检测，得到检测结果；
当所述检测方式为所述精度检测时，所述服务器分别将各所述视频帧输入所述单帧网络和所述特征偏移网络，以利用所述单帧网络对各所述视频帧进行物体检测，以及利用所述特征偏移网络根据各所述视频帧的播放顺序对各所述视频帧进行多次的物体检测；所述检测结果包括检测特征图和特征点偏移值；及
所述服务器根据所述特征点偏移值将对应的检测特征图中的特征点进行偏移，得到偏移特征图，并根据所述检测特征图和偏移特征图中的坐标框确定物体信息。
一种物体检测装置，包括：
接收模块，用于接收终端发送的待检测视频和物体检测指令，所述待检测视频中包括视频帧；
确定模块，用于根据所述物体检测指令确定检测方式，所述检测方式包括快速检测和精度检测；
检测模块，用于当所述检测方式为所述快速检测时，根据各所述视频帧的关键帧标志位将各所述视频帧输入预先训练好的用于物体检测的单帧网络或特征偏移网络，以利用所述单帧网络或特征偏移网络对所述视频帧进行物体检测，得到检测结果；
检测模块还用于当所述检测方式为所述精度检测时，分别将各所述视频帧输入所述单帧网络和所述特征偏移网络，以利用所述单帧网络对各所述视频帧进行物体检测，以及利用所述特征偏移网络根据各所述视频帧的播放顺序对各所述视频帧进行多次的物体检测；所述检测结果包括检测特征图和特征点偏移值；及
偏移模块，用于根据所述特征点偏移值将各所述视频帧输入预先训练好的用于物体检测的单帧网络和特征偏移网络，并根据所述检测特征图和偏移特征图中的坐标框确定物体信息。
一种计算机设备，包括存储器和一个或多个处理器，所述存储器中储存有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述一个或多个处理器执行以下步骤：
接收终端发送的待检测视频和物体检测指令，所述待检测视频中包括视频帧；
根据所述物体检测指令确定检测方式，所述检测方式包括快速检测和精度检测；
当所述检测方式为所述快速检测时，根据各所述视频帧的关键帧标志位将各所述视频帧输入预先训练好的用于物体检测的单帧网络或特征偏移网络，以利用所述单帧网络或特征偏移网络对所述视频帧进行物体检测，得到检测结果；
当所述检测方式为所述精度检测时，分别将各所述视频帧输入所述单帧网络和所述特征偏移网络，以利用所述单帧网络对各所述视频帧进行物体检测，以及利用所述特征偏移网络根据各所述视频帧的播放顺序对各所述视频帧进行多次的物体检测；所述检测结果包括检测特征图和特征点偏移值；及
根据所述特征点偏移值将对应的检测特征图中的特征点进行偏移，得到偏移特征图，并根据所述检测特征图和偏移特征图中的坐标框确定物体信息。一个或多个存储有计算机可读指令的非易失性存储介质，计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行以下步骤：
接收终端发送的待检测视频和物体检测指令，所述待检测视频中包括视频帧；
根据所述物体检测指令确定检测方式，所述检测方式包括快速检测和精度检测；
当所述检测方式为所述快速检测时，根据各所述视频帧的关键帧标志位将各所述视频帧输入预先训练好的用于物体检测的单帧网络或特征偏移网络，以利用所述单帧网络或特征偏移网络对所述视频帧进行物体检测，得到检测结果；
当所述检测方式为所述精度检测时，分别将各所述视频帧输入所述单帧网络和所述特征偏移网络，以利用所述单帧网络对各所述视频帧进行物体检测，以及利用所述特征偏移网络根据各所述视频帧的播放顺序对各所述视频帧进行多次的物体检测；所述检测结果包括检测特征图和特征点偏移值；及
根据所述特征点偏移值将对应的检测特征图中的特征点进行偏移，得到偏移特征图，并根据所述检测特征图和偏移特征图中的坐标框确定物体信息。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征和优点将从说明书、附图以及权利要求书变得明显。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。
图1为根据一个或多个实施例中物体检测方法的应用场景图。
图2为根据一个或多个实施例中物体检测方法的流程示意图。
图3为根据一个或多个实施例中基于检测方式，利用预设的单帧网络和特征偏移网络对各视频帧进行检测，得到检测结果步骤的流程示意图。
图4为另一个实施例中基于检测方式，利用预设的单帧网络和特征偏移网络对各视频帧进行检测，得到检测结果步骤的流程示意图。
图5为根据一个或多个实施例中物体检测装置的框图。
图6为根据一个或多个实施例中计算机设备的框图。
具体实施方式
为了使本申请的技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
本申请提供的物体检测方法，可以应用于如图1所示的应用环境中。该应用环境涉及终端102和服务器104，终端102通过网络与服务器104进行通信。终端102获取到待检测视频和物体检测指令时，可以由终端102单独实现上述物体检测方法。也可以通过终端102将待检测视频和物体检测指令发送给服务器104，服务器104实现物体检测方法。具体地，服务器104接收终端102发送的待检测视频和物体检测指令，待检测视频中包括视频帧。服务器104根据物体检测指令确定检测方式，检测方式包括快速检测和精度检测。当检测方式为快速检测时，服务器104根据各视频帧的关键帧标志位将各视频帧输入预先训练好的用于物体检测的单帧网络或特征偏移网络，以利用单帧网络或特征偏移网络对视频帧进行物体检测，得到检测结果；当检测方式为精度检测时，服务器104分别将各视频 帧输入单帧网络和特征偏移网络，以利用单帧网络对各视频帧进行物体检测，以及利用特征偏移网络根据各视频帧的播放顺序对各视频帧进行多次的物体检测；检测结果包括检测特征图和特征点偏移值；服务器104根据特征点偏移值将对应的检测特征图中的特征点进行偏移，得到偏移特征图，并根据检测特征图和偏移特征图中的坐标框确定物体信息。终端102可以但不限于是各种个人计算机、笔记本电脑、智能手机、平板电脑和便携式可穿戴设备，服务器104可以用独立的服务器或者是多个服务器组成的服务器集群来实现。
在一些实施例中，如图2所示，提供了一种物体检测方法，以该方法应用于图1中的服务器为例进行说明，包括以下步骤：
步骤S202，接收终端发送的待检测视频和物体检测指令，待检测视频中包括视频帧。
待检测视频是指需要进行物体检测的视频，视频帧则是用于组成视频的视频图像。例如，待检测视频包括但不限于监控设备的监控视频。而物体检测指令是指用于请求服务器进行物体检测的指令。
具体地，当用户有物体检测需求时，通过操作用户终端给服务器下发物体检测指令，以及随物体检测指令向服务器发送本次进行物体检测的待检测视频。
步骤S204，根据物体检测指令确定检测方式，检测方式包括快速检测和精度检测。
检测方式包括快速检测和精度检测，通过不同的物体检测确定本次进行物体检测的方式。具体地，当用户需要进行物体检测时，通过操作终端发起物体检测任务和选择对应的检测方式，终端根据用户的操作生成对应的物体检测指令。也就是说，若物体检测任务的需求是快速获取检测结果的，则用户在操作终端发起物体检测任务时同步在终端页面选择快速检测方式，当终端接收到用户选择了快速检测方式后，根据用户所选择的快速检测方式生成快速物体检测指令，则生成的物体检测指令为快速物体检测指令。服务器接收到快速物体检测指令时，即可响应该快速物体检测指令确定检测方式为快速检测，则基于快速检测方式进行快速的物体检测。同样的，当用户需要高精度的物体检测结果的，则在下发物体检测指令时同步选择精度检测方式，则生成的物体检测指令为精度物体检测指令。服务器接收到精度物体检测指令时，即可响应该精度物体检测指令进行高精度的物体检测。或者，当用户操作终端发起物体检测任务和选择对应的检测方式时，终端可以直接生成物体检测指令，将用户所选择的方式随物体检测指令发送给服务器，即物体检测指令中包括检测方式。服务器通过对物体检测指令进行判断，确定物体检测指令所包括的检测方式，从而确定物体检测的检测方式。
步骤S206，当检测方式为快速检测时，根据各视频帧的关键帧标志位将各视频帧输入预先训练好的用于物体检测的单帧网络或特征偏移网络，以利用单帧网络或特征偏移网络对视频帧进行物体检测，得到检测结果。
步骤S208，当检测方式为精度检测时，分别将各视频帧输入单帧网络和特征偏移网络，以利用单帧网络对各视频帧进行物体检测，以及利用特征偏移网络根据各视频帧的播放顺序对各视频帧进行多次的物体检测；检测结果包括检测特征图和特征点偏移值。
单帧网络和特征偏移网络均为预先根据视频样本训练好，用于物体检测的神经网络，视频样本是指已标注物体的视频帧。单帧网络为传统的逐帧检测算法网络，可以为现有任意一种，例如faster rcnn检测网络、yolov3检测网络等。特征偏移网络是一种近似光流网络的目标检测网络，利用flownet作为网络的基础架构。与传统的光流网络相比，特征偏移网络的输出层从计算图像中每个像素点的偏移值更改为只计算特征图中每个像素点的偏移值。单帧网络得到的检测结果是检测特征图，特征偏移网络得到的检测结果是特征点偏移值。基于不同的检测方式将各视频帧输入至预先训练好的单帧网络和特征偏移网络进行物体检测。
具体地，基于不同的检测方式，各视频帧输入至单帧网络和特征偏移网络的形式不一样。当检测方式为快速检测时，需要根据各视频帧对应的关键帧标志位对所有的视频帧进行关键帧和非关键帧的划分。属于关键帧的视频帧利用单帧网络进行检测，属于非关键帧的视频帧利用特征偏移网络进行检测。而当检测方式为精度检测时，所有的视频帧需要同时利用单帧网络和特征偏移网络进行检测，即所有视频帧输入到单帧网络进行检测之外，还需要将视频帧输入到特征偏移网络进行检测。并且，特征偏移网络会根据各视频帧的播放顺序对各视频帧进行多次的物体检测，即根据各视频帧正序的播放顺序检测一次，以及根据各视频帧倒序的播放顺序检测一次。
步骤S210，根据特征点偏移值将对应的检测特征图中的特征点进行偏移，得到偏移特征图，并根据检测特征图和偏移特征图中的坐标框确定物体信息。
检测特征图是单帧网络对视频帧进行检测输出的特征图，而特征点偏移值是特征偏移网络对视频帧进行检测，得到的该视频帧中的特征点相对于上一帧视频帧中的特征点的偏移值，即相对应的特征点之间的变化程度。也就是通过相邻视频帧之间的特征点的偏移值就能得到视频中物体偏移的程度，即物体的移动变化程度。物体信息是指被检测出的物体所属的类别。
具体地，由于利用网络模型进行物体检测，网络模型输出的特征图上包括被检测出物体的坐标框，则根据该坐标框可直接确定物体信息。即，当视频帧属于关键帧，也就是利用单帧网络对该视频帧进行检测，得到的是该视频帧对应的检测特征图，则根据单帧检测网络输出的检测特征图上的坐标框直接确定物体信息。而当视频帧属于非关键帧，也就是利用特征偏移网络检测对该视频帧进行检测，得到的是该视频帧相对于上一帧视频帧中特征点的变化程度，即特征点偏移值，则进一步根据该视频帧对应的特征点偏移值以及对应的上一帧视频帧的检测结果得到该视频帧的偏移特征图。可以理解，该偏移特征图与检测特征图相同，同样包括物体坐标框，根据偏移特征图即可直接确定该视频帧中物体的物体信息。上一帧视频帧对应的检测结果为上一帧视频帧对应的检测特征图或者偏移特征图。
也就是说，无论视频帧是关键帧还是非关键帧，都有对应的检测特征图。关键帧对应的检测特征图利用单帧网络可以直接得到，而非关键帧对应的检测特征图是利用特征偏 移网络得到特征点偏移值后，根据对应上一帧视频帧的检测特征图和本视频帧的特征点偏移值计算得到的本视频帧的偏移特征图。即，当得到检测特征图和特征点偏移值时，根据检测特征图和特征点偏移值计算得到属于非关键帧的视频帧对应的偏移特征图。也就是说，根据检测特征图和偏移特征图中的坐标框直接确定视频帧中的物体信息。
根据上一帧视频帧的检测特征图和本视频帧的特征点偏移值得到本视频帧的偏移特征图的步骤，具体包括：基于特征点偏移值，对对应的上一帧视频帧的检测特征图或是偏移特征图进行偏移。若对应的上一帧视频帧为关键帧，则是检测特征图，对应的上一视频帧是非关键帧，则是偏移特征图。经过偏移得到的特征图即为本视频帧的偏移特征图，该偏移特征图即为本视频帧的检测特征图。也就是说，当特征点偏移值为向右平移2个单位，则将检测特征图中所对应的特征点向右平移2个单位。
上述物体检测方法中，当接收到待检测视频和物体检测指令后，根据物体检测指令确定检测方式，从而保证根据用户需求使用准确的检测方式。然后基于检测方式，且利用预设的单帧网络和特征偏移网络对待检测视频中的各视频帧进行检测，根据检测得到的检测特征图和特征点偏移值确定物体的信息，从而保证上述方法不仅能够通过单帧网络保证检测的精度，还能利用特征偏移网络得到能够表明各视频帧变化程度的特征点偏移值，不仅利用了帧间信息还提高了检测速度。
在一些实施例中，当检测方式为快速检测时，根据各视频帧的关键帧标志位将各视频帧输入预先训练好的用于物体检测的单帧网络或特征偏移网络，以利用单帧网络或特征偏移网络对视频帧进行物体检测，得到检测结果的步骤，具体包括：依次将待检测视频中的各视频帧作为当前帧，获取关键帧标志位。基于获取的关键帧标志位，从单帧网络和特征偏移网络中确定当前帧对应的检测网络。利用当前帧对应的检测网络对当前帧进行物体检测，得到当前帧对应的检测结果。根据检测结果更改关键帧标志位，返回获取关键帧标志位的步骤，直至视频帧均得到对应的检测结果。
依次将待检测视频中的各视频帧作为当前帧，即为从各视频帧中确定当前帧时需按照视频帧的顺序依次确定，例如，若待检测视频中的视频帧按播放顺序为视频帧1、视频帧2和视频帧3，则首先将视频帧1作为当前帧，得到视频帧1的检测结果后在将视频帧2作为当前帧，得到视频帧2的检测结果后在将视频帧3作为当前帧。
关键帧标志位则是指用于表示当前获取的视频帧是否为关键帧的标志，即用于表示当前帧是否为关键帧的标志。关键帧标志位包括true(正确)和false(错误)，若关键帧标志位为true，则表示当前获取的视频帧为关键帧，若关键帧标志位为false，则表示当前获取的视频帧为非关键帧。
具体地，当确定一帧视频帧作为当前帧后，获取当前的关键帧标志位。基于关键帧标志位为当前帧选择对应的检测网络，检测网络为单帧网络和特征偏移网络中任意一种。若获取的关键帧标志位为true，则表示当前帧为关键帧，关键帧对应的检测网络为单帧网络，则选择单帧网络作为当前帧的检测网络。而若关键帧标志位为false，则表示当前 帧为非关键帧，非关键帧对应的检测网络为特征偏移网络，则选择特征偏移网络作为当前帧的检测网络。获取到当前帧对应的检测网络后，利用对应的检测网络对当前帧进行检测。也就是说，当前帧对应的检测网络为单帧网络，则利用单帧网络对当前帧进行检测。当前帧对应的检测网络为特征偏移网络，则利用特征偏移网络对当前帧进行检测。然后，根据检测得到的检测结果更改关键帧标志位，并且重新获取关键帧标志位，作为下个当前帧的关键帧标志位，直到所有的视频帧都有对应的检测结果为止。
由于是依次将待检测视频中的各视频帧作为当前帧，也就是说第一次作为当前帧的视频帧为待检测视频中的第一帧视频帧。而第一帧视频帧作为第一个当前帧是本次物体检测的开始，所以第一帧视频帧对应的关键帧标志位为预设的初始值。在本实施例中，由于特征偏移网络的检测需要当前帧以及对应的上一帧视频帧进行偏移检测，而第一帧视频帧是没有对应的上一帧视频帧的。因此，关键帧标志位为的初始值为true，即待检测视频中的第一帧视频帧一定被作为关键帧。
在一些实施例中，如图3所示的流程图，当检测方式为快速检测时，基于检测方式，利用预设的单帧网络和特征偏移网络对各视频帧进行检测，得到检测结果具体包括以下步骤：
步骤S302，确定当前帧。
步骤S304，获取关键帧标志位。
步骤S306，基于获取的关键帧标志位，从单帧网络和特征偏移网络中确定当前帧对应的检测网络。
步骤S308，利用当前帧对应的检测网络对当前帧进行检测，得到当前帧对应的检测结果。
步骤S310，根据检测结果更改关键帧标志位，返回步骤S302，从各视频帧中确定当前帧，直至视频帧均得到对应的检测结果。
具体地，以待检测视频含有三帧视频帧为例进行说明，首先确定视频帧1为当前帧，获取当前的关键帧标志位。而关键帧标志位为true，则表示视频帧1为关键帧。那么，视频帧1对应的检测网络则为单帧网络，则选择单帧网络作为视频帧1的检测网络。利用单帧网络对视频帧1进行检测，得到视频帧1对应的检测结果。并且，根据视频帧1的检测结果更改关键帧标志位。然后，按顺序确定视频帧2为当前帧，获取当前的关键帧标志位，而该关键帧标志位已经根据视频帧1的检测结果进行更改了。可能该关键帧标志位没有变，还是true，也可能已经变为false了，假设经过视频帧1的检测结果将该关键帧标志位改为了false，则表示视频帧2为非关键帧。那么，视频帧2对应的检测网络为特征偏移网络，则选择特征偏移网络作为视频帧2的检测网络。利用特征偏移网络对视频帧2进行检测，得到视频帧2对应的检测结果。同样的，根据视频帧2的检测结果再次更改关键帧标志位。最后，确定视频帧3为当前帧，具体检测过程同视频帧1和视频帧2一致，在此不再赘述。
在本实施例中，利用视频帧之间信息冗余较大的特点，通过划分关键帧和非关键帧，使用单帧网络对关键帧进行检测。利用单帧网络保证检测精度的同时下，采用特征偏移网络对非关键帧进行偏移检测，只需要快速检测确定连续帧之间物体的变化程度就能得到对应的特征图，不仅利用了帧间信息，还提升了检测速度。
在一些实施例中，基于获取的关键帧标志位，从单帧网络和特征偏移网络中确定当前帧对应的检测网络，具体包括：当根据关键帧标志位确定当前帧为关键帧时，将单帧网络作为当前帧对应的检测网络。当根据关键帧标志位确定当前帧为非关键帧时，将特征偏移网络作为当前帧对应的检测网络。
具体地，若获取的关键帧标志位为true，则表示当前帧为关键帧，关键帧对应的检测网络为单帧网络，则选择单帧网络作为当前帧的检测网络。而若关键帧标志位为false，则表示当前帧为非关键帧，非关键帧对应的检测网络为特征偏移网络，则选择特征偏移网络作为当前帧的检测网络。
在一些实施例中，利用当前帧对应的检测网络对当前帧进行物体检测，得到当前帧对应的检测结果的步骤，具体包括：若当前帧对应的检测网络为单帧网络，则利用单帧网络对当前帧进行单帧的物体检测，得到的检测特征图作为当前帧对应的检测结果。若当前帧对应的检测网络为特征偏移网络，则利用特征偏移网络对当前帧以及当前帧对应的上一帧视频帧进行特征点偏移值的检测，得到的特征点偏移值作为当前帧对应的检测结果。
具体地，当视频帧对应的检测网络为单帧网络时，即可直接将该视频帧输入至单帧网络中，利用单帧网络直接对该视频帧进行单帧检测，得到的检测结果为该视频帧的检测特征图。而，当视频帧对应的检测网络为特征偏移网络时，需要检测该视频帧的偏移值，因此除了将该视频帧输入至特征偏移网络之外，还有将该视频帧对应的上一帧视频帧输入至特征偏移网络一起进行偏移检测。即利用特征偏移网络得到的检测结果为该视频帧中的特征点相对上一帧视频帧中的特征点的偏移值。
在一些实施例中，根据检测结果更改关键帧标志位的步骤，具体包括：
当检测结果为检测特征图时，更改关键帧标志位。或者，
当检测结果为特征点偏移值时，基于特征点偏移值计算得到物体偏移均值，并将物体偏移均值累加至预设的物体偏移总值。若物体偏移总值不小于门限值，则将物体偏移总值清零，以及更改关键帧标志位。
具体地，当检测结果为检测特征图时，直接将关键帧标志位进行更改。例如关键帧标志位为true，则更改为false。
当检测结果为特征点偏移值时，基于该特征点偏移值计算得到物体偏移均值，然后将物体偏移均值累加到物体偏移总值中，最后根据物体偏移总值决定是否更改标志位。即，当物体偏移总值大于等于预设的门限值时，则更改关键帧标志位，并同时将物体偏移总值清零。反之，物体偏移总值小于预设的门限值时，不更改关键帧标志位也不清零。预设的物体偏移总值的初始值为0，通过一次次将计算得到的物体偏移均值进行累加，使得物体 偏移总值达到门限值时才会再次清除到0，而物体偏移总值一旦清零就意味着需要更改关键帧标志位。
在一些实施例中，计算物体偏移均值具体包括：根据视频帧的特征点偏移值，对对应上一帧视频帧对应的特征图进行偏移，得到视频帧对应的偏移特征图，由于上一帧视频帧中的特征图包括坐标框，即根据特征点偏移值得到的偏移特征图也有对应的偏移的坐标框。因此，确定该偏移特征图中的坐标框，获取该坐标框内所有特征点对应的特征点偏移值，基于坐标框内特征点对应的特诊点偏移值做均值运算，得到的均值为物体偏移均值。上一帧视频帧为关键帧时，上一帧视频帧对应的特征图为检测特征图。上一帧视频帧为非关键帧时，上一帧视频帧对应的特征图为偏移特征图。
在一些实施例中，如图4所示，当检测方式为精度检测时，检测结果中的特征点偏移值包括特征点正偏移值和特征点反偏移值。则，当检测方式为精度检测时，分别将各视频帧输入单帧网络和特征偏移网络，以利用单帧网络对各视频帧进行物体检测，以及利用特征偏移网络根据各视频帧的播放顺序对各视频帧进行多次的物体检测的步骤，具体包括以下步骤：
步骤S402，利用单帧网络对各视频帧进行单帧的物体检测，得到各视频帧对应的检测特征图。
当确定检测方式为精度检测时，表示需要进行高精度的物体检测。则无需如快速检测一般将各视频帧进行关键帧、非关键帧的划分后，再输入到对应的网络进行检测。
具体地，将待检测视频中的各视频帧均输入到单帧网络中进行检测，得到各视频帧对应的检测特征图。例如，假设待检测视频帧包括视频帧1、视频帧2和视频帧3，则将视频帧1、视频帧2和视频帧3均输入到单帧网络中，分别得到视频帧1对应的检测特征图、视频帧2对应的检测特征图和视频帧3对应的检测特征图。
步骤S404，利用特征偏移网络对各视频帧进行特征点正序偏移值的检测，得到各视频帧对应的特征点正偏移值。
步骤S406，利用特征偏移网络对各视频帧进行特征点倒序偏移值的检测，得到各视频帧对应的特征点反偏移值。
特征点正偏移值是指视频帧中的特征点相对上一帧视频帧中特征点的偏移，而特征点反偏移值则是指视频帧中的特征点相对下一帧视频帧中特征点的偏移。正序偏移检测是指特征偏移网络接收正序的视频帧，倒序偏移检测是指特征偏移网络接收倒序的视频帧。正序即是按照待检测视频的正播放顺序，例如视频帧1、视频帧2和视频帧3。而倒序即是按照待检测视频的反播放顺序，例如视频帧3、视频帧2和视频帧1。
具体地，将待检测视频中的各视频帧均输入到特征偏移网络中进行检测。而为了充分利用连续视频帧的上下文信息，除了将待检测视频中的各视频帧按照正序输入至特征偏移网络中进行正序偏移检测之外，还将待检测视频中的各视频帧倒序输入至特征偏移网络中进行反序偏移检测。通过正序偏移检测可以得到每一帧视频帧相对上一帧视频帧的特征 点偏移值，即特征点正偏移值，以视频帧1、视频帧2和视频帧3为例，即根据视频帧1和视频帧2计算得到视频帧2相对于视频帧1的特征点正偏移值。同样的，通过倒序偏移检测可以得到每一帧视频帧相对下一帧视频帧的特征点偏移值，即特征点反偏移值，以视频帧1、视频帧2和视频帧3为例，即根据视频帧2和视频帧3计算得到视频帧2相对于视频帧3的特征点反偏移值。
在本实施例中，通过对待检测视频中的各视频帧进行三遍不同的物体检测，虽然进行三次检测使得检测速度有所下降，但是充分利用了连续视频帧的上下文信息，通过更多的信息进行检测从而提高物体检测的精度，尤其在运动比较模糊的情况下运用精度检测，能够提高检测的准确率。
在一些实施例中，当检测方式为精度检测，且检测结果中的特征点偏移值包括特征点正偏移值和特征点反偏移值时，则根据特征点偏移值将对应的检测特征图中的特征点进行偏移，得到偏移特征图，并根据检测特征图和偏移特征图中的坐标框确定物体信息的步骤，具体包括：根据视频帧对应的特征点正偏移值，将视频帧对应的上一帧视频帧的检测特征图中的特征点进行偏移，生成视频帧对应的正偏移特征图。根据视频帧对应的特征点反偏移值，将视频帧对应的下一帧视频帧的检测特征图中的特征点进行偏移，生成视频帧对应的反偏移特征图。将视频帧的检测特征图、正偏移特征图以及反偏移特征图进行特征融合，生成最终的特征图，根据最终的特征图中的坐标框确定物体信息。
具体地，基于视频帧的特征点正偏移值，对对应的上一帧视频帧的检测特征图进行偏移，得到视频帧对应的正偏移特征图。例如，基于视频帧2的特征点正偏移值对视频帧1的检测特征图进行偏移，得到的特征图为视频帧2对应的正偏移特征图。相同的，基于视频帧的特征点反偏移值，对对应的下一帧视频帧的检测特征图进行偏移，得到视频帧对应的反偏移特征图。例如，基于视频帧2的特征点反偏移值对视频帧3的检测特征图进行偏移，得到的特征图为视频帧2对应的反偏移特征图。而确定物体信息时，即将视频帧对应的三张特征图进行特征融合，即将视频帧对应的检测特征图、正偏移特征图以及反偏移特征图进行特征融合，生成最终的特征图。特征融合包括但不限于取平均值或者加权平均的方法。例如，采用平均值即三张特征图相加除以3，加权平均则是根据预设的权重分别相乘后相加即可。由于待检测视频中的第一帧视频帧没有对应的上一帧视频帧，以及最后一帧视频帧没有对应的下一帧视频帧，因此，第一帧视频帧无法根据上一帧视频帧的检测特征图生成没有正偏移特征图，即第一帧视频帧没有正偏移特征图。而最后一帧视频帧无法根据下一帧视频帧的检测特征图生成反偏移特征图，即最后一帧视频帧没有反偏移特征图。
在一些实施例中，关键帧标志位根据预设的等间隔规则进行设定，等间隔规则是指包括关键帧划分的规则，例如，若等间隔规则规定每隔2帧取一帧视频帧为关键帧，则相隔的两帧视频帧为非关键帧。也就是说，每3帧中有两帧非关键帧和一帧关键帧。在待检测视频帧的第一帧一定为关键帧的基础上，则可以简单表示为“关键帧、非关键帧、非关键 帧、关键帧……”。同样的，当根据等间隔规则确定好关键帧和非关键帧后，同样是关键帧输入至单帧网络进行单帧检测，得到检测特征图。而为非关键帧的视频帧则同样将该非关键帧以及对应的上一帧视频帧输入至特征偏移网络进行偏移检测，从而得到非关键帧的特征点偏移值。也就是说，当接收到待检测视频后，根据预设的等间隔规则预先将待检测视频中的各视频帧进行关键帧划分，确定各视频帧是关键帧还是非关键帧，后续直接调用对应的网络进行检测，无需对关键帧标志位设定初始值，也无需进行关键标志位的更改。在本实施例中，利用等间隔循环的方法可以适用于偏静止的视频场景，可以进一步提升检测速度。
应该理解的是，虽然图2-4的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，图2-4中的至少一部分步骤可以包括多个子步骤或者多个阶段，这些子步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些子步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。
在一些实施例中，如图5所示，提供了一种物体检测装置，包括：接收模块502、确定模块504、检测模块506和偏移模块508，
接收模块502，用于接收终端发送的待检测视频和物体检测指令，待检测视频中包括视频帧。
确定模块504，用于根据物体检测指令确定检测方式，检测方式包括快速检测和精度检测。
检测模块506，用于当检测方式为快速检测时，根据各视频帧的关键帧标志位将各视频帧输入预先训练好的用于物体检测的单帧网络或特征偏移网络，以利用单帧网络或特征偏移网络对所述视频帧进行物体检测，得到检测结果。
检测模块506还用于当检测方式为精度检测时，分别将各视频帧输入单帧网络和特征偏移网络，以利用单帧网络对各视频帧进行物体检测，以及利用特征偏移网络根据各视频帧的播放顺序对各视频帧进行多次的物体检测；检测结果包括检测特征图和特征点偏移值。
偏移模块508，用于根据特征点偏移值将对应的检测特征图中的特征点进行偏移，得到偏移特征图，并根据检测特征图和偏移特征图中的坐标框确定物体信息。
在一些实施例中，检测模块506还用于依次将待检测视频中的各视频帧作为当前帧，获取关键帧标志位；基于获取的关键帧标志位，从单帧网络和特征偏移网络中确定当前帧对应的检测网络；利用当前帧对应的检测网络对当前帧进行物体检测，得到当前帧对应的检测结果；根据检测结果更改关键帧标志位，返回获取关键帧标志位的步骤，直至各视频帧均得到对应的检测结果。
在一些实施例中，检测模块506还用于当根据关键帧标志位确定当前帧为关键帧时，将单帧网络作为当前帧对应的检测网络；当根据关键帧标志位确定当前帧为非关键帧时，将特征偏移网络作为当前帧对应的检测网络。
在一些实施例中，检测模块506还用于若当前帧对应的检测网络为单帧网络，则利用单帧网络对当前帧进行单帧的物体检测，得到的检测特征图作为当前帧对应的检测结果；若当前帧对应的检测网络为特征偏移网络，则利用特征偏移网络对当前帧以及当前帧对应的上一帧视频帧进行特征点偏移值的检测，得到的特征点偏移值作为当前帧对应的检测结果。
在一些实施例中，检测模块506还用于当检测结果为检测特征图时，更改关键帧标志位，并返回获取关键帧标志位的步骤，直至各视频帧均得到对应的检测结果；或
当检测结果为特征点偏移值时，基于特征点偏移值计算得到物体偏移均值，并将物体偏移均值累加至预设的物体偏移总值；若物体偏移总值不小于门限值，则将物体偏移总值清零，以及更改关键帧标志位，并返回获取关键帧标志位的步骤，直至各视频帧均得到对应的检测结果。
在一些实施例中，检测模块506还用于利用单帧网络对各视频帧进行单帧的物体检测，得到各视频帧对应的检测特征图；利用特征偏移网络对各视频帧进行特征点正序偏移值的检测，得到各视频帧对应的特征点正偏移值；利用特征偏移网络对各视频帧进行特征点倒序偏移值的检测，得到各视频帧对应的特征点反偏移值。
在一些实施例中，偏移模块508还用于根据视频帧对应的特征点正偏移值，将视频帧对应的上一帧视频帧的检测特征图中的特征点进行偏移，生成视频帧对应的正偏移特征图；根据视频帧对应的特征点反偏移值，将视频帧对应的下一帧视频帧的检测特征图中的特征点进行偏移，生成视频帧对应的反偏移特征图；将视频帧的检测特征图、正偏移特征图以及反偏移特征图进行特征融合，生成最终的特征图；根据最终的特征图中的坐标框确定物体信息。
关于物体检测装置的具体限定可以参见上文中对于物体检测方法的限定，在此不再赘述。上述物体检测装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行以上各个模块对应的操作。
在一些实施例中，提供了一种计算机设备，该计算机设备可以是服务器，其内部结构图可以如图6所示。该计算机设备包括通过系统总线连接的处理器、存储器、网络接口和数据库。该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统、计算机可读指令和数据库。该内存储器为非易失性存储介质中的操作系统和计算机可读指令的运行提供环境。该计算机设备的数据库用于存储数据。该计算机设备的网络接口用于与外部的终端通过网络连接通信。该计算机可读指令被处理器执行时以实现一种物体检测方法。
本领域技术人员可以理解，图6中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
一种计算机设备，包括存储器和一个或多个处理器，存储器中存储有计算机可读指令，计算机可读指令被处理器执行时实现本申请任意一个实施例中提供的物体检测方法的步骤。
一个或多个存储有计算机可读指令的非易失性存储介质，计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器实现本申请任意一个实施例中提供的物体检测方法的步骤。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一非易失性计算机可读取存储介质中，该计算机程序在执行时，可包括如上述各方法的实施例的流程。本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。
以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。

Claims (20)

  1. 一种物体检测方法，包括：
    服务器接收终端发送的待检测视频和物体检测指令，所述待检测视频中包括视频帧；
    所述服务器根据所述物体检测指令的类型确定检测方式，所述检测方式包括快速检测和精度检测；
    当所述检测方式为所述快速检测时，所述服务器根据各所述视频帧的关键帧标志位将各所述视频帧输入预先训练好的用于物体检测的单帧网络或特征偏移网络，以利用所述单帧网络或特征偏移网络对所述视频帧进行物体检测，得到检测结果；
    当所述检测方式为所述精度检测时，所述服务器分别将各所述视频帧输入所述单帧网络和所述特征偏移网络，以利用所述单帧网络对各所述视频帧进行物体检测，以及利用所述特征偏移网络根据各所述视频帧的播放顺序对各所述视频帧进行多次的物体检测；所述检测结果包括检测特征图和特征点偏移值；及
    所述服务器根据所述特征点偏移值将对应的检测特征图中的特征点进行偏移，得到偏移特征图，并根据所述检测特征图和偏移特征图中的坐标框确定物体信息。
  2. 根据权利要求1所述的方法，其特征在于，所述服务器根据各所述视频帧的关键帧标志位将各所述视频帧输入至预先训练好的用于物体检测的单帧网络或特征偏移网络，以利用所述单帧网络或特征偏移网络对各所述视频帧进行物体检测，得到检测结果，包括：
    所述服务器依次将待检测视频中的各所述视频帧作为当前帧，获取关键帧标志位；
    所述服务器基于获取的关键帧标志位，从所述单帧网络和特征偏移网络中确定当前帧对应的检测网络；
    所述服务器利用当前帧对应的检测网络对当前帧进行物体检测，得到当前帧对应的检测结果；及
    所述服务器根据所述检测结果更改所述关键帧标志位，返回获取关键帧标志位的步骤，直至各所述视频帧均得到对应的检测结果。
  3. 根据权利要求2所述的方法，其特征在于，所述服务器基于获取的关键帧标志位，从所述单帧网络和特征偏移网络中确定当前帧对应的检测网络，包括：
    当所述服务器根据所述关键帧标志位确定当前帧为关键帧时，将所述单帧网络作为当前帧对应的检测网络；及
    当所述服务器根据所述关键帧标志位确定当前帧为非关键帧时，将所述特征偏移网络作为当前帧对应的检测网络。
  4. 根据权利要求2所述的方法，其特征在于，所述服务器利用当前帧对应的检测网络对当前帧进行物体检测，得到当前帧对应的检测结果，包括：
    若当前帧对应的检测网络为所述单帧网络，所述服务器则利用所述单帧网络对所述当前帧进行单帧的物体检测，得到的检测特征图作为当前帧对应的检测结果；及
    若当前帧对应的检测网络为所述特征偏移网络，所述服务器则利用所述特征偏移网络 对当前帧以及当前帧对应的上一帧视频帧进行特征点偏移值的检测，得到的特征点偏移值作为当前帧对应的检测结果。
  5. 根据权利要求2所述的方法，其特征在于，所述服务器根据所述检测结果更改所述关键帧标志位，返回获取关键帧标志位的步骤，直至各所述视频帧均得到对应的检测结果，包括：
    当所述检测结果为检测特征图时，所述服务器更改所述关键帧标志位，并返回获取关键帧标志位的步骤，直至各所述视频帧均得到对应的检测结果；
    或
    当所述检测结果为特征点偏移值时，所述服务器基于所述特征点偏移值计算得到物体偏移均值，并将所述物体偏移均值累加至预设的物体偏移总值；及
    若所述物体偏移总值不小于门限值，所述服务器则将所述物体偏移总值清零，以及更改所述关键帧标志位，并返回获取关键帧标志位的步骤，直至各所述视频帧均得到对应的检测结果。
  6. 根据权利要求1所述的方法，其特征在于，所述特征点偏移值包括特征点正偏移值和特征点反偏移值；
    所述服务器分别将各所述视频帧输入所述单帧网络和所述特征偏移网络，以利用所述单帧网络和所述特征偏移网络分别对各所述视频帧进行物体检测，得到检测结果，包括：
    当所述服务器确定所述检测方式为所述精度检测时，利用所述单帧网络对各所述视频帧进行单帧的物体检测，得到各所述视频帧对应的检测特征图；
    所述服务器利用特征偏移网络对各所述视频帧进行特征点正序偏移值的检测，得到各所述视频帧对应的特征点正偏移值；及
    所述服务器利用特征偏移网络对各所述视频帧进行特征点倒序偏移值的检测，得到各所述视频帧对应的特征点反偏移值。
  7. 根据权利要求6所述的方法，其特征在于，所述服务器根据所述特征点偏移值将对应的检测特征图中的特征点进行偏移，得到偏移特征图，并根据所述检测特征图和偏移特征图中的坐标框确定物体信息，包括：
    所述服务器根据所述视频帧对应的特征点正偏移值，将所述视频帧对应的上一帧视频帧的检测特征图中的特征点进行偏移，生成所述视频帧对应的正偏移特征图；
    所述服务器根据所述视频帧对应的特征点反偏移值，将所述视频帧对应的下一帧视频帧的检测特征图中的特征点进行偏移，生成所述视频帧对应的反偏移特征图；
    所述服务器将所述视频帧的检测特征图、正偏移特征图以及反偏移特征图进行特征融合，生成最终的特征图；及
    根据所述最终的特征图中的坐标框确定物体信息。
  8. 一种物体检测装置，包括：
    接收模块，用于接收终端发送的待检测视频和物体检测指令，所述待检测视频中包 括视频帧；
    确定模块，用于根据所述物体检测指令的类型确定检测方式，所述检测方式包括快速检测和精度检测；
    检测模块，用于当所述检测方式为所述快速检测时，根据各所述视频帧的关键帧标志位将各所述视频帧输入预先训练好的用于物体检测的单帧网络或特征偏移网络，以利用所述单帧网络或特征偏移网络对所述视频帧进行物体检测，得到检测结果；
    检测模块还用于当所述检测方式为所述精度检测时，分别将各所述视频帧输入所述单帧网络和所述特征偏移网络，以利用所述单帧网络对各所述视频帧进行物体检测，以及利用所述特征偏移网络根据各所述视频帧的播放顺序对各所述视频帧进行多次的物体检测；所述检测结果包括检测特征图和特征点偏移值；及
    偏移模块，用于根据所述特征点偏移值将对应的检测特征图中的特征点进行偏移，得到偏移特征图，并根据所述检测特征图和偏移特征图中的坐标框确定物体信息。
  9. 根据权利要求8所述的装置，其特征在于，所述检测模块还用于
    依次将所述待检测视频中的各所述视频帧作为当前帧，获取关键帧标志位；
    基于获取的关键帧标志位，从所述单帧网络和特征偏移网络中确定当前帧对应的检测网络；
    利用当前帧对应的检测网络对当前帧进行物体检测，得到当前帧对应的检测结果；及根据所述检测结果更改所述关键帧标志位，返回获取关键帧标志位的步骤，直至各所述视频帧均得到对应的检测结果。
  10. 根据权利要求9所述的装置，其特征在于，所述检测模块还用于
    当根据所述关键帧标志位确定当前帧为关键帧时，将所述单帧网络作为当前帧对应的检测网络；及
    当根据所述关键帧标志位确定当前帧为非关键帧时，将所述特征偏移网络作为当前帧对应的检测网络。
  11. 一种计算机设备，包括存储器及一个或多个处理器，所述存储器中储存有计算机可读指令，所述计算机可读指令被所述一个或多个处理器执行时，使得所述一个或多个处理器执行以下步骤：
    接收终端发送的待检测视频和物体检测指令，所述待检测视频中包括视频帧；
    根据所述物体检测指令的类型确定检测方式，所述检测方式包括快速检测和精度检测；
    当所述检测方式为所述快速检测时，根据各所述视频帧的关键帧标志位将各所述视频帧输入预先训练好的用于物体检测的单帧网络或特征偏移网络，以利用所述单帧网络或特征偏移网络对所述视频帧进行物体检测，得到检测结果；
    当所述检测方式为所述精度检测时，分别将各所述视频帧输入所述单帧网络和所述特征偏移网络，以利用所述单帧网络对各所述视频帧进行物体检测，以及利用所述特征偏移 网络根据各所述视频帧的播放顺序对各所述视频帧进行多次的物体检测；所述检测结果包括检测特征图和特征点偏移值；及
    根据所述特征点偏移值将对应的检测特征图中的特征点进行偏移，得到偏移特征图，并根据所述检测特征图和偏移特征图中的坐标框确定物体信息。
  12. 根据权利要求11所述的计算机设备，其特征在于，所述处理器执行所述计算机可读指令时还执行以下步骤：
    依次将所述待检测视频中的各所述视频帧作为当前帧，获取关键帧标志位；
    基于获取的关键帧标志位，从所述单帧网络和特征偏移网络中确定当前帧对应的检测网络；
    利用当前帧对应的检测网络对当前帧进行物体检测，得到当前帧对应的检测结果；及根据所述检测结果更改所述关键帧标志位，返回获取关键帧标志位的步骤，直至各所述视频帧均得到对应的检测结果。
  13. 根据权利要求12所述的计算机设备，其特征在于，所述处理器执行所述计算机可读指令时还执行以下步骤：
    当根据所述关键帧标志位确定当前帧为关键帧时，将所述单帧网络作为当前帧对应的检测网络；及
    当根据所述关键帧标志位确定当前帧为非关键帧时，将所述特征偏移网络作为当前帧对应的检测网络。
  14. 根据权利要求12所述的计算机设备，其特征在于，所述处理器执行所述计算机可读指令时还执行以下步骤：
    若当前帧对应的检测网络为所述单帧网络，则利用所述单帧网络对所述当前帧进行单帧的物体检测，得到的检测特征图作为当前帧对应的检测结果；及
    若当前帧对应的检测网络为所述特征偏移网络，则利用所述特征偏移网络对当前帧以及当前帧对应的上一帧视频帧进行特征点偏移值的检测，得到的特征点偏移值作为当前帧对应的检测结果。
  15. 根据权利要求12所述的计算机设备，其特征在于，所述处理器执行所述计算机可读指令时还执行以下步骤：
    当所述检测结果为检测特征图时，更改所述关键帧标志位，并返回获取关键帧标志位的步骤，直至各所述视频帧均得到对应的检测结果；
    或
    当所述检测结果为特征点偏移值时，基于所述特征点偏移值计算得到物体偏移均值，并将所述物体偏移均值累加至预设的物体偏移总值；及
    若所述物体偏移总值不小于门限值，则将所述物体偏移总值清零，以及更改所述关键帧标志位，并返回获取关键帧标志位的步骤，直至各所述视频帧均得到对应的检测结果。
  16. 一个或多个存储有计算机可读指令的非易失性计算机可读存储介质，所述计算机 可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行以下步骤：
    接收终端发送的待检测视频和物体检测指令，所述待检测视频中包括视频帧；
    根据所述物体检测指令的类型确定检测方式，所述检测方式包括快速检测和精度检测；
    当所述检测方式为所述快速检测时，根据各所述视频帧的关键帧标志位将各所述视频帧输入预先训练好的用于物体检测的单帧网络或特征偏移网络，以利用所述单帧网络或特征偏移网络对所述视频帧进行物体检测，得到检测结果；
    当所述检测方式为所述精度检测时，分别将各所述视频帧输入所述单帧网络和所述特征偏移网络，以利用所述单帧网络对各所述视频帧进行物体检测，以及利用所述特征偏移网络根据各所述视频帧的播放顺序对各所述视频帧进行多次的物体检测；所述检测结果包括检测特征图和特征点偏移值；及
    根据所述特征点偏移值将对应的检测特征图中的特征点进行偏移，得到偏移特征图，并根据所述检测特征图和偏移特征图中的坐标框确定物体信息。
  17. 根据权利要求16所述的存储介质，其特征在于，所述计算机可读指令被所述处理器执行时还执行以下步骤：
    依次将所述待检测视频中的各所述视频帧作为当前帧，获取关键帧标志位；
    基于获取的关键帧标志位，从所述单帧网络和特征偏移网络中确定当前帧对应的检测网络；
    利用当前帧对应的检测网络对当前帧进行物体检测，得到当前帧对应的检测结果；及
    根据所述检测结果更改所述关键帧标志位，返回获取关键帧标志位的步骤，直至各所述视频帧均得到对应的检测结果。
  18. 根据权利要求17所述的存储介质，其特征在于，所述计算机可读指令被所述处理器执行时还执行以下步骤：
    当根据所述关键帧标志位确定当前帧为关键帧时，将所述单帧网络作为当前帧对应的检测网络；及
    当根据所述关键帧标志位确定当前帧为非关键帧时，将所述特征偏移网络作为当前帧对应的检测网络。
  19. 根据权利要求17所述的存储介质，其特征在于，所述计算机可读指令被所述处理器执行时还执行以下步骤：
    若当前帧对应的检测网络为所述单帧网络，则利用所述单帧网络对所述当前帧进行单帧的物体检测，得到的检测特征图作为当前帧对应的检测结果；及
    若当前帧对应的检测网络为所述特征偏移网络，则利用所述特征偏移网络对当前帧以及当前帧对应的上一帧视频帧进行特征点偏移值的检测，得到的特征点偏移值作为当前帧对应的检测结果。
  20. 根据权利要求17所述的存储介质，其特征在于，所述计算机可读指令被所述处 理器执行时还执行以下步骤：
    当所述检测结果为检测特征图时，更改所述关键帧标志位，并返回获取关键帧标志位的步骤，直至各所述视频帧均得到对应的检测结果；
    或
    当所述检测结果为特征点偏移值时，基于所述特征点偏移值计算得到物体偏移均值，并将所述物体偏移均值累加至预设的物体偏移总值；及
    若所述物体偏移总值不小于门限值，则将所述物体偏移总值清零，以及更改所述关键帧标志位，并返回获取关键帧标志位的步骤，直至各所述视频帧均得到对应的检测结果。
PCT/CN2019/103195 2019-06-25 2019-08-29 物体检测方法、装置、计算机设备和存储介质 Ceased WO2020258499A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910554748.9A CN110427816B (zh) 2019-06-25 2019-06-25 物体检测方法、装置、计算机设备和存储介质
CN201910554748.9 2019-06-25

Publications (1)

Publication Number Publication Date
WO2020258499A1 true WO2020258499A1 (zh) 2020-12-30

Family

ID=68408630

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/103195 Ceased WO2020258499A1 (zh) 2019-06-25 2019-08-29 物体检测方法、装置、计算机设备和存储介质

Country Status (2)

Country Link
CN (1) CN110427816B (zh)
WO (1) WO2020258499A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113590877A (zh) * 2021-08-05 2021-11-02 杭州海康威视数字技术股份有限公司 获取标注数据的方法及装置

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9449216B1 (en) * 2013-04-10 2016-09-20 Amazon Technologies, Inc. Detection of cast members in video content
CN107578011A (zh) * 2017-09-05 2018-01-12 中国科学院寒区旱区环境与工程研究所 视频关键帧的判定方法及装置
CN108256506A (zh) * 2018-02-14 2018-07-06 北京市商汤科技开发有限公司 一种视频中物体检测方法及装置、计算机存储介质

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9710712B2 (en) * 2015-01-16 2017-07-18 Avigilon Fortress Corporation System and method for detecting, tracking, and classifiying objects
CN108229307B (zh) * 2017-11-22 2022-01-04 北京市商汤科技开发有限公司 用于物体检测的方法、装置和设备
CN109543519B (zh) * 2018-10-15 2022-04-15 天津大学 一种用于物体检测的深度分割引导网络
CN109902620A (zh) * 2019-02-26 2019-06-18 广州二元科技有限公司 一种改良的视频物体检测提升稳定性的方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9449216B1 (en) * 2013-04-10 2016-09-20 Amazon Technologies, Inc. Detection of cast members in video content
CN107578011A (zh) * 2017-09-05 2018-01-12 中国科学院寒区旱区环境与工程研究所 视频关键帧的判定方法及装置
CN108256506A (zh) * 2018-02-14 2018-07-06 北京市商汤科技开发有限公司 一种视频中物体检测方法及装置、计算机存储介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113590877A (zh) * 2021-08-05 2021-11-02 杭州海康威视数字技术股份有限公司 获取标注数据的方法及装置

Also Published As

Publication number Publication date
CN110427816A (zh) 2019-11-08
CN110427816B (zh) 2023-09-08

Similar Documents

Publication Publication Date Title
CN108805898B (zh) 视频图像处理方法和装置
WO2020052476A1 (zh) 特征点定位方法、存储介质和计算机设备
US10489918B1 (en) Video object tracking
CN110634153A (zh) 目标跟踪模板更新方法、装置、计算机设备和存储介质
US20190122329A1 (en) Face Replacement and Alignment
US20110075025A1 (en) System and method producing high definition video from low definition video
WO2019042419A1 (zh) 图像跟踪点获取方法、设备及存储介质
CN108921131B (zh) 一种生成人脸检测模型、三维人脸图像的方法及装置
US20220351009A1 (en) Method and system for self-supervised learning of pillar motion for autonomous driving
CN111652245B (zh) 车辆轮廓检测方法、装置、计算机设备及存储介质
CN113704276B (zh) 地图更新方法、装置、电子设备及计算机可读存储介质
CN113239996B (zh) 基于目标检测的主动学习方法、装置、设备及存储介质
US20200065663A1 (en) Classifying Time Series Image Data
CN108109175A (zh) 一种图像特征点的跟踪方法及装置
CN115115991A (zh) 视频特征提取方法、装置、设备及存储介质
CN111526369B (zh) 视频的运动估计方法、装置、电子设备和存储介质
CN111882494B (zh) 位姿图处理方法、装置、计算机设备和存储介质
WO2020258499A1 (zh) 物体检测方法、装置、计算机设备和存储介质
CN111079523B (zh) 物体检测方法、装置、计算机设备和存储介质
CN109447022A (zh) 一种镜头类型识别方法及装置
CN116468753A (zh) 目标追踪方法、装置、设备、存储介质和程序产品
CN109784238A (zh) 一种确定待识别对象的方法及装置
CN111275741B (zh) 目标跟踪方法、装置、计算机设备和存储介质
CN116071396B (zh) 一种同步定位方法
CN117218693B (zh) 人脸属性预测网络生成方法、人脸属性预测方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19935530

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19935530

Country of ref document: EP

Kind code of ref document: A1