CN116567418A

CN116567418A - Image processing method, movement detection system, electronic device, and storage medium

Info

Publication number: CN116567418A
Application number: CN202310288785.6A
Authority: CN
Inventors: 杜承阳; 伍吉兵
Original assignee: Alibaba Cloud Computing Ltd
Current assignee: Alibaba Cloud Computing Ltd
Priority date: 2023-03-20
Filing date: 2023-03-20
Publication date: 2023-08-08

Abstract

The application provides an image processing method, a mobile detection system, electronic equipment and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: determining a target object from a first image acquired by an image acquisition device; according to the pixel movement information among the multi-frame images acquired by the image acquisition device, determining parameter adjustment information of the image acquisition device; the parameter adjustment information comprises angle adjustment information, wherein the angle adjustment information is used for enabling a target object to exist in an imaging range of the image acquisition device; acquiring an enlarged image of the target object by using the second image, and determining the category of the target object based on the enlarged image; the second image is an image acquired by the image acquisition device after parameter adjustment according to the parameter adjustment information. According to the technical scheme, the small targets can be accurately amplified, identified and classified, and then the detection level and accuracy of the mobile detection system on the small targets are improved.

Description

Image processing method, movement detection system, electronic device, and storage medium

Technical Field

The present disclosure relates to the field of image processing technologies, and in particular, to an image processing method, a motion detection system, an electronic device, and a storage medium.

Background

Object detection is the basis for solving high-level visual tasks such as segmentation, scene understanding, object tracking, image description, event detection and the like. Small target detection has long been a difficulty in target detection, which aims to accurately detect small targets with few visual features in the image. In a real scene, because a large number of small targets exist, the small target detection has wide application prospect and plays an important role in the fields of automatic driving, intelligent medical treatment, defect detection, aerial image analysis and the like.

Disclosure of Invention

The embodiment of the application provides an image processing method, a mobile detection system, electronic equipment and a storage medium, so as to realize automatic detection of a small target.

In a first aspect, an embodiment of the present application provides an image processing method, including: determining a target object from a first image acquired by an image acquisition device; determining parameter adjustment information of the image acquisition device according to pixel movement information among multi-frame images acquired by the image acquisition device; the parameter adjustment information comprises angle adjustment information, and the angle adjustment information is used for enabling the target object to exist in an imaging range of the image acquisition device; acquiring an enlarged image of the target object by using a second image, and determining the category of the target object based on the enlarged image; the second image is acquired by the image acquisition device after parameter adjustment according to the parameter adjustment information.

In a second aspect, an embodiment of the present application provides an image processing method, including: acquiring an image containing a target object; extracting features of an image containing the target object to obtain a feature vector of the target object; based on the difference between the characteristic vector of the target object and the characteristic vector of the class to be selected, a difference value vector and a difference degree vector are obtained; wherein the degree of variance vector is positively correlated with the variance value vector; and determining the category of the target object from at least one category to be selected according to the difference value vector and the difference degree vector.

In a third aspect, embodiments of the present application provide a movement detection system, including: moving the carrier; the image acquisition device is arranged on the mobile carrier and is used for acquiring images; the image processing device is used for receiving the acquired image and executing the method provided by any embodiment of the application.

In a fourth aspect, embodiments of the present application provide an electronic device including a memory, a processor, and a computer program stored on the memory, the processor implementing the method provided by any of the embodiments of the present application when the computer program is executed.

In a fifth aspect, embodiments of the present application provide a computer readable storage medium having a computer program stored therein, the computer program, when executed by a processor, implementing a method provided by any embodiment of the present application.

Compared with the prior art, the technical scheme of the embodiment of the application has the following advantages:

on one hand, aiming at a target object of a small target, the motion of the image acquisition device is estimated by utilizing pixel movement information among multiple frames of images acquired by the image acquisition device, so that parameter adjustment information required to be compensated by the image acquisition device is determined; the image acquisition device is controlled to perform angle compensation (reverse control) based on the parameter adjustment information, so that the image acquisition device can still acquire a target object after the parameter adjustment is performed; and then, acquiring an amplified image of the target object after the image acquisition device carries out parameter adjustment according to the parameter adjustment information, determining the class of the target object according to the amplified image, effectively tracking the small target in the mobile detection scene in real time, realizing accurate amplification and identification classification of the small target on the basis of real-time tracking, and further improving the detection level and accuracy of the mobile detection system on the small target.

On the other hand, for the target object of the small target, the difference feature vector of the target object is extracted, wherein the difference feature vector comprises a difference value vector and a difference degree vector between the feature vector of the target object and the feature vector of the class to be selected, and the difference degree vector is positively correlated with the difference value vector, namely the larger the difference value vector is, the larger the difference degree vector is, so that the difference feature vector has larger difference degree representation capability compared with the original feature vector, the difference discrimination degree can be improved, the target object with weak information density can be effectively distinguished, and the classification effectiveness problem under the weak information density is solved. Further, the algorithm can be constructed on a lightweight network, so that the algorithm can be widely applied to equipment with limited computing capacity, such as mobile equipment of mobile phones, tablet computers, vehicles and the like.

The foregoing summary is for the purpose of the specification only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the present application will become apparent by reference to the drawings and the following detailed description.

Drawings

In the drawings, the same reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily drawn to scale. It is appreciated that these drawings depict only some embodiments according to the disclosure and are not therefore to be considered limiting of its scope.

FIG. 1 illustrates an architecture diagram of a motion detection system provided by an embodiment of the present application;

fig. 2 shows an architecture diagram of a mobile inspection system provided in an embodiment of the present application;

fig. 3 shows a flowchart of an image processing method according to an embodiment of the present application;

fig. 4 shows a flowchart of an image processing method provided in the second embodiment of the present application;

fig. 5 is a diagram illustrating an example of an image processing method according to a second embodiment of the present application;

fig. 6 shows a flowchart of an image processing method provided in the third embodiment of the present application;

fig. 7 shows a block diagram of an electronic device provided in a sixth embodiment of the present application.

Detailed Description

Hereinafter, only certain exemplary embodiments are briefly described. As will be recognized by those of skill in the pertinent art, the described embodiments may be modified in various different ways without departing from the spirit or scope of the present application. Accordingly, the drawings and description are to be regarded as illustrative in nature and not as restrictive.

In order to facilitate understanding of the technical solutions of the embodiments of the present application, the following description is given of related technologies of the embodiments of the present application, and the following related technologies may be optionally combined with the technical solutions of the embodiments of the present application as an alternative, which all belong to the protection scope of the embodiments of the present application.

The following terms are used herein

And (3) mobile inspection: the target scene is inspected in real time through an image acquisition device arranged on the mobile carrier, and automatic recognition and automatic proposal setting of the illegal and illegal targets are realized through an artificial intelligence (Artificial Intelligence, AI) algorithm.

Urban mobile inspection: the urban scene is inspected in real time through an image acquisition device arranged on the mobile carrier, and automatic identification and automatic proposal setting of illegal and illegal targets are realized through an AI algorithm.

Small target: may also be referred to as tiny objects, which generally refer to objects that are more difficult to identify and classify, such as: the image size of the small object is smaller than a preset threshold. In one definition, the preset threshold may be a proportional threshold, and when the duty ratio of a certain target in the image frame is less than or equal to the preset proportional threshold, the target may be considered as a small target. In another definition, the preset threshold may be a pixel threshold, usually preset to 50 pixels by 50 pixels or 32 pixels by 32 pixels, and when the pixel resolution of the image of a certain target is smaller than the pixel threshold, the target may be considered as a small target, such as garbage scraps, tiny open fire points, and the like.

Optical flow): is the instantaneous speed of the pixel motion of the spatially moving object on the viewing imaging plane. This is also equivalent to displacement of the target point when the time interval is small, such as between two consecutive frames of video.

Optical flow method: the corresponding relation between the previous frame and the current frame is found by utilizing the change of pixels in the image sequence in the time domain and the correlation between images of adjacent frames, so that the motion information of the target between the adjacent frames is calculated.

Example 1

Aiming at the detection scene of the small target, the focal length is often required to be adjusted by an image acquisition device in the related technology, and the small target is identified and classified after being optically amplified. This approach may be used for stationary scenes, i.e. scenes where the carrier on which the image acquisition device is mounted is not moving. However, for a mobile detection scene, such as a mobile inspection scene, since the photographing position of the image pickup device continuously changes at a high speed, a small target has left the image pickup area during focusing.

Taking a mobile inspection scene as an example, as a key application in the technical field of the Internet of things (Internet of Things, ioT), AI mobile inspection is mainly oriented to urban governance scenes, namely, automatic discovery of illegal and illegal events is realized through a mobile carrier, and basic perceptibility is provided for realizing intelligent upgrading of urban governance. In the traditional inspection scene, the base layer gridding personnel often need to manually inspect the street, manually find the problem and report to the city operation center. In order to solve the problems of high labor cost and the like in the scheme. An AI automated inspection scheme is started to be tried in the related art. The scheme is characterized in that urban scene videos are often obtained in real time through cameras arranged on a carrier, and automatic AI analysis and automatic reporting are realized through vehicle-mounted computing nodes on the basis. Automated inspection schemes are commonly well-behaved in the detection of large targets (e.g., parking violations, mobile walkers). However, when the small object (such as various garbage) is oriented, if the focal length is adjusted by the image acquisition device, the small object is identified and classified after being optically enlarged, the small object is still separated from the image acquisition area during focusing due to continuous high-speed change of the shooting position of the image acquisition device, so that the detection capability is weak, and therefore, the scheme cannot be used for promise or production and delivery of mobile inspection scenes such as road garbage detection.

In the image processing method of the embodiment of the application, aiming at a target object of a small target, the motion of an image acquisition device is estimated by utilizing pixel movement information among multi-frame images acquired by the image acquisition device, so that parameter adjustment information which needs to be compensated by the image acquisition device is determined; the image acquisition device is controlled to perform angle compensation (reverse control) based on the parameter adjustment information, so that the image acquisition device can still acquire a target object after the parameter adjustment is performed; and then, acquiring an amplified image of the target object after the image acquisition device carries out parameter adjustment according to the parameter adjustment information, determining the class of the target object according to the amplified image, effectively tracking the small target in the mobile detection scene in real time, realizing accurate amplification and identification classification of the small target on the basis of real-time tracking, and further improving the detection level and accuracy of the mobile detection system on the small target.

The image processing method can be used in a movement detection scene with a small target object. In this embodiment, the vehicle on which the image capturing device is mounted is a target detection scene of a mobile vehicle (such as a vehicle or an unmanned aerial vehicle). The mobile detection scene can be a mobile inspection, such as urban mobile inspection, or an unmanned aerial vehicle follow-up detection scene, which is not limited in the embodiment of the application. The detection of the target object is performed in a mobile detection scene, such as the detection of small target garbage (such as paper scraps) in a road garbage inspection scene, and the detection of small target disaster (such as tiny open fire points) in a disaster early warning inspection scene. In the subsequent application, early warning or prompting can be performed according to the category of the target object. For example, when the category of the target object accords with a preset category, such as paper dust, tiny open fire points, and the like, early warning information or prompt information, such as image information, bounding box, physical position, category, early warning level, and the like of the target object can be generated.

Fig. 1 shows an architecture diagram of a motion detection system provided in an embodiment of the present application. As shown in fig. 1, the movement detection system includes a movement carrier 101, an image acquisition device 102, and an image processing device 103.

The mobile carrier may be a mobile carrier such as a vehicle or an unmanned aerial vehicle, and may be used to mount the image capturing device 102. Since the image capturing device 102 is mounted on the mobile carrier 101, the capturing position of the image capturing device 102 is also changed based on the change in the position of the mobile carrier 101. The image acquisition device 102 may acquire images. Specifically, the image capturing device 102 may capture a video stream, such as a patrol video, in real time, and each frame of image in the video stream is a captured image. Illustratively, the image capture device 102 may change the focal point or focal length based on the focal length adjustment, such as optically magnifying the target object; the image capturing device 102 may implement a rotation operation based on the angle adjustment, thereby adjusting the image capturing range.

The image processing apparatus 103 may acquire an image including a small target from the image acquisition apparatus 102 and perform the image processing method of the embodiment of the present application based on the image to determine the category of the small target. For example: the image processing apparatus 103 obtains a video stream from the image capturing apparatus 102, determines a small object from a plurality of frame images in the video stream, determines parameter adjustment information, such as focal length adjustment information and/or parameter adjustment information, of the image capturing apparatus 102 based on pixel movement information between the plurality of frame images, and sends control signaling (such as signaling based on a system private standard) to the image capturing apparatus 102. The control signaling is used to control the image acquisition device 102 to perform parameter adjustment operations such as focal length adjustment and/or angle adjustment. After the image capturing device 102 performs the reference adjustment, the image processing device 103 obtains another image from the image capturing device 102, and obtains an enlarged image of the small object using the image, and further obtains the category of the small object based on the enlarged image.

In one application example, the image processing apparatus 103 may be deployed on a client, which may be a hardware device (e.g., a server, a terminal device) or a hardware chip with a data processing function, and the hardware chip may be a central processing unit (Central Processing Unit, CPU), a graphics processor (Graphics Processing, GPU), a field programmable gate array (Field Programmable Gate Array, FPGA), a network processor (Neural-network Processing Unit, NPU), an AI accelerator card, or a data processor (Data Processing Unit, DPU), or the like. The client may be a functional module in a software form, an Application (APP), or the like. For example, the image processing device 103 may be mounted on a mobile carrier 101 (e.g., a vehicle, an unmanned aerial vehicle, etc.).

In another application example, the image processing apparatus 103 may be deployed on a server, where the server may be a local computing device or a cloud computing platform that provides computing power, storage, and network resources, and a mode of the cloud computing platform to provide services to the outside may be infrastructure as a service (Infrastructure as a Service, iaaS), platform as a service (Platform as a Service, paaS), software as a service (SaaS), or Data as a service (Data-as-a-service, daaS). The server side can provide an image processing function by utilizing own computing resources, and a specific application architecture can be built according to service requirements.

The mobile detection system may be specifically a mobile inspection system, that is, the image processing method provided in the embodiment of the present application may be applied to a mobile inspection scene. As shown in fig. 2, in the mobile inspection system, the mobile carrier is a vehicle; the image capturing device is a vehicle-mounted camera 202, and is used for capturing real-time inspection video to provide multi-frame images, at least part of which includes an image of a small target.

Further, the in-vehicle system may further include an in-vehicle hard disk recorder (Digital Video Recorder, DVR) for capturing video streams of the in-vehicle camera 202 in real time and completing storage and distribution, such as forwarding images to an in-vehicle algorithm box. In the mobile inspection system, the image processing device may be a vehicle algorithm box in a vehicle-mounted system, and is configured to execute the method according to the embodiment of the present application, including generating parameter adjustment information of the vehicle-mounted camera 202 and determining a category of the small target.

For example, the in-vehicle algorithm box may include an algorithm analysis module, a signaling interaction module, and a stream access module. The stream access module can acquire multi-frame images based on communication with the vehicle-mounted DVR; the algorithm analysis module can generate parameter adjustment information based on the image and determine the category of the small target; the signaling interaction module may send control signaling to the on-board DVR based on a real-time streaming protocol (Real Time Streaming Protocol, RTSP) to send parameter adjustment information to the on-board DVR. The on-vehicle DVR can control the on-vehicle camera 202 according to the parameter adjustment information in a manner of sending control signaling (such as signaling based on the private standard of the system) based on RTSP, so as to realize the parameter adjustment of the on-vehicle camera 202, such as zooming (focal length adjustment), rotation (angle adjustment) and the like.

It should be noted that, the application scenario or the application example of the image processing method provided in the embodiment of the present application is for ease of understanding, and the application of the image processing method in the embodiment of the present application is not specifically limited.

Fig. 3 shows a flowchart of an image processing method according to an embodiment of the present application. The image processing method can be applied to an image processing apparatus. As shown in fig. 3, the image processing method includes:

step S301: the target object is determined from the first image acquired by the image acquisition device.

For example, the first image may be derived based on an image acquired by the image acquisition device 102 or the in-vehicle camera 202. It should be noted that, in the embodiments of the present application, the "first" and the "second" are used to distinguish two images before and after parameter adjustment of the image capturing device, and not limited to the first image and the second image (which will be described in detail below) as adjacent frame images, and not limited to the "first image" as the first frame image.

For example, a general object detection model may be used to detect the first image, and further determine the object from the first image. The general purpose object detection model may be a regression-based image object detection model, such as YOLO (You Only Look Once) model or lightweight detection model SSD (Single Shot Multi-Box Detector) model, and the like. For example: the first image is input into the YOLO model, and bounding boxes and categories of a plurality of targets can be directly obtained. Wherein the bounding box may be directly annotated at the corresponding position of the first image for representing the identified object. These targets may include either small targets (see definition of the term section) or large targets (compared to small targets), where the class of large targets has a high confidence, but for small targets the class detection based on the generic target detection model is not good.

For example, a bounding box of a target object is used to label the target object in a first image, the coordinates of the bounding box in the first image being denoted (x, y, x+w, y+h), where x denotes the abscissa position of the bounding box in the first image, y denotes the ordinate position of the bounding box in the first image, w denotes the width of the bounding box, and h denotes the height of the bounding box.

In the embodiment of the application, the target object is a small target, that is, the image size of the small target is smaller than a preset threshold value, and the small target is difficult to identify and classify by adopting a general target detection model. For example: the preset threshold may be a proportion threshold, and the duty ratio of the target object in the first image is less than or equal to the preset proportion threshold. Another example is: the preset threshold may be a pixel threshold, such as 50 pixels by 50 pixels or 32 pixels by 32 pixels, of an image of the target object taken from the first image, with a pixel resolution less than the pixel threshold. In a mobile inspection scene, the target object is mainly small targets such as tiny garbage (such as paper scraps), tiny disaster points (such as tiny open fire points) and the like.

Step S302: according to the pixel movement information among the multi-frame images acquired by the image acquisition device, determining parameter adjustment information of the image acquisition device; the parameter adjustment information comprises angle adjustment information, and the angle adjustment information is used for enabling a target object to exist in an imaging range of the image acquisition device.

The pixel movement information is the positional change information of the same object in both images. In the time domain, the two images have a sequence. For example: the method comprises the steps that adjacent frame images, namely a previous frame image and a current frame image, are adopted, and position change information of the same target is determined according to acquisition time intervals; or non-adjacent frame images, and determining the position change information of the same target according to the acquisition time interval of the two frame images; and a plurality of groups of adjacent frame images can be adopted, and the obtained plurality of pieces of position change information are averaged to obtain the pixel movement information. The two images may include a first image, although other two images than the first image may be used for determination.

Thus, the pixel movement information may characterize an image movement vector caused by the motion of the vehicle mounted by the image capturing device, and may be expressed as (move_x, move_y). Based on the pixel movement information, the image acquisition device is subjected to motion estimation, and the target object can be tracked in real time.

For example, the optical flow method may be used to determine the pixel movement information, that is, extracting optical flow features of multiple frames of images, and then matching the optical flow features of multiple frames of images with the same target, so as to obtain the pixel movement information based on the position change information of the optical flow features of the same target. In the embodiment of the present application, when motion estimation of the image capturing device is performed, optical flow features may be adopted, and other algorithm features may also be adopted, such as Scale-invariant feature transform (Scale-invariant feature transform, SIFT) features, acceleration robust features (Speeded Up Robust Feature, SURF), and the like, which are not limited in this embodiment of the present application.

The parameter adjustment information may include angle adjustment information such as a rotation angle of the image capturing device. After determining the pixel movement information, it may be determined how much the angle adjustment information of the acquisition device, i.e. the image acquisition device is rotated, may keep the target object within the imaging range of the image acquisition device.

The angle adjustment information may be a rotation angle of the image capturing device. For example, after determining the pixel movement information (move_x, move_y), angle adjustment information of the acquisition device, that is, how much the image acquisition device is rotated, may be determined, and the target object may be maintained within the imaging range. The angle adjustment information may be (-move_x, -move_y) Scale, where Scale is a correction factor to correct the scaling (may also be called a conversion coefficient) between the coordinate system of the control signaling layer and the imaging coordinate system of the different image capturing devices.

It should be noted that the execution sequence of step S301 and step S302 may be concurrent, that is, the determination of the pixel movement information may be performed while the determination of the target object identification is performed.

Step S303: acquiring an enlarged image of the target object by using the second image, and determining the category of the target object based on the enlarged image; the second image is an image acquired by the image acquisition device after parameter adjustment according to the parameter adjustment information.

By way of example, by means of the interaction of the control signaling between the image processing device and the image acquisition device, the image acquisition device can be controlled to perform an angular adjustment such that the imaging region is moved (-move_x, -move_y), thereby ensuring that the target object can be captured in the second image.

And acquiring an enlarged image of the target object by using the second image. For example: and taking the position expectation (x, y) - (move_x, move_y) of the target object in the second image as the interest position, and carrying out optical amplification to obtain an amplified image of the target object. Another example is: and intercepting the image of the target object from the second image, and carrying out digital image amplification on the image so as to obtain an amplified image of the target object.

Because the enlarged image of the target object contains richer pixel information, the accuracy rate of the category of the target object obtained by carrying out target detection on the enlarged image of the target object is greatly improved.

In one embodiment, an enlarged image of the target object may be obtained based on optical magnification. Specifically, the parameter adjustment information includes focal length adjustment information, and in step S302, the parameter adjustment information of the image capturing device is determined according to pixel movement information between images captured by the image capturing device, including: determining angle adjustment information according to the pixel movement information; estimating a second pixel position of the target object in the second image according to the pixel movement information and the first pixel position of the target object in the first image; and determining focal length adjustment information of the image acquisition device by taking the second pixel position as a focusing and amplifying target.

The first pixel position may be represented as (x, y), the pixel movement information may be represented as (move_x, move_y), so that the estimated second pixel position may be represented as (x, y) - (move_x, move_y), and the second pixel position (x, y) - (move_x, move_y) is further used as a focusing and amplifying target, so as to obtain corresponding focal length adjustment information, and the image acquisition device performs focal length adjustment according to the focal length adjustment information, and then performs optical amplification on the image of the target object, thereby realizing accurate tracking and amplifying of the target object.

Further, in step S303, acquiring an enlarged image of the target object using the second image includes: an enlarged image of the target object is acquired from the second image. That is, after the image acquisition device optically enlarges the image of the target object by adjusting the focal length, an enlarged image of the target object can be directly obtained based on the second image acquired by the image acquisition device.

In another embodiment, an enlarged image of the target object may be obtained based on the manner in which the digital image is enlarged. Specifically, in step S303, an enlarged image of the target object is acquired using the second image, including: determining a second pixel position of the target object in the second image according to the pixel movement information and the first pixel position of the target object in the first image; acquiring an image of the target object from the second image based on the second pixel position and the parameter adjustment information; the image of the target object is magnified to obtain a magnified image of the target object.

For example: the first pixel position may be expressed as (x, y), the pixel movement information may be expressed as (move_x, move_y), and thus the second pixel position may be expressed as (x, y) - (move_x, move_y), and the bounding box of the target object in the second image may be expressed as (x, y, x+w, y+h) - (move_x, move_y), that is, the image located in the bounding box is the image of the target object, and the digital image of the image in the bounding box is amplified to obtain the amplified image of the target object. The digital image can be enlarged by adopting interpolation algorithm, such as Nearest Neighbor interpolation algorithm (Nearest Neighbor), bilinear interpolation algorithm (Bilinear Interpolation), bicubic interpolation algorithm (Bicubic Interpolation) and the like.

The amplified image of the target object is obtained based on the digital image amplification mode, and a mature image amplification algorithm can be utilized without adjusting the focal length of the image acquisition device, so that the performance requirement and cost of hardware are reduced to a certain extent.

Example two

Fig. 4 shows a flowchart of an image processing method according to an embodiment of the present application. The image processing method can be applied to an image processing apparatus. The movement detection system and the application scenario corresponding to the image processing method can be referred to the first embodiment, and will not be described herein.

As shown in fig. 4, the image processing method includes:

step S401: determining a target object from a first image acquired by an image acquisition device;

step S402: determining parameter adjustment information of the image acquisition device according to pixel movement information among images acquired by the image acquisition device; the parameter adjustment information comprises angle adjustment information, wherein the angle adjustment information is used for enabling a target object to exist in an imaging range of the image acquisition device;

step S403: determining whether the angle adjustment information accords with the current angle adjustment allowance of the image acquisition device; if so (corresponding to the yes branch of fig. 4), the process proceeds to step S404;

step S404: a magnified image of the target object is acquired using the second image.

Wherein step S401 may employ the same or similar embodiment as step S301, step S402 may employ the same or similar embodiment as step S302, and step S404 may employ the same or similar embodiment as step S303.

In step S403, the current angular adjustment margin of the image pickup device may be understood as an angular adjustment section that can be achieved by control of the image pickup device. For example, the angle adjustment information may be expressed as (-move_x, -move_y) Scale, and by determining whether (-move_x, -move_y) Scale is within a controllable angle adjustment interval of the image capturing apparatus, if so, it is determined that the angle adjustment information conforms to a current angle adjustment margin of the image capturing apparatus, and if not, it is determined that the angle adjustment information does not conform to the current angle adjustment margin of the image capturing apparatus.

In one embodiment, in step S403, if the angle adjustment information does not match the current angle adjustment margin of the image capturing apparatus (corresponding to the no branch of fig. 4), the process proceeds to step S405; step S405: an image containing the target object is detected, and the category of the target object is determined based on the detection result.

The size of the image containing the target object is smaller than a preset threshold, namely the target object is a small target. For example: the preset threshold may be a preset proportion threshold, and when the duty ratio of the target object in a certain image is smaller than or equal to the proportion threshold, the image size of the target object may be considered to be smaller than the preset threshold; another example is: the predetermined threshold may be a pixel threshold, such as 50 pixels by 50 pixels or 32 pixels by 32 pixels, and when the pixel resolution of the image of the target object is less than the pixel threshold, the image size of the target object may be considered to be less than the predetermined threshold.

For example, an image of the target object may be obtained from the first image, for example, in step S401, the first image may be detected using a general target detection model (refer to the related description in the first embodiment), so as to obtain a bounding box of the target object, so as to label the target object in the first image, and further obtain the image of the target object from the first image using the bounding box of the target object as an acquisition boundary.

For example, if the angle adjustment information (-move_x, -move_y) Scale is not within the controllable angle adjustment interval of the image capturing device, it is determined that the angle adjustment information does not conform to the current angle adjustment margin of the image capturing device, that is, when a clearer small target image cannot be captured through angle adjustment of the image capturing device due to too fast movement speed of the moving carrier, the small target classification algorithm may be utilized to detect the image containing the target object, and determine the class of the target object.

The small target classification algorithm can be a classification algorithm based on data enhancement, such as by increasing the training sample number of the small target, so as to enhance the detection performance of the classification algorithm on the small target; the small target classification algorithm can be a classification algorithm based on multi-scale learning, for example, multi-scale feature fusion is carried out by considering the characterization information of the shallow layer and the semantic information of the deep layer at the same time, so that the feature extraction of the small target is facilitated, and the detection performance of the small target is improved.

Because the target object is taken as a small target, the corresponding effective pixels are too few, and the feature vector of the target object is not enough in the correction feature richness, if the feature vector is directly input into a logistic regression model (such as a Softmax regression model) for classification, the classification result is difficult to converge and the like. Therefore, the embodiment of the application also provides a small target classification algorithm based on decision enhancement, which is used for detecting the image containing the target object and determining the category of the target object based on the detection result.

Specifically, in step S405, detecting an image including a target object, and determining a category of the target object based on the detection result may include: extracting features of an image containing a target object to obtain a feature vector of the target object; based on the difference between the characteristic vector of the target object and the characteristic vector of the class to be selected, a difference value vector and a difference degree vector are obtained; wherein the difference degree vector is positively correlated with the difference value vector; and determining the category of the target object from at least one category to be selected according to the difference value vector and the difference degree vector.

Illustratively, the image of the target object is utilized to perform feature extraction on the target object, so as to obtain a feature vector of the target object. For example, a feature extractor may be used, including but not limited to a residual network (ResNet), a lightweight network (MobileNet) that concentrates on the mobile terminal or embedded device, and the like. Illustratively, the feature vector of the target object may be denoted as X, with its dimension labeled as E.

Illustratively, there are M for M classification problems, i.e., the candidate categories. It may be assumed that each candidate class has N cluster center vectors within the feature space of the candidate class, i.e. N sub-classes for each candidate class. Thus, a clustering center vector matrix of dimension e×m×n, denoted as W, can be constructed.

By calculating the difference between the feature vector X of the target object and the feature vector W of the class to be selected, a difference value vector can be obtained. The degree of variance vector is positively correlated with the magnitude of variance vector, i.e., the greater the magnitude of variance vector. The disparity vector is a non-negative trainable parameter. It may be a set of parameter values, i.e. one parameter value for each class to be selected, or it may be a linear function of the vector of difference values.

Further, a disparity feature vector of the target object is defined, which is a sum of a disparity degree vector and a disparity value vector, so that the larger the disparity value vector is, the larger the disparity feature vector is, that is, the disparity feature vector of the target object can be understood as a feature vector of decision enhancement and also can be understood as a feature vector of disparity enhancement. Thus, the disparity feature vector has a greater disparity representation capability than the original feature vector.

Illustratively, after the difference feature vector (sum of the difference value vector and the difference degree vector) is Softmax, a classification result, that is, a class of the target object, can be obtained, and the classification result can more effectively distinguish the small-scale target object with weak information density.

In one embodiment, the distance between the feature vector of the target object and the plurality of cluster center vectors of the class to be selected may be calculated; and taking the maximum distance among the distances as a difference value vector between the characteristic vector of the target object and the characteristic vector of the candidate class.

N distances can be obtained for each class to be selected by calculating the distances between the feature vector of the target object and N clustering center vectors of the class to be selected; and selecting the maximum value from the N distances to obtain a difference value vector corresponding to the class to be selected, wherein the difference value vector is used for representing the maximum difference degree between the characteristic vector of the target object and the class to be selected. The distance can be calculated, for example, by using an L2 norm, and the maximum degree of difference between the feature vector of the target object and each candidate class, that is, the difference value vector, is obtained by maximum Pooling (Max-Pooling), which is expressed as MaxPooling (|x-w|).

Further, the difference degree vector is positively correlated with the difference value vector, that is, the larger the distance is, the larger the difference value vector is, and the larger the difference feature vector is, so the difference feature vector of the target object can be also understood as a feature vector with enhanced distance.

The disparity degree vector may be expressed as exp (M), and the disparity feature vector is a sum of the disparity degree vector and the disparity value vector, and may be expressed as MaxPooling (||x-w||) +exp (M). The difference degree vector exp (M) is positively correlated with the difference value vector MaxPooling (|X-W|), and when the distance is larger, the difference value vector MaxPooling (|X-W|) is larger, and the difference feature vector MaxPooling (|X-W|) is larger+exp (M) is larger. Compared with the original feature vector, the difference feature vector has larger difference degree representation capability, and the classification result obtained by Softmax of the difference feature vector can more effectively distinguish small-scale targets (target objects serving as small targets) with weak information density.

Illustratively, embodiments of the present application provide a small target enhanced classification model based on a decision enhanced small target classification algorithm. In the training stage of the small target enhancement classification model, the small target enhancement classification model can extract the feature vector of difference enhancement based on the image of the small target, so as to influence the convergence of the loss function. I.e. the difference feature vector is subjected to Softmax to obtain the prediction type of the target object. If the difference between the predicted category and the candidate category of the target object is larger, the loss value of the loss function is larger, gradient feedback is carried out on the loss value, and then model parameters are adjusted, so that a small target enhanced classification model meeting convergence conditions is obtained through training. In the application stage, the image of the target object is input into a trained small target enhanced classification model, and a classification result (the class of the target object) with higher accuracy can be obtained. In a mobile detection scene, through the small target enhanced classification model provided by the embodiment of the application, the specific category of the target object, such as paper dust, tiny open fire points and the like, can be judged based on the image of the target object of the small target.

Fig. 5 shows an application example diagram of the image processing method of the embodiment of the present application. In this application example, the image capturing device is specifically a camera. As shown in fig. 5, a first image acquired by a camera is acquired, a general foreground object detection algorithm (such as a general object detection model) is utilized to detect the first image, and if a small object is found, camera adjustment evaluation (such as determining angle adjustment information of the camera) is performed based on video inter-frame motion estimation (such as determining pixel movement information) of the camera; if the camera has adjustment allowance, namely angle adjustment information accords with the current angle adjustment allowance, camera control is carried out based on a motion estimation result, namely the camera is controlled to carry out angle adjustment according to the angle adjustment information, and a target object of a small target is optically amplified according to focal length adjustment information to obtain an amplified image of the target object, and then secondary detection is carried out on the amplified image of the target object, such as classification detection is carried out on the amplified image of the target object based on a general target detection model, so that the class of the target object is obtained; if the camera does not have the adjustment allowance, namely the angle adjustment information does not accord with the current angle adjustment allowance, small target enhancement classification is carried out, namely the small target enhancement classification algorithm is utilized to detect the image of the small target (target object) so as to obtain the category of the target object.

According to the image processing method, through motion estimation of the image acquisition device and real-time control in a motion process, shooting angle compensation is timely carried out, and accurate amplification of a small target is achieved. Providing a physical basis for the subsequent secondary identification and classification of the small target; by providing a small target enhancement classification model capable of effectively distinguishing small targets, the classification of the small targets is effectively judged by the small target enhancement classification model when the amplification of target objects (small targets) cannot be realized by the control of the image acquisition device. Further, by changing the model structure of the small target enhanced classification model (for example, the extracted feature vector is a difference feature vector) and adding decision penalty (difference degree vector), the discrimination of the existing classifier is effectively improved, the classification validity problem under weak information density is solved, and the detection and judgment capability of the mobile detection system on the small target is enhanced.

Example III

The small target has the problem of low resolution ratio relative to the large-scale or medium-scale target, the low-resolution small target has less visual information, the characteristic with the discrimination is difficult to extract, and the small target is extremely easy to be interfered by environmental factors, so that the detection model is difficult to accurately position and identify the small target. In the related art, the classification algorithm for the small object may be a classification algorithm based on data enhancement, for example, by increasing the number of training samples of the small object, so as to enhance the detection performance of the classification algorithm on the small object. However, this approach relies on an increased number of samples, which are more constrained by the samples, and thus affect the robustness of the algorithm. The classification algorithm of the small target can also be a classification algorithm based on multi-scale learning, for example, multi-scale feature fusion is carried out by considering the characterization information of the shallow layer and the semantic information of the deep layer at the same time, so that the feature extraction of the small target is facilitated, and the detection performance of the small target is further improved. However, this approach requires a large network structure, and cannot provide a lightweight model structure, and thus is not suitable for use in devices with limited computing power, such as mobile devices like cell phones, tablet computers, vehicles, and the like.

The embodiment of the application provides an image processing method which can be applied to detection classification scenes aiming at small targets, namely, the categories of the small targets are obtained by detecting images of the small targets. Specifically, for a target object of a small target, by extracting a difference feature vector of the target object, including a difference value vector and a difference degree vector between the feature vector of the target object and a feature vector of a class to be selected, the difference degree vector is positively correlated with the difference value vector, that is, the larger the difference value vector is, the larger the difference degree vector is, so that the difference feature vector has a larger difference degree representation capability compared with an original feature vector, thereby improving the difference discrimination degree, further effectively distinguishing the target object with weak information density, and solving the classification validity problem under weak information density. Further, the algorithm can be constructed on a lightweight network, so that the algorithm can be widely applied to equipment with limited computing capacity, such as mobile equipment of mobile phones, tablet computers, vehicles and the like.

Fig. 6 shows a flowchart of an image processing method provided according to an embodiment of the present application. The image processing method can be applied to an image processing apparatus. In an application example, the image processing apparatus may be deployed on a client, where the client may be a hardware device (such as a server or a terminal device) or a hardware chip with a data processing function, and the hardware chip may be a CPU, GPU, FPGA, NPU, AI accelerator card or a DPU. The client may be a functional module in a software form, an APP, or the like. In another application example, the image processing apparatus may be deployed on a server, where the server may be a local computing device or a cloud computing platform that provides computing power, storage, and network resources, and a mode of the cloud computing platform to provide services to the outside may be IaaS, paaS, saaS or DaaS. The server side can provide an image processing function by utilizing own computing resources, and a specific application architecture can be built according to service requirements.

As shown in fig. 6, the image processing method may include:

step S601: an image containing a target object is acquired.

The image of the target object can be acquired by the image acquisition device. The image acquisition device can be mounted on a mobile carrier, such as the image acquisition device in embodiment one or embodiment two; the image acquisition device can also be arranged on a fixed carrier, for example, the image acquisition device can be a conference room camera, a security camera or a mobile phone/tablet computer camera which is in a fixed state when acquiring images, and the like. The embodiments of the present application are not limited in this regard.

For example, the image capturing device may directly capture an image of the target object; or, the target object is identified from the full-frame image acquired by the image acquisition device, so as to obtain an image of the target object, for example: the full-picture image of the general target detection model can be adopted for detection, so that the boundary box of the target object is obtained, and the boundary box of the target object is used for intercepting the image of the target object from the full-picture image. The image acquisition device can acquire a video stream, and the full-frame image is each frame image in the video stream.

Step S602: and extracting the characteristics of the image containing the target object to obtain the characteristic vector of the target object.

Illustratively, the image containing the target object is used for extracting the characteristics of the target object, so as to obtain the characteristic vector of the target object. For example, a feature extractor may be provided, including but not limited to ResNet, mobileNet, etc. Illustratively, the feature vector of the target object may be denoted as X, with its dimension labeled as E.

Step S603: based on the difference between the characteristic vector of the target object and the characteristic vector of the class to be selected, a difference value vector and a difference degree vector are obtained; wherein the disparity vector is positively correlated with the disparity value vector.

Step S604: and determining the category of the target object from at least one category to be selected according to the difference value vector and the difference degree vector.

In one embodiment, in step S603, obtaining a difference value vector based on a difference between the feature vector of the target object and the feature vector of the candidate class may include: calculating the distance between the feature vector of the target object and a plurality of clustering center vectors of the class to be selected; the maximum distance among the plurality of distances is taken as a disparity value vector.

N distances can be obtained for each class to be selected by calculating the distances between the feature vector of the target object and N clustering center vectors of the class to be selected; and selecting the maximum value from the N distances to obtain a difference value vector corresponding to the class to be selected, wherein the difference value vector is used for representing the maximum difference degree between the characteristic vector of the target object and the class to be selected. The distance can be calculated through L2 norm, and the maximum difference degree between the feature vector of the target object and each candidate class is obtained through Max-Pooling, namely a difference value vector, which is expressed as MaxPooling (|X-W|).

Illustratively, the image processing method according to the embodiment of the application also provides a small target enhanced classification model. In the training stage of the small target enhancement classification model, the small target enhancement classification model can extract the feature vector of difference enhancement based on the image of the small target, so as to influence the convergence of the loss function. I.e. the difference feature vector is subjected to Softmax to obtain the prediction type of the target object. If the difference between the predicted category and the candidate category of the target object is larger, the loss value of the loss function is larger, gradient feedback is carried out on the loss value, and then model parameters are adjusted, so that a small target enhanced classification model meeting convergence conditions is obtained through training. In the application stage, the image of the target object is input into a trained small target enhanced classification model, and a classification result (the class of the target object) with higher accuracy can be obtained.

According to the image processing method, the degree of distinction of the existing classifier is effectively improved by changing the model structure of the small target enhanced classification model (for example, the extracted feature vector is the difference feature vector) and adding decision penalty (difference degree vector), the classification effectiveness problem under weak information density is solved, and the detection and decision capability of the small target is enhanced.

Example IV

Corresponding to the methods provided in the first and second embodiments of the present application, the embodiments of the present application provide an image processing apparatus, which may include: the target object determining module is used for determining a target object from the first image acquired by the image acquisition device; the parameter adjustment information determining module is used for determining parameter adjustment information of the image acquisition device according to pixel movement information among the multi-frame images acquired by the image acquisition device; the parameter adjustment information comprises angle adjustment information, wherein the angle adjustment information is used for enabling a target object to exist in an imaging range of the image acquisition device; the category determining module of the target object is used for acquiring an enlarged image of the target object by using the second image and determining the category of the target object based on the enlarged image; the second image is an image acquired by the image acquisition device after parameter adjustment according to the parameter adjustment information.

In one embodiment, the parameter adjustment information further includes focal length adjustment information, and the parameter adjustment information determining module is further configured to: determining angle adjustment information according to the pixel movement information; estimating a second pixel position of the target object in the second image according to the pixel movement information and the first pixel position of the target object in the first image; and determining focal length adjustment information of the image acquisition device by taking the second pixel position as a focusing and amplifying target.

In one embodiment, the category determination module of the target object is further configured to obtain a magnified image of the target object from the second image.

In one embodiment, the category determination module of the target object is further configured to: determining a second pixel position of the target object in the second image according to the pixel movement information and the first pixel position of the target object in the first image; acquiring an image of the target object from the second image based on the second pixel position and the parameter adjustment information; the image of the target object is magnified to obtain a magnified image of the target object.

In one embodiment, the apparatus further comprises a determining module configured to determine whether the angle adjustment information matches a current angle adjustment margin of the image capturing apparatus before acquiring the enlarged image of the target object using the second image; in the case of coincidence, the respective module is caused to perform acquisition of a magnified image of the target object using the second image.

In one embodiment, the apparatus further comprises: the judging module is used for determining whether the angle adjustment information accords with the current angle adjustment allowance of the image acquisition device before the second image is used for acquiring the enlarged image of the target object; and the image detection module is used for detecting the image containing the target object under the condition of non-coincidence and determining the category of the target object based on the detection result.

In one embodiment, the image detection module is configured to: extracting features of an image containing a target object to obtain a feature vector of the target object; based on the difference between the characteristic vector of the target object and the characteristic vector of the class to be selected, a difference value vector and a difference degree vector are obtained; wherein the difference degree vector is positively correlated with the difference value vector; and determining the category of the target object from at least one category to be selected according to the difference value vector and the difference degree vector.

In one embodiment, the image detection module is further configured to: calculating the distance between the feature vector of the target object and a plurality of clustering center vectors of the class to be selected; the maximum distance among the plurality of distances is taken as a disparity value vector.

In one embodiment, the duty cycle of the target object in the first image is less than a preset scale threshold, or the pixel resolution of the target object in the first image is less than a preset pixel threshold.

The functions of each module in each device of the embodiments of the present application may be referred to the corresponding descriptions in the above methods, and have corresponding beneficial effects, which are not described herein.

Example five

Corresponding to the method provided in the third embodiment of the present application, the embodiment of the present application provides an image processing apparatus, which may include: the image acquisition module is used for acquiring an image containing a target object; the feature vector extraction module is used for extracting features of the image containing the target object to obtain a feature vector of the target object; the difference determining module is used for obtaining a difference value vector and a difference degree vector based on the difference between the characteristic vector of the target object and the characteristic vector of the class to be selected; wherein the degree of variance vector is positively correlated with the variance value vector; and the category determining module is used for determining the category of the target object from at least one category to be selected according to the difference value vector and the difference degree vector.

In one embodiment, the variance determining module is further configured to: calculating the distance between the feature vector of the target object and a plurality of clustering center vectors of the class to be selected; and taking the maximum distance among a plurality of the distances as the difference value vector.

Example six

Fig. 7 is a block diagram of an electronic device used to implement an embodiment of the present application. As shown in fig. 7, the electronic device includes: a memory 701 and a processor 702, the memory 701 storing a computer program executable on the processor 702. The processor 702, when executing the computer program, implements the methods of the embodiments described above. The number of memories 701 and processors 702 may be one or more.

The electronic device further includes: and the communication interface 703 is used for communicating with external equipment and performing data interaction transmission.

If the memory 701, the processor 702, and the communication interface 703 are implemented independently, the memory 701, the processor 702, and the communication interface 703 may be connected to each other and perform communication with each other through buses. The bus may be an industry standard architecture (Industry Standard Architecture, ISA) bus, an external device interconnect (Peripheral Component Interconnect, PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in fig. 7, but not only one bus or one type of bus.

Alternatively, in a specific implementation, if the memory 701, the processor 702, and the communication interface 703 are integrated on a chip, the memory 701, the processor 702, and the communication interface 703 may communicate with each other through internal interfaces.

Embodiments of the present application provide a computer readable storage medium storing a computer program which, when executed by a processor, implements a method provided in any of the embodiments of the present application.

The embodiment of the application also provides a chip, which comprises a processor, and is used for calling the instructions stored in the memory from the memory and running the instructions stored in the memory, so that the communication device provided with the chip executes the method provided by any embodiment of the application.

The embodiment of the application also provides a chip, which comprises: the input interface, the output interface, the processor and the memory are connected through an internal connection path, the processor is used for executing codes in the memory, and when the codes are executed, the processor is used for executing the method provided by any embodiment of the application.

It should be appreciated that the processor may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processors, digital signal processors (Digital Signal Processing, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field Programmable gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or any conventional processor or the like. It is noted that the processor may be a processor supporting an advanced reduced instruction set machine (Advanced RISC Machines, ARM) architecture.

Alternatively, the memory may include a read-only memory and a random access memory, and may also include a nonvolatile random access memory. The memory may be volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), programmable ROM (PROM), erasable Programmable ROM (EPROM), electrically Erasable EPROM (EEPROM), or flash Memory, among others. Volatile memory can include random access memory (Random Access Memory, RAM), which acts as external cache memory. By way of example, and not limitation, many forms of RAM are available. For example: static RAM (SRAM), dynamic RAM (Dynamic Random Access Memory, DRAM), synchronous DRAM (SDRAM), double Data Rate Synchronous DRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), and Direct RAM (DRRAM).

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions in accordance with the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. Computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another.

It should be noted that, user information (including, but not limited to, user equipment information, user personal information, user operation information, etc.) and data (including, but not limited to, data for processing, analyzed data, stored data, presented data, etc.) and the like, which are information and data authorized by a user or sufficiently authorized by each party, and the collection, use and processing of related information and data need to comply with related laws and regulations and standards of related countries and regions, and are provided with corresponding operation entries for the user to select authorization or rejection.

In the description of the present specification, a description referring to the terms "one embodiment," "some embodiments," "examples," "particular examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.

Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.

Any process or method description in a flowchart or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process. And the scope of the preferred embodiments of the present application includes additional implementations in which functions may be performed in a substantially simultaneous manner or in an opposite order from that shown or discussed, including in accordance with the functions that are involved.

Logic and/or steps represented in the flow diagrams or otherwise described herein, e.g.: may be considered a ordered listing of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.

It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the methods of the embodiments described above may be performed by a program that, when executed, comprises one or a combination of the steps of the method embodiments, instructs the associated hardware to perform the method.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules described above, if implemented in the form of software functional modules and sold or used as a stand-alone product, may also be stored in a computer-readable storage medium. The storage medium may be a read-only memory, a magnetic or optical disk, or the like.

The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think of various changes or substitutions within the technical scope of the present application, and these should be covered in the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image processing method, comprising:

determining a target object from a first image acquired by an image acquisition device;

determining parameter adjustment information of the image acquisition device according to pixel movement information among multi-frame images acquired by the image acquisition device; the parameter adjustment information comprises angle adjustment information, and the angle adjustment information is used for enabling the target object to exist in an imaging range of the image acquisition device;

acquiring an enlarged image of the target object by using a second image, and determining the category of the target object based on the enlarged image; the second image is acquired by the image acquisition device after parameter adjustment according to the parameter adjustment information.

2. The method of claim 1, wherein the parameter adjustment information further comprises focus adjustment information, and determining the parameter adjustment information of the image capturing device according to pixel movement information between the plurality of frames of images captured by the image capturing device comprises:

Determining the angle adjustment information according to the pixel movement information;

estimating a second pixel position of the target object in the second image according to the pixel movement information and the first pixel position of the target object in the first image;

and determining focal length adjustment information of the image acquisition device by taking the second pixel position as a focusing and amplifying target.

3. The method of claim 2, wherein acquiring the magnified image of the target object using the second image comprises:

and acquiring the enlarged image from the second image.

4. The method of claim 1, wherein acquiring the magnified image of the target object using the second image comprises:

determining a second pixel position of the target object in the second image according to the pixel movement information and the first pixel position of the target object in the first image;

acquiring an image of the target object from the second image based on the second pixel position and the parameter adjustment information;

and amplifying the image of the target object to obtain the amplified image.

5. The method of claim 1, wherein prior to acquiring the magnified image of the target object using the second image, further comprising:

Determining whether the angle adjustment information accords with the current angle adjustment allowance of the image acquisition device;

and in case of coincidence, performing the acquisition of the enlarged image of the target object by using the second image.

6. The method of claim 1, wherein prior to acquiring the magnified image of the target object using the second image, further comprising:

and if the image containing the target object does not accord with the classification, detecting the image containing the target object and determining the classification of the target object.

7. The method of claim 6, wherein detecting the image containing the target object and determining the class of the target object based on the detection result comprises:

extracting features of an image containing the target object to obtain a feature vector of the target object;

based on the difference between the characteristic vector of the target object and the characteristic vector of the class to be selected, a difference value vector and a difference degree vector are obtained; wherein the degree of variance vector is positively correlated with the variance value vector;

and determining the category of the target object from at least one category to be selected according to the difference value vector and the difference degree vector.

8. The method of claim 7, wherein deriving a disparity value vector based on a disparity between the feature vector of the target object and a feature vector of a candidate class comprises:

calculating the distance between the feature vector of the target object and a plurality of clustering center vectors of the class to be selected;

and taking the maximum distance among a plurality of the distances as the difference value vector.

9. The method of any of claims 1 to 8, wherein a duty cycle of the target object in the first image is less than a preset scale threshold or a pixel resolution of the target object in the first image is less than a preset pixel threshold.

10. An image processing method, comprising:

acquiring an image containing a target object;

11. The method of claim 10, wherein deriving a disparity value vector based on a disparity between the feature vector of the target object and a feature vector of a candidate class comprises:

12. A movement detection system, comprising:

moving the carrier;

the image acquisition device is arranged on the mobile carrier and is used for acquiring images;

image processing means for receiving an acquired image and performing the method of any of claims 1 to 11.

13. An electronic device comprising a memory, a processor and a computer program stored on the memory, the processor implementing the method of any one of claims 1 to 11 when the computer program is executed.

14. A computer readable storage medium having a computer program stored therein, which when executed by a processor, implements the method of any of claims 1 to 11.