CN114332024A

CN114332024A - Abnormality detection method, apparatus, medium, and program product

Info

Publication number: CN114332024A
Application number: CN202111649690.XA
Authority: CN
Inventors: 陈术义; 杨言若; 湛波; 王玉斌
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2021-12-30
Filing date: 2021-12-30
Publication date: 2022-04-12

Abstract

The present disclosure provides an anomaly detection method, apparatus, medium and program product, relating to the field of artificial intelligence techniques such as deep learning and computer vision. One embodiment of the method comprises: acquiring a target image; determining a boundary point set in a target image according to a pre-trained depth estimation model; aligning an image domain corresponding to a key point set in a target image with an image domain corresponding to a boundary point set, and determining a deviation point set; and determining the target image as an abnormal image in response to the fact that the deviation of the deviation point set meets a preset deviation threshold value and/or the variance of the deviation point set meets a preset variance threshold value.

Description

Abnormality detection method, apparatus, medium, and program product

Technical Field

The present disclosure relates to the field of computers, and more particularly, to deep learning and computer vision, and more particularly, to an anomaly detection method, apparatus, medium, and program product.

Background

At present, an anomaly detection algorithm for data returning is crucial to model iteration of deep learning, however, the amount of data returned by the data returning is large, the effective data amount is very low, and in most cases, the effective data amount is less than 1%, and it is difficult to screen out effective data.

Taking obstacle detection as an example, two main methods are currently available on a large scale: (1) based on the manual elimination method, the returned data is analyzed into an image, an obstacle model detected by key points on an input line is manually checked data by data, and whether the detection of the model is correct or not is judged. (2) The judgment method based on the complementation of a plurality of sensors comprises the following steps: and (4) carrying out coincidence judgment on the boundary of the obstacle model detected by the key point on the line through an additional sensor such as a laser radar. When the two detected boundary regions are not consistent, the frame image is an abnormal image.

Disclosure of Invention

The embodiment of the disclosure provides an abnormality detection method, an abnormality detection device, an abnormality detection medium, and a program product.

In a first aspect, an embodiment of the present disclosure provides an anomaly detection method, including: acquiring a target image; determining a boundary point set in a target image according to a pre-trained depth estimation model; aligning an image domain corresponding to a key point set in a target image with an image domain corresponding to a boundary point set, and determining a deviation point set; and determining the target image as an abnormal image in response to the fact that the deviation of the deviation point set meets a preset deviation threshold value and/or the variance of the deviation point set meets a preset variance threshold value.

In a second aspect, an embodiment of the present disclosure provides a method for optimizing an image recognition model, including: detecting a target image output by the image recognition model as an abnormal image in response to detection by the method as described in the first aspect; and optimizing the image recognition model by using the target image in response to the number of the abnormal images exceeding a preset number threshold.

In a third aspect, an embodiment of the present disclosure provides an abnormality detection apparatus, including: an image acquisition module configured to acquire a target image; a first determination module configured to determine a set of boundary points in a target image according to a pre-trained depth estimation model; the second determining module is configured to align an image domain corresponding to the key point set in the target image with an image domain corresponding to the boundary point set, and determine a deviation point set; a third determining module configured to determine the target image as an abnormal image in response to the deviation of the set of deviation points satisfying a preset deviation threshold and/or the variance of the set of deviation points satisfying a preset variance threshold.

In a fourth aspect, an embodiment of the present disclosure provides an apparatus for optimizing an image recognition model, including: an image detection module configured to detect a target image output by the image recognition model as an abnormal image in response to detection of the target image by the apparatus as described in the third aspect; a model optimization module configured to optimize the image recognition model using the target image in response to a number of abnormal images exceeding a preset number threshold.

In a fifth aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in the first or second aspect.

In a sixth aspect, embodiments of the present disclosure propose a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method as described in the first or second aspect.

In a seventh aspect, the disclosed embodiments propose a computer program product comprising a computer program that, when executed by a processor, implements the method as described in the first or second aspect.

The abnormality detection method, apparatus, medium, and program product provided by the embodiments of the present disclosure first acquire a target image; then determining a boundary point set in the target image according to a pre-trained depth estimation model; then aligning an image domain corresponding to the key point set in the target image with an image domain corresponding to the boundary point set, and determining a deviation point set; and finally, determining that the target image is an abnormal image in response to the fact that the deviation of the deviation point set meets a preset deviation threshold value and/or the variance of the deviation point set meets a preset variance threshold value. The method comprises the steps of estimating depth of a target image based on a depth estimation model to obtain a boundary point set, aligning the boundary point set with a key point set in the target image, and determining the target image as an abnormal image when variance of the deviation point set meets a preset variance threshold and/or deviation meets a preset deviation threshold.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

Other features, objects, and advantages of the disclosure will become apparent from a reading of the following detailed description of non-limiting embodiments which proceeds with reference to the accompanying drawings. The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is an exemplary system architecture diagram in which the present disclosure may be applied;

FIG. 2 is a flow diagram of one embodiment of an anomaly detection method according to the present disclosure;

FIG. 3 is a flow diagram of one embodiment of an anomaly detection method according to the present disclosure;

FIG. 4 is a flow diagram for one embodiment of an anomaly detection method according to the present disclosure;

FIG. 5 is a flow diagram of one embodiment of a method of optimizing an image recognition model according to the present disclosure;

FIG. 6 is a schematic structural diagram of one embodiment of an anomaly detection device according to the present disclosure;

FIG. 7 is a schematic structural diagram of one embodiment of an apparatus for optimizing an image recognition model according to the present disclosure;

FIG. 8 is a block diagram of an electronic device used to implement an embodiment of the disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

It should be noted that, in the present disclosure, the embodiments and features of the embodiments may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

FIG. 1 illustrates an exemplary system architecture 100 to which embodiments of the anomaly detection methods and apparatus or the method and apparatus for optimizing an image recognition model of the present disclosure may be applied.

As shown in fig. 1, the system architecture 100 may include

terminal devices

101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the

terminal devices

101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The

terminal devices

101, 102, 103 may have installed thereon various client applications, intelligent interactive applications, such as image processing applications, image browsing applications, and so on.

The

terminal apparatuses

101, 102, and 103 may be hardware or software. When the

terminal devices

101, 102, and 103 are hardware, the terminal devices may be electronic products that perform human-Computer interaction with a user through one or more modes of a keyboard, a touch pad, a touch screen, a remote controller, voice interaction, or handwriting equipment, such as a PC (Personal Computer), a mobile phone, a smart phone, a PDA (Personal Digital Assistant), a wearable device, a PPC (Pocket PC, palmtop), a tablet Computer, a smart car machine, a smart television, a smart speaker, a tablet Computer, a laptop Computer, a desktop Computer, and the like. When the

terminal apparatuses

101, 102, 103 are software, they can be installed in the above-described electronic apparatuses. It may be implemented as multiple pieces of software or software modules, or as a single piece of software or software module. And is not particularly limited herein.

The server 105 may obtain a target image; determining a boundary point set in a target image according to a pre-trained depth estimation model; aligning an image domain corresponding to a key point set in a target image with an image domain corresponding to a boundary point set, and determining a deviation point set; and determining the target image as an abnormal image in response to the fact that the deviation of the deviation point set meets a preset deviation threshold value and/or the variance of the deviation point set meets a preset variance threshold value.

The server 105 may be hardware or software. When the server 105 is hardware, it may be implemented as a distributed server cluster composed of a plurality of servers, or may be implemented as a single server. When the server 105 is software, it may be implemented as multiple pieces of software or software modules (e.g., to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein.

It should be noted that the abnormality detection method or the method for optimizing the image recognition model provided by the embodiment of the present disclosure is generally executed by the server 105, and accordingly, an abnormality detection apparatus or an apparatus for optimizing the image recognition model is generally disposed in the server 105.

It should be understood that the number of electronic devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of electronic devices, networks, and servers, as desired for implementation.

With continued reference to FIG. 2, a flow 200 of one embodiment of an anomaly detection method according to the present disclosure is shown. The abnormality detection method may include the steps of:

step 201, acquiring a target image.

In this embodiment, the execution subject of the abnormality detection method (e.g., the server 105 shown in fig. 1) may acquire the target image from a local or via a network (e.g., the network described by 104 shown in fig. 1). The target image may be an image including at least one target object, the target image may be one or more frames of images in a video, and the target image is an image that needs to be detected whether the target image is abnormal or not.

Step 202, determining a boundary point set of the target image according to a pre-trained depth estimation model.

In an embodiment, the executing subject may obtain the set of boundary points of the target image by inputting the target image into a pre-trained depth estimation model. The set of boundary points may include boundary points of a target object in the target image.

In this embodiment, the pre-trained depth estimation model may be a convolutional neural network of various structures. The skilled person can construct the depth estimation model according to the actual application requirements (such as which layers are included, the number of layers per layer, the size of the convolution kernel, etc.). The depth estimation model can be a monocular or binocular depth estimation model.

Step 203, aligning an image domain corresponding to the key point set in the target image with an image domain corresponding to the boundary point, and determining a deviation point set.

In this embodiment, the executing entity may align the image domain corresponding to the key point set in the target image with the image domain corresponding to the boundary point set in step 202, to determine the deviation points existing between the image domain corresponding to the key point set and the image domain corresponding to the boundary point set, and form all the deviation points into a deviation point set. The image domain may be a region in the target image.

And 204, in response to the fact that the deviation of the deviation point set meets a preset deviation threshold value and/or the variance of the deviation point set meets a preset variance threshold value, determining that the target image is an abnormal image.

In this embodiment, the executing main body may determine a deviation and a variance of a deviation point when determining the deviation point between the key point set and the boundary point set in the process of executing step 203; and then, when the deviation of the deviation point set meets a preset deviation threshold value and/or the variance of the deviation point set meets a preset variance threshold value, determining that the target image is an abnormal image. The abnormal image may be an image that does not meet a relevant standard, for example, a relevant product quality, and the like.

The anomaly detection method provided by the embodiment of the disclosure includes the steps of firstly, acquiring a target image; then determining a boundary point set in the target image according to a pre-trained depth estimation model; then aligning an image domain corresponding to the key point set in the target image with an image domain corresponding to the boundary point set, and determining a deviation point set; and finally, determining that the target image is an abnormal image in response to the fact that the deviation of the deviation point set meets a preset deviation threshold value and/or the variance of the deviation point set meets a preset variance threshold value. Estimating the depth of the target image based on the depth estimation model to obtain a boundary point set, aligning the boundary point set with a key point set in the target image, and determining the target image as an abnormal image when the variance of the deviation point set meets a preset variance threshold and/or the deviation meets a preset deviation threshold.

With further reference to fig. 3, fig. 3 illustrates a flow 300 of one embodiment of an anomaly detection method according to the present disclosure. The abnormality detection method may include the steps of:

step 301, a target image is acquired.

Step 302, inputting the target image into a pre-trained key point detection model to obtain a key point set in the target image.

In this embodiment, the executing entity (for example, the server 105 shown in fig. 1) may input the target image into a pre-trained key point detection model, so as to obtain a key point set in the target image.

In this embodiment, the pre-trained keypoint detection model may be a convolutional neural network of various structures. The skilled person can construct the keypoint detection model according to the actual application requirements (such as which layers are included, the number of layers of each layer, the size of the convolution kernel, etc.).

In an example, the set of key points in the target image may also be obtained through other existing or future detection key points, for example, a manner of manually labeling key points, which is not described herein again.

Step 303, determining a boundary point set in the target image according to the pre-trained depth estimation model.

In an embodiment, the executing subject may obtain the set of boundary points of the target image by inputting the target image into a pre-trained depth estimation model.

And 304, aligning an image domain corresponding to the key point set in the target image with an image domain corresponding to the boundary point set, and determining a deviation point set.

In this embodiment, the executing entity may align the image domain corresponding to the key point set in the target image with the image domain corresponding to the boundary point set in step 202, to determine the deviation points existing between the image domain corresponding to the key point set and the image domain corresponding to the boundary point set, and form all the deviation points into a deviation point set.

And 305, in response to the fact that the deviation of the deviation point set meets a preset deviation threshold value and/or the variance of the deviation point set meets a preset variance threshold value, determining the target image as an abnormal image.

In this embodiment, the executing main body may determine a deviation and a variance of a deviation point when determining the deviation point between the key point set and the boundary point set in the process of executing step 203; and then, when the deviation of the deviation point set meets a preset deviation threshold value and/or the variance of the deviation point set meets a preset variance threshold value, determining that the target image is an abnormal image.

It should be noted that step 302 and step 303 may be performed simultaneously, or step 303 is performed before step 302, or step 302 is performed before step 303 is performed.

In this embodiment, the specific operations of

steps

301, 303, 304, and 305 have been described in detail in

steps

201, 202, 203, and 204 in the embodiment shown in fig. 2, and are not described herein again.

As can be seen from fig. 3, compared with the embodiment corresponding to fig. 2, the anomaly detection method in this embodiment highlights a step of inputting the target image into a pre-trained keypoint detection model to obtain a keypoint set in the target image. Therefore, the scheme described in this embodiment can input the target image into the pre-trained key point detection model to obtain the key point set in the target image; then determining a boundary point set in the target image according to a pre-trained depth estimation model; then aligning an image domain corresponding to the key point set in the target image with an image domain corresponding to the boundary point set, and determining a deviation point set; and finally, determining that the target image is an abnormal image in response to the fact that the deviation of the deviation point set meets a preset deviation threshold value and/or the variance of the deviation point set meets a preset variance threshold value. The method comprises the steps of obtaining a key point set in a target image based on a key point detection model, estimating depth of the target image based on a depth estimation model, aligning a boundary point set with the key point set in the target image, and determining the target image as an abnormal image when variance of a deviation point set meets a preset variance threshold and/or deviation meets a preset deviation threshold.

In some optional implementations of this embodiment, determining a set of boundary points in the target image according to a pre-trained depth estimation model includes: inputting a target image into a depth estimation model trained in advance to obtain a disparity map; carrying out inverse transformation on the disparity map to obtain a depth map; and fitting the depth map to determine a boundary point set of the depth map.

In this implementation, the executing entity may input the target image into a depth estimation model trained in advance to obtain a disparity map; then, carrying out inverse transformation on the disparity map to obtain a depth map; and then, fitting the depth map to determine a boundary point set of the depth map.

In one example, the disparity map may be an image formed by numerical values of distances of any object in the image from the photographing device.

In one example, inverse transforming the disparity map to obtain a depth map may include: when converting the disparity map into a depth map, the transformation may be performed by inverse transformation.

In one example, fitting the depth map to determine a set of boundary points for the depth map may include: and performing plane fitting on the depth map to determine a boundary point set of the depth map.

Correspondingly, in this example, the fitting may include fitting using a ranac algorithm, rotation of a space vector, or the like.

In the implementation mode, a disparity map of the target image can be obtained based on a depth estimation model trained in advance; then, carrying out inverse transformation on the disparity map to obtain a depth map of the target image; then, the depth map is fitted to obtain a boundary point set of the depth map.

In some optional implementations of this embodiment, fitting the depth map to determine a boundary point set of the depth map includes: fitting the depth map to obtain a plane map; and taking the nearest value in the longitudinal direction of the plane map as a boundary point set.

In this implementation, the execution body may fit the depth map to obtain a plan view; then, the nearest value in the longitudinal direction of the plane map is taken as a boundary point set. The longitudinal direction may be a Y direction of a coordinate system. The above-mentioned most recent value may be a minimum value.

With further reference to fig. 4, fig. 4 illustrates a flow 400 of one embodiment of an anomaly detection method according to the present disclosure. The abnormality detection method may include the steps of:

step 401, a target image is acquired.

Step 402, inputting the target image into a pre-trained key point detection model to obtain a key point set in the target image.

Step 403, determining a boundary point set in the target image according to the pre-trained depth estimation model.

Step 404, aligning an image domain corresponding to the key point set with an image domain corresponding to the boundary point set to obtain a first point set corresponding to the key point set and a second point set corresponding to the boundary point set, wherein the first point set comprises points in the key point set in an area where the image domain corresponding to the key point set and the image domain corresponding to the boundary point set are aligned; the second set of points includes points in the set of boundary points that are within the region where the image domain corresponding to the set of keypoints and the image domain corresponding to the set of boundary points are aligned.

In this embodiment, the execution subject may align the key point set and the boundary point set to obtain a first point set corresponding to the key point set and a second point set corresponding to the boundary point set. The first set of points includes points in the set of keypoints that are within a region where the image domain corresponding to the set of keypoints and the image domain corresponding to the set of boundary points are aligned. The second set of points includes points in the set of boundary points in a region where the image domain corresponding to the set of keypoints and the image domain corresponding to the set of boundary points are aligned. The aligned region may be a portion where the image domain corresponding to the key point set and the image domain corresponding to the boundary point set overlap.

Step 405, obtaining a deviation point set according to the average deviation point by point in the longitudinal direction of the image domain of the first point set and the second point set.

In this embodiment, the executing entity may obtain the deviation point set according to a deviation of each point in the longitudinal direction of the image domain of the first point set and the second point set.

And step 406, in response to that the deviation of the deviation point set meets a preset deviation threshold and/or the variance of the deviation point set meets a preset variance threshold, determining that the target image is an abnormal image.

It should be noted that, step 402 and step 403 may be executed simultaneously, or step 403 is executed before step 402 is executed, or step 402 is executed before step 403 is executed.

In this embodiment, the specific operations of

steps

401, 402, 403, and 406 have been described in detail in

steps

301, 302, 303, and 305 in the embodiment shown in fig. 3, and are not described again here.

As can be seen from fig. 4, compared with the embodiment corresponding to fig. 3, the anomaly detection method in this embodiment highlights that the key point sets and the boundary point sets are aligned to obtain the first point set corresponding to the key point set and the second point set corresponding to the boundary point set, where the first point set includes the points in the key point set that are aligned to the key point set and the boundary point set; the second point set comprises points which are aligned with the key point set and the boundary point set and are in the boundary point set; and obtaining a deviation point set according to the average deviation point by point in the longitudinal direction of the image domain of the first point set and the second point set. Therefore, the scheme described in this embodiment can input the target image into the pre-trained key point detection model to obtain the key point set in the target image; then determining a boundary point set in the target image according to a pre-trained depth estimation model; then aligning the key point set and the boundary point set to obtain a first point set corresponding to the key point set and a second point set corresponding to the boundary point set, wherein the first point set comprises points aligned with the key point set and the boundary point set and points in the key point set; the second point set comprises points which are aligned with the key point set and the boundary point set and are in the boundary point set; obtaining a deviation point set according to the average point-by-point deviation of the image domains of the first point set and the second point set in the longitudinal direction; and finally, determining that the target image is an abnormal image in response to the fact that the deviation of the deviation point set meets a preset deviation threshold value and/or the variance of the deviation point set meets a preset variance threshold value. The method comprises the steps of obtaining a key point set in a target image based on a key point detection model, estimating depth of the target image based on a depth estimation model, aligning a boundary point set with the key point set in the target image, and determining the target image as an abnormal image when variance of a deviation point set meets a preset variance threshold and/or deviation meets a preset deviation threshold.

With further reference to fig. 5, fig. 5 illustrates a flow 500 of one embodiment of a method of optimizing an image recognition model according to the present disclosure. The method of optimizing an image recognition model may comprise the steps of:

step 501, responding to the detection of the target image output by the image recognition model as an abnormal image through an abnormal detection method.

In this embodiment, an execution subject of the method for optimizing the image recognition model (for example, the server 105 shown in fig. 1) may detect whether or not the target image output by the image recognition model is an abnormal image by the abnormality detection method corresponding to any one of fig. 2 to 4, that is, not consistent with the output of the image recognition model (not an abnormal image). The above-mentioned target image recognition model may be used for recognizing the target image, for example, recognizing a category, recognizing a location, and the like.

In this embodiment, the pre-trained image recognition model may be a convolutional neural network of various structures. The skilled person can construct the image recognition model according to the actual application requirements (such as which layers are included, the number of layers per layer, the size of the convolution kernel, etc.).

Step 502, in response to the number of abnormal images exceeding a preset number threshold, optimizing an image recognition model using the target image.

In this implementation, when it is detected that the target image output by the image recognition model is an abnormal image by the abnormality detection method, the number of the abnormal images may be counted; when the number of the abnormal images exceeds a certain number threshold, determining that the recognition accuracy of the image recognition model is low, and at this time, performing optimization training on the image recognition model again through the target images and the corresponding labels until a loss function of the image recognition model after the optimization training meets a preset iteration cutoff condition.

It should be noted that the preset number threshold may be set according to the accuracy of the image recognition model and/or according to the task amount of training. The iteration cutoff condition may be set according to the image recognition accuracy.

The method for optimizing the image recognition model provided by the embodiment of the disclosure includes the steps of firstly detecting whether a target image output by the image recognition model is an abnormal image or not through an abnormality detection method, and when the target image is determined to be the abnormal image, determining whether the number of the abnormal images exceeds a preset number threshold or not; when the number of the abnormal images exceeds a preset number threshold, the target image is used for carrying out optimization training on the image recognition model, and therefore the image recognition accuracy of the image recognition model can be optimized.

With further reference to fig. 6, as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an anomaly detection apparatus, which corresponds to the method embodiment shown in fig. 2, and which is particularly applicable to various electronic devices.

As shown in fig. 6, the abnormality detection apparatus 600 of the present embodiment may include: an image acquisition module 601, a first determination module 602, a second determination module 603, and a third determination module 604. Wherein, the image obtaining module 601 is configured to obtain a target image; a first determination module 602 configured to determine a set of boundary points in the target image according to a pre-trained depth estimation model; a second determining module 603 configured to align the set of key points and the set of boundary points in the target image, and determine a set of deviation points; a third determining module 604 configured to determine the target image as an abnormal image in response to the deviation of the set of deviation points satisfying a preset deviation threshold and/or the variance of the set of deviation points satisfying a preset variance threshold.

In the present embodiment, abnormality detection apparatus 600 includes: the specific processing of the image obtaining module 601, the first determining module 602, the second determining module 603, and the third determining module 604 and the technical effects thereof can refer to the related descriptions of step 201 and step 204 in the corresponding embodiment of fig. 2, which are not repeated herein.

In some optional implementations of this embodiment, the anomaly detection apparatus further includes: the first obtaining module is configured to input the target image into a pre-trained key point detection model to obtain a key point set in the target image.

In some optional implementations of this embodiment, the first determining module 602 includes: a first obtaining unit configured to input a target image into a depth estimation model trained in advance, and obtain a disparity map; a second obtaining unit configured to perform inverse transformation on the disparity map to obtain a depth map; and the third obtaining unit is configured to fit the depth map to obtain a boundary point set of the depth map.

In some optional implementations of this embodiment, the third obtaining unit is further configured to: fitting the depth map to obtain a plane map; and taking the nearest value in the longitudinal direction of the plane map as a boundary point set.

In some optional implementations of this embodiment, the second determining module 603 is further configured to: aligning the key point set and the boundary point set to obtain a first point set corresponding to the key point set and a second point set corresponding to the boundary point set, wherein the first point set comprises points which are aligned with the key point set and the boundary point set and points in the key point set; the second point set comprises points which are aligned with the key point set and the boundary point set and are in the boundary point set; and obtaining a deviation point set according to the average point-by-point deviation of the image domains of the first point set and the second point set in the longitudinal direction.

With further reference to fig. 7, as an implementation of the method shown in the above figures, the present disclosure provides an embodiment of an apparatus for optimizing an image recognition model, where the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 5, and the apparatus may be applied to various electronic devices.

As shown in fig. 7, the apparatus 700 for optimizing an image recognition model according to the present embodiment may include: an image detection module 701 and a model optimization module 702. Wherein, the image detection module 701 is configured to respond to the detection of the target image output by the image recognition model as an abnormal image by the abnormal detection device; a model optimization module 702 configured to optimize the image recognition model using the target image in response to the number of abnormal images exceeding a preset number threshold.

In the present embodiment, in the apparatus 700 for optimizing an image recognition model: the specific processing of the image detection module 701 and the model optimization module 702 and the technical effects thereof can refer to the related description of step 501 and step 502 in the corresponding embodiment of fig. 5, which is not repeated herein.

The present disclosure also provides an electronic device, a readable storage medium, a computer program product, a traffic control system, according to embodiments of the present disclosure.

FIG. 8 illustrates a schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The calculation unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.

A number of components in the device 800 are connected to the I/O interface 805, including: an input unit 806, such as a keyboard, a mouse, or the like; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, or the like; and a communication unit 809 such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

Computing unit 801 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and the like. The calculation unit 801 executes the respective methods and processes described above, such as the abnormality detection method or the method of optimizing the image recognition model. For example, in some embodiments, the anomaly detection method or the method of optimizing the image recognition model may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and/or installed onto device 800 via ROM 802 and/or communications unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the anomaly detection method or the apparatus for optimizing an image recognition model described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured in any other suitable way (e.g. by means of firmware) as a means of performing an anomaly detection method or optimizing an image recognition model.

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.

Artificial intelligence is the subject of studying computers to simulate some human mental processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), both at the hardware level and at the software level. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing, and the like; the artificial intelligence software technology mainly comprises a computer vision technology, a voice recognition technology, a natural voice processing technology, machine learning/deep learning, a big data processing technology, a knowledge map technology and the like.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in this disclosure may be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions mentioned in this disclosure can be achieved, and are not limited herein.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. An anomaly detection method comprising:

acquiring a target image;

determining a boundary point set in the target image according to a pre-trained depth estimation model;

aligning an image domain corresponding to the key point set in the target image with an image domain corresponding to the boundary point set, and determining a deviation point set;

and determining the target image as an abnormal image in response to the fact that the deviation of the deviation point set meets a preset deviation threshold value and/or the variance of the deviation point set meets a preset variance threshold value.

2. The method of claim 1, wherein the set of keypoints in the target image is determined based on:

and inputting the target image into a pre-trained key point detection model to obtain a key point set in the target image.

3. The method of claim 1 or 2, wherein the determining the set of boundary points in the target image according to a pre-trained depth estimation model comprises:

inputting the target image into a depth estimation model trained in advance to obtain a disparity map;

carrying out inverse transformation on the disparity map to obtain a depth map;

and fitting the depth map to obtain a boundary point set of the depth map.

4. The method of claim 3, wherein said fitting the depth map to determine the set of boundary points for the depth map comprises:

fitting the depth map to obtain a plane map;

and taking the nearest value in the longitudinal direction of the plane map as the boundary point set.

5. The method according to any one of claims 1-3, wherein said aligning image fields corresponding to the set of keypoints in the target image with image fields corresponding to the set of boundary points, and determining a set of deviation points, comprises:

aligning the image domain corresponding to the key point set with the image domain corresponding to the boundary point set to obtain a first point set corresponding to the key point set and a second point set corresponding to the boundary point set, wherein the first point set comprises points in the key point set in an area where the image domain corresponding to the key point set and the image domain corresponding to the boundary point set are aligned; the second set of points includes points in the set of boundary points that are within a region where the image domain corresponding to the set of keypoints and the image domain corresponding to the set of boundary points are aligned;

and obtaining a deviation point set according to the average point-by-point deviation of the image domains of the first point set and the second point set in the longitudinal direction.

6. A method of optimizing an image recognition model, comprising:

detecting a target image output by the image recognition model as an abnormal image in response to detection of the target image by the method according to any one of claims 1 to 5;

and optimizing the image recognition model by utilizing the target image in response to the number of abnormal images exceeding a preset number threshold.

7. An abnormality detection device comprising:

an image acquisition module configured to acquire a target image;

a first determination module configured to determine a set of boundary points in the target image according to a pre-trained depth estimation model;

a second determining module, configured to align an image domain corresponding to the set of key points in the target image with an image domain corresponding to the set of boundary points, and determine a set of deviation points;

a third determination module configured to determine that the target image is an abnormal image in response to a deviation of the set of deviation points satisfying a preset deviation threshold and/or a variance of the set of deviation points satisfying a preset variance threshold.

8. The apparatus of claim 7, further comprising:

a first obtaining module configured to input the target image into a pre-trained keypoint detection model, obtaining a set of keypoints in the target image.

9. The apparatus of claim 7 or 8, wherein the first determining means comprises:

a first obtaining unit configured to input the target image into a depth estimation model trained in advance, and obtain a disparity map;

a second obtaining unit configured to perform inverse transformation on the disparity map to obtain a depth map;

and the third obtaining unit is configured to fit the depth map to obtain a boundary point set of the depth map.

10. The apparatus of claim 9, wherein the third deriving unit is further configured to:

fitting the depth map to obtain a plane map;

11. The apparatus of any of claims 7-9, wherein the second determining module is further configured to:

12. An apparatus for optimizing an image recognition model, comprising:

an image detection module configured to detect a target image output by the image recognition model as an abnormal image in response to detection of the target image by the apparatus according to any one of claims 7 to 11;

a model optimization module configured to optimize the image recognition model using the target image in response to a number of abnormal images exceeding a preset number threshold.

13. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

14. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-6.

15. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-6.