CN117831009A - Dynamic abnormal state identification method and device, electronic equipment and storage medium - Google Patents

Abstract

The application relates to the technical field of computers, in particular to a dynamic abnormal state identification method, a dynamic abnormal state identification device, electronic equipment and a storage medium, which belong to the technical means of intelligent driving. In the technical scheme provided by the embodiment of the application, a plurality of images of a target in a visual field range are acquired based on unit time and unit frequency, and the images are respectively input into an anomaly identification model to determine whether the images have anomaly characteristics or not; and when the number of abnormal features is greater than a preset number, determining that the target in the visual field range has motion abnormality. According to the technical scheme, on the premise that no biological feature sensor is added, a pure machine vision means is utilized, a novel fatigue prediction flow method is provided, image data enhancement is preferentially carried out by optimizing a visual recognition algorithm structure, the influence of background light on machine vision is eliminated, and therefore the recognition accuracy of fatigue driving under extreme light conditions is improved.

Description

Dynamic abnormal state identification method and device, electronic equipment and storage medium

Technical Field

The application relates to the technical field of computers, in particular to a dynamic abnormal state identification method, a dynamic abnormal state identification device, electronic equipment and a storage medium, which belong to the technical means of intelligent driving.

Background

The existing fatigue monitoring system cannot accurately predict the state of a driver, and especially when the driver runs at night, the driver is easy to miss detection and misdetect in a mode simply based on visual identification due to background light.

The current driving fatigue identification mainly comprises the following methods: 1. and simulating a running track based on the running state (such as a yaw angle, acceleration and a steering angle), judging that the driving behavior of the deviation of the yaw rate and the ideal track exceeds a threshold value is fatigue driving, and performing forced intervention correction by the vehicle controller. Specific details correspond to patent 1. 2. Based on the detection of the physical condition of the driver (such as heart rate, pulse and grip strength) and the state of the vehicle (such as vehicle speed and direction), whether the driver is in a fatigue driving state or not is judged. Specific details correspond to patent 2.

Both of the above methods are conventional detection methods based on sensors, rather than machine learning detection methods based on deep learning models. The detection method based on machine learning at present comprises the following steps: 1. face key point recognition is carried out based on a YOLO V3 algorithm, 6 points around eyes are extracted, and whether a driver is in a fatigue state or not is judged by estimating the opening and closing states and the time length of the eyes according to the distance ratio of the longitudinal direction to the transverse direction of the eyes. And specifically corresponds to patent 3. 2. Human skeleton monitoring is carried out based on Fast R-CNN lightweight convolutional neural network, and whether the human skeleton is in a fatigue state or not is judged according to the human body posture and the positions of the head and the hand.

The above prior art has the following drawbacks:

disadvantage 1: under the conditions of driving at night or strong light, the facial features of a driver cannot be accurately identified, fatigue judgment cannot be carried out, and missed detection and false detection are caused.

Disadvantage 2: the method for detecting the biological characteristics of the driver by using the brain wave and other sensors can improve the accuracy, but increases the cost of the whole vehicle and the electrical complexity of the whole vehicle, and meanwhile, the normal driving behavior of the driver can be influenced by wearing the equipment.

Disadvantage 3: at present, the detection accuracy is not high enough whether the detection method is based on a deep learning model or a biological characteristic.

The fatigue driving recognition patents under the normal light conditions are all fatigue driving recognition patents, and the system is invalid due to the loss of characteristic information in the dark or strong light condition. Therefore, it is desirable to provide a method that is free from inaccuracy in recognition due to changes in external environment such as light.

Disclosure of Invention

In order to solve the problems existing in the prior art, the embodiment of the application provides a fatigue driving state identification method which can not depend on biological characteristics and solve the problem of inaccurate characteristic identification caused by external environment such as light change.

In order to achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

in a first aspect, a method for identifying a dynamic abnormal state is provided, the method comprising: acquiring a plurality of images of a target in a visual field range based on unit time and unit frequency, and respectively inputting the images into an anomaly identification model to determine whether the images have anomaly characteristics; the abnormal recognition model is obtained through training through a data set containing abnormal labels; and when the number of the abnormal features is larger than a preset number, determining that the target in the visual field range has motion abnormality.

Further, the anomaly identification model is a transducer model established based on an outlooker mechanism; determining whether the plurality of images have abnormal features through determining whether the plurality of images have abnormal features in an abnormal recognition model, and inputting the images into the abnormal recognition model, wherein the determining whether the plurality of images have abnormal features comprises the following steps: extracting the characteristics of the image, carrying out fine coding on the local spatial characteristics to obtain fine coding information, and generating global information by the fine coding information, wherein the global information is the spatial characteristic information of the image generated in an abnormal state.

Further, the method further comprises the step of performing enhancement processing on the image before the image is input into the anomaly identification model, wherein the enhancement processing is performed on the image.

Further, the enhancement processing includes dividing the image according to a dividing unit to obtain a plurality of independent image blocks with the same size, performing histogram uniform distribution processing on the independent image blocks to obtain gray values of pixel points in the independent image blocks, and uniformly distributing the pixel points corresponding to the pixel points exceeding the gray threshold to all other pixels to obtain the enhanced image.

Further, the anomaly identification model includes: the segmentation unit is used for segmenting the image to obtain a plurality of image blocks and converting each image block into a matrix with the same size; the coding unit is established based on an outlooker mechanism and is used for carrying out fine coding on the image, converting the common features in the image into spatial information of a matrix and labeling the weights of the features; and the transducer unit is used for generating global information for the fine coded information.

Further, the coding unit comprises four consecutive coding blocks, each of the coding blocks comprising a first branch for generating a refined representation and a second branch for generating aggregated global information.

Further, the first branch comprises a normalization layer, an outlooker attention layer and a regularization layer; the second branch includes a normalization layer, a multi-layer perceptron module, and a regularization layer.

Further, the converter unit comprises 14 continuous converter blocks, each converter block comprises a mask self-attention module and a front feedback layer, a normalization layer is arranged between the mask self-attention module and the front feedback layer, and a normalization layer is arranged at the output end of the front feedback layer.

In a second aspect, there is provided a dynamic abnormal state identification apparatus, the apparatus comprising: the abnormal feature identification code module is used for acquiring a plurality of images of the target in the visual field range based on unit time unit frequency, and respectively inputting the images into the abnormal identification model to determine whether the images have abnormal features or not; the abnormal recognition model is obtained through training through a data set containing abnormal labels; and the abnormality judging module is used for determining that the target in the visual field range has motion abnormality when the abnormal characteristic quantity is larger than a preset quantity.

In a third aspect, there is provided an electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, which when executed by the one or more processors, cause the one or more processors to implement the method as described in any of the preceding claims.

In a fourth aspect, there is provided a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements a method as described in any of the above.

In a fifth aspect, a computer program product is provided, comprising a computer program which, when executed by a processor, implements a method according to any of the preceding claims.

In the technical scheme provided by the embodiment of the application, a plurality of images of a target in a visual field range are acquired based on unit time and unit frequency, and the images are respectively input into an anomaly identification model to determine whether the images have anomaly characteristics or not; and when the number of abnormal features is greater than a preset number, determining that the target in the visual field range has motion abnormality. According to the technical scheme, on the premise that no biological feature sensor is added, a pure machine vision means is utilized, a novel fatigue prediction flow method is provided, image data enhancement is preferentially carried out by optimizing a visual recognition algorithm structure, the influence of background light on machine vision is eliminated, and therefore the recognition accuracy of fatigue driving under extreme light conditions is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

The methods, systems, and/or programs in the accompanying drawings will be described further in terms of exemplary embodiments. These exemplary embodiments will be described in detail with reference to the drawings. These exemplary embodiments are non-limiting exemplary embodiments, wherein the exemplary numbers represent like mechanisms throughout the various views of the drawings.

Fig. 1 is a flowchart of a dynamic abnormal state identification method provided in an embodiment of the present application.

Fig. 2 is a block diagram of a dynamic abnormal state recognition device according to an embodiment of the present application.

Fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Fig. 4 is a schematic structural diagram of an anomaly identification model provided in an embodiment of the present application.

Detailed Description

In order to better understand the technical solutions described above, the following detailed description of the technical solutions of the present application is provided through the accompanying drawings and specific embodiments, and it should be understood that the specific features of the embodiments and embodiments of the present application are detailed descriptions of the technical solutions of the present application, and not limit the technical solutions of the present application, and the technical features of the embodiments and embodiments of the present application may be combined with each other without conflict.

In the following detailed description, numerous specific details are set forth by way of examples in order to provide a thorough understanding of the relevant teachings. However, it will be apparent to one skilled in the art that the present application may be practiced without these details. In other instances, well-known methods, procedures, systems, components, and/or circuits have been described at a relatively high-level, without detail, in order to avoid unnecessarily obscuring aspects of the present application.

The flowcharts are used in this application to describe implementations performed by systems according to embodiments of the present application. It should be clearly understood that the execution of the flowcharts may be performed out of order. Rather, these implementations may be performed in reverse order or concurrently. Additionally, at least one other execution may be added to the flowchart. One or more of the executions may be deleted from the flowchart.

Before describing embodiments of the present invention in further detail, the terms and terminology involved in the embodiments of the present invention will be described, and the terms and terminology involved in the embodiments of the present invention will be used in the following explanation.

(1) In response to a condition or state that is used to represent the condition or state upon which the performed operation depends, the performed operation or operations may be in real-time or with a set delay when the condition or state upon which it depends is satisfied; without being specifically described, there is no limitation in the execution sequence of the plurality of operations performed.

(2) Based on the conditions or states that are used to represent the operations that are being performed, one or more of the operations that are being performed may be in real-time or with a set delay when the conditions or states that are being relied upon are satisfied; without being specifically described, there is no limitation in the execution sequence of the plurality of operations performed.

First, some terms that may appear in the embodiments of the present application will be explained.

Transformer model: is based on the neural network model established by the attention mechanism

Outlooker mechanism: an attention mechanism.

In the prior art, aiming at a fatigue driving recognition method, whether a driver is in a fatigue state is mainly recognized through a biological characteristic and an image characteristic recognition method. Wherein the biological characteristics are mainly heart rate, pulse and grip strength, wherein the acquisition of the characteristics is performed by adding a biological sensor, and the added sensor is liable to cause inconvenience to a driver. The method mainly extracts key points of human faces, human body postures and head recognition by an image processing technology aiming at an image characteristic recognition method, and determines whether a driver is in a fatigue state or not according to the position information of the characteristic points; for example, for extracting 6 points around the eyes, the opening and closing state and the duration of the eyes are estimated by the ratio of the distances of the longitudinal direction and the transverse direction of the eyes to judge whether the driver is in a tired state. However, this solution can lead to failure of the system in the dark or in the presence of intense light.

With respect to the above background art, the following prior patent information is obtained by searching:

1. the invention name is as follows: driver fatigue detection, invention patent application number: 201510096624.2.

technical summary: data relating to the yaw rate of the vehicle over a period of time is collected. An actual trajectory and an ideal trajectory of the vehicle over a period of time are calculated based in part on the yaw rate data. A deviation of the yaw rate associated with the actual trajectory from the yaw rate associated with the ideal trajectory is determined. The variance of the deviation is calculated. If the variance exceeds a predetermined variance threshold, an indication of a dangerous driving condition is provided.

2. The invention name is as follows: automobile driver fatigue detection system and method, invention patent application number: 201910578854.0.

technical summary: the invention relates to the technical field of fatigue driving, in particular to a system and a method for detecting fatigue of an automobile driver. Comprises a vehicle-mounted computer, a respiration sensor, a pulse sensor, a route detection module the device comprises a vehicle speed detection module, a camera, a face recognition module and a voice alarm module. When the respiratory rate of the driver, the pulse rate of the driver and the eye closing time of the driver all exceed the calibration values; the running path of the vehicle is twisted and the steering lamp is not turned on, and the running path is not corrected within the calibration time; when the vehicle speed is gradually increased, the driver is judged to be in a fatigue driving state and an alarm is sent. The respiration frequency of the driver, the pulse frequency of the driver, the vehicle driving path data, the vehicle speed information and the eye closing duration data of the driver are used as the fatigue degree judgment standards of the driver, so that the limitation of a detection method can be eliminated, and meanwhile, the accuracy of a detection result can be ensured.

3. The invention name is as follows: driver fatigue monitoring system based on deep learning, and the invention has the following patent application number: 202010735477.X.

Technical summary: the invention discloses a driver fatigue monitoring system based on deep learning and a use method thereof, wherein the driver fatigue monitoring system based on deep learning comprises an image acquisition module, a face detection module, an image processing module and a server module; the invention provides a driver fatigue monitoring system based on deep learning and a use method thereof, and the fatigue monitoring system can be used for monitoring the fatigue driving, dangerous driving, line of sight deviation and other behaviors of a driver on a current hardware platform.

4. The invention name is as follows: automobile driver fatigue detection system and method, invention patent application number: 201910578854.0.

technical summary: the invention provides a driver fatigue monitoring and early warning method and a system, wherein the method comprises the steps of collecting images and outputting fatigue early warning information, and further comprises the following steps: human body detection is carried out on the acquired images; detecting a human skeleton model; performing head and/or hand positioning; judging the positions of the head and/or the hands; and carrying out fatigue judgment according to the human skeleton model and/or the head and/or hand positions. The fatigue monitoring and early warning method and system for the driver can effectively monitor and early warn the fatigue of the driver aiming at the characteristic of the closed space driving of the deep sea long-distance navigation equipment, and overcome the defect that the fatigue monitoring technology based on the face image processing is not suitable for fatigue monitoring of deep sea long-distance sea personnel. Meanwhile, the neural network is repeatedly trained by a large number of abundant samples, so that the detection precision and speed of the neural network on the related parts of the human body are higher, and the robustness is better.

The above-disclosed patents are directed to the recognition of fatigue driving states by collecting biological characteristics of a driver in patent 1 and patent 2, and the recognition of fatigue driving states by a deep learning method by collecting facial characteristics and human body state characteristics of a driver in comparison document 3 and comparison document 4.

Referring to fig. 1, in order to solve the problems of the prior art and the technical problems in the provided prior art, the present application is to provide a dynamic abnormal state identification method for identifying whether a driver has fatigue driving behavior/state, where the method specifically includes:

s110, acquiring a plurality of images of the target in the visual field range based on unit time and unit frequency, and respectively inputting the images into an anomaly identification model to determine whether the images have anomaly characteristics.

In the embodiment of the application, the abnormal characteristic is a fatigue state of the driver, and the fatigue state is obtained by acquiring an image including the driver through an image acquisition device arranged in front of the driver cabin facing the driver, and identifying the image including the driver through an abnormal identification model. Wherein the anomaly identification model is obtained through training through a data set containing anomaly tags. That is, the images for the anomaly identification model containing the fatigue state tags are obtained by training, and the images for the non-fatigue state tags, specifically the awake tag images, are also included for the training data in order to increase the integrity of the training, and the non-vigilant tag images are included.

In the anomaly identification model in the embodiment of the application, whether the driver is in a fatigue state or not in the image is obtained by identifying the collected image, so that the problem that the image expression capability is interfered by extreme light is avoided in order to ensure the expressive capability of a plurality of video images input into the anomaly identification model, and the image is required to be enhanced before the plurality of video frame images are identified.

In the embodiment of the application, aiming at the enhancement processing based on the self-adaptive histogram method, specifically, an image is cut into 64 non-overlapping images with the size of 8X8, histogram uniform distribution processing is performed on each image to obtain gray values of pixel points in the related image, and the pixel points corresponding to the gray threshold exceeding the gray threshold are redistributed uniformly to all other pixels to obtain the enhanced image.

Wherein, the principle formula of uniformly distributing the CLAHE aiming at the histogram in a single area is as follows:

(when H _i ＜H _max When, at that time);

(when H _i ＞H _max When, at that time); wherein i represents a gray value, H _i Represents the number of pixels corresponding to the gray value i, H _max Representing the set maximum gray value threshold.

And iterates the process until the portion exceeding the maximum gray threshold is negligible. The 64 areas execute the process respectively, namely the image contrast redistribution is completed, and the contrast of the image is softer.

The image used for carrying out the subsequent abnormality identification is obtained through the image enhancement processing method.

Referring to fig. 4, a schematic structural diagram of an anomaly identification model according to an embodiment of the present application includes a segmentation unit, a coding unit, and a transducer unit. The image segmentation unit is used for segmenting the image to obtain a plurality of image blocks, and each image block is converted into a matrix with the same size. The method is established based on an outlooker mechanism aiming at a coding unit, and the coding unit is used for carrying out fine coding on the image, converting the common features in the image into spatial information of a matrix and labeling the weights of the features. For the transducer unit, for generating global information for the fine encoded information.

Specifically, the structure for the dividing unit includes 4 convolution layers.

Specifically, the coding unit comprises four continuous coding blocks, each coding block comprises a first branch and a second branch, the first branch is used for generating a refined representation, and the second branch is used for generating aggregated global information. The method comprises the steps of aiming at a first branch, including a normalization layer, an outlooker attention layer and a regularization layer; the second branch includes a normalization layer, a multi-layer perceptron module, and a regularization layer.

Specifically, the converter unit comprises 14 continuous converter blocks, each converter block comprises a mask self-attention module and a front feedback layer, a normalization layer is arranged between the mask self-attention module and the front feedback layer, and a normalization layer is arranged at the output end of the front feedback layer.

According to the embodiment of the application, the outlooker mechanism is fused with the transducer model framework, the deep learning neural network for fatigue detection is built, and the fatigue detection can be realized through the image recognition method through the model.

The data set with fatigue state label image, wakefulness label image and non-vigilance label image is constructed as described above for the training of this model, and the data set is divided into three sub-data sets, namely a training data set, a testing data set and a verification data set in sequence. The training data set is a group of marked images, is used for training a model and modifying parameters through a supervised learning method, and enables the model to find common facial features of a face which is labeled as fatigue through a large amount of data. Based on the test dataset, the performance of the model can be quantified, such as accuracy and sensitivity. Finally, the validation data set is also used to provide unbiased assessment as is the test data set, but it is more focused on tuning of the super-parameters.

The fatigue state label in the data set can be established by establishing a basic fatigue state label, the establishing process carries out label establishment through an expert, a plurality of basic fatigue state label images are obtained, the characteristics corresponding to the basic fatigue state label are obtained through characteristic extraction of the plurality of basic fatigue state label images, the characteristics are fused to obtain common characteristics, the corresponding fatigue state label image is screened out in the data set based on the common characteristics, and the complexity and the screening cost of manual screening are reduced through the process.

The data set in the embodiment of the application has men and women, the background is high light or dark light, and the effect under extreme light can be verified.

The loss function adopted for training of the embodiment of the application is a cross entropy loss function, and the two classification problems are optimized. The principle formula is as follows:

and S120, when the number of abnormal features is larger than a preset number, determining that the target in the visual field range has motion abnormality.

For the step S110 of identifying whether an abnormal state exists in one image, and for the dynamic process under the driving environment in the embodiment of the present application, in order to ensure the accuracy of the overall judgment, it is necessary to determine whether the abnormal state exists in the multiple images in the continuous time period, and determine whether the driver is in the fatigue state in the current time period based on the abnormal state conditions of the multiple images.

Specifically, for the embodiment of the application, multiple images are acquired in a unit time, and when the driver in two images in the multiple images is in a fatigue state, the driver on the surface is in a fatigue state at this stage, that is, the dynamic state is in an abnormal state.

The image capturing device in the embodiment of the application is a video capturing device, the images are extracted multiple video frame images in a unit time, and the inputs of the abnormality recognition model are the multiple video frame images. And judging whether the unit time is in a fatigue state or not, namely identifying a plurality of video frame images acquired in the unit time, and judging the unit time to be in the fatigue state when the plurality of video frame images in the unit time meet an abnormality judgment rule.

In the embodiment of the present application, when the unit time is 2S and the abnormality determination rule is that any five video frame images among the plurality of video frame images acquired in the 2S time are determined to be in an abnormal state, it is described that the driver is in a fatigue state.

Referring to FIG. 2, a block diagram of some embodiments of a dynamic abnormal state identification apparatus is shown.

As shown in fig. 2, the dynamic abnormal state recognition apparatus 200 includes:

the abnormal feature identification code module 210 is configured to collect a plurality of images of the object in the field of view based on the unit frequency of the unit time, and input the images into the abnormal feature identification model respectively to determine whether the images have abnormal features; the abnormal recognition model is obtained through training through a data set containing abnormal labels;

the abnormality determination module 220 determines that the object in the visual field has a motion abnormality when the number of abnormal features is greater than a preset number.

Referring to fig. 3, an electronic device provided in an embodiment of the present application is shown, where the electronic device 300 includes: a memory 301 and a processor 302 coupled to the memory 301, the processor 302 being configured to perform the dynamic exception state identification method of any of the previous embodiments based on instructions stored in the memory 301.

The electronic device may also include one or more power supplies 303, one or more wired or wireless network interfaces 304, one or more input/output interfaces 305, one or more keyboards 306, and the like.

In a specific embodiment, an electronic device comprises a memory, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may comprise one or more modules, and each module may comprise a series of computer-executable instructions in a lancing device guidance device, and configured for execution by one or more processors, the one or more programs comprising computer-executable instructions for:

acquiring a plurality of images of a target in a visual field range based on unit time and unit frequency, and respectively inputting the images into an anomaly identification model to determine whether the images have anomaly characteristics;

and when the number of the abnormal features is larger than a preset number, determining that the target in the visual field range has motion abnormality.

The following describes each component of the processor in detail:

wherein in the present embodiment, the processor is a specific integrated circuit (application specific integrated circuit, ASIC), or one or more integrated circuits configured to implement embodiments of the present application, such as: one or more microprocessors (digital signal processor, DSPs), or one or more field programmable gate arrays (field programmable gate array, FPGAs).

Alternatively, the processor may perform various functions, such as performing the method shown in fig. 1 described above, by running or executing a software program stored in memory, and invoking data stored in memory.

In a particular implementation, the processor may include one or more microprocessors, as one embodiment.

The memory is configured to store a software program for executing the solution of the present application, and the processor is used to control the execution of the software program, and the specific implementation manner may refer to the above method embodiment, which is not described herein again.

Alternatively, the memory may be read-only memory (ROM) or other type of static storage device that can store static information and instructions, random access memory (random access memory, RAM) or other type of dynamic storage device that can store information and instructions, but may also be, without limitation, electrically erasable programmable read-only memory (electrically erasable programmable read-only memory, EEPROM), compact disc read-only memory (compact disc read-only memory) or other optical disk storage, optical disk storage (including compact disc, laser disc, optical disc, digital versatile disc, blu-ray disc, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and that can be accessed by a computer. The memory may be integrated with the processor or may exist separately and be coupled to the processing unit through an interface circuit of the processor, which is not specifically limited in the embodiments of the present application.

It should be noted that the structure of the processor shown in this embodiment is not limited to the apparatus, and an actual apparatus may include more or less components than those shown in the drawings, or may combine some components, or may be different in arrangement of components.

In addition, the technical effects of the processor may refer to the technical effects of the method described in the foregoing method embodiments, which are not described herein.

It should be appreciated that the processor in embodiments of the present application may be other general purpose processors, digital signal processors (digital signal processor, DSP), application specific integrated circuits (application specific integrated circuit, ASIC), off-the-shelf programmable gate arrays (field programmable gate array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

It should also be appreciated that the memory in embodiments of the present application may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile memory may be a read-only memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an electrically Erasable EPROM (EEPROM), or a flash memory. The volatile memory may be random access memory (random access memory, RAM) which acts as an external cache. By way of example but not limitation, many forms of random access memory (random access memory, RAM) are available, such as Static RAM (SRAM), dynamic Random Access Memory (DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced Synchronous Dynamic Random Access Memory (ESDRAM), synchronous Link DRAM (SLDRAM), and direct memory bus RAM (DR RAM).

The above embodiments may be implemented in whole or in part by software, hardware (e.g., circuitry), firmware, or any other combination. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present application are all or partially produced. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more sets of available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium may be a solid state disk.

In the present application, "at least one" means one or more, and "a plurality" means two or more. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b, or c may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or plural.

It should be understood that, in various embodiments of the present application, the sequence numbers of the foregoing processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation process of the embodiments of the present application.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (12)

1. A method for identifying a dynamic abnormal state, the method comprising:

acquiring a plurality of images of a target in a visual field range based on unit time and unit frequency, and respectively inputting the images into an anomaly identification model to determine whether the images have anomaly characteristics; the abnormal recognition model is obtained through training through a data set containing abnormal labels;

and when the number of the abnormal features is larger than a preset number, determining that the target in the visual field range has motion abnormality.

2. The method for identifying a dynamic abnormal state according to claim 1, wherein the abnormal identification model is a transducer model established based on an outlooker mechanism; determining whether the plurality of images have abnormal features through determining whether the plurality of images have abnormal features in an abnormal recognition model, and inputting the images into the abnormal recognition model, wherein the determining whether the plurality of images have abnormal features comprises the following steps:

extracting the characteristics of the image, carrying out fine coding on the local spatial characteristics to obtain fine coding information, and generating global information by the fine coding information, wherein the global information is the spatial characteristic information of the image generated in an abnormal state.

3. The dynamic abnormal state recognition method according to claim 1, further comprising performing enhancement processing on the image before inputting the image to the abnormal recognition model, the enhancement processing.

4. The method for recognizing dynamic abnormal state according to claim 3, wherein the enhancing process comprises dividing the image according to a dividing unit to obtain a plurality of independent image blocks with the same size, performing histogram uniform distribution process on the independent image blocks to obtain gray values of pixel points in the independent image blocks, and uniformly distributing the pixel points corresponding to the pixel points exceeding the gray threshold to all other pixels to obtain the enhanced image.

5. The dynamic abnormal state identification method according to claim 2, wherein the abnormal state identification model includes: the segmentation unit is used for segmenting the image to obtain a plurality of image blocks and converting each image block into a matrix with the same size; the coding unit is established based on an outlooker mechanism and is used for carrying out fine coding on the image, converting the common features in the image into spatial information of a matrix and labeling the weights of the features; and the transducer unit is used for generating global information for the fine coded information.

6. The method of claim 5, wherein the coding unit comprises four consecutive coding blocks, each coding block comprising a first branch for generating a refined representation and a second branch for generating aggregated global information.

7. The method for identifying a dynamic abnormal state according to claim 6, wherein the first branch comprises a normalization layer, an outlooker attention layer, and a regularization layer; the second branch includes a normalization layer, a multi-layer perceptron module, and a regularization layer.

8. The method of claim 5, wherein the fransformer unit comprises 14 consecutive fransformer blocks, each of the fransformer blocks comprising a mask self-attention module and a feed-forward layer, a normalization layer being disposed between the mask self-attention module and the feed-forward layer, and a normalization layer being disposed at an output of the feed-forward layer.

9. A dynamic abnormal state identification device, the device comprising:

the abnormal feature identification code module is used for acquiring a plurality of images of the target in the visual field range based on unit time unit frequency, and respectively inputting the images into the abnormal identification model to determine whether the images have abnormal features or not; the abnormal recognition model is obtained through training through a data set containing abnormal labels;

and the abnormality judging module is used for determining that the target in the visual field range has motion abnormality when the abnormal characteristic quantity is larger than a preset quantity.

10. An electronic device, comprising:

one or more processors;

a storage device having one or more programs stored thereon,

when executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-8.

11. A storage medium having a computer program stored thereon, wherein,

the computer program, when executed by a processor, implements the method of any of claims 1-8.

12. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any of claims 1-8.