CN117831135B - Human trace detection method based on image processing - Google Patents
Human trace detection method based on image processing Download PDFInfo
- Publication number
- CN117831135B CN117831135B CN202410240151.8A CN202410240151A CN117831135B CN 117831135 B CN117831135 B CN 117831135B CN 202410240151 A CN202410240151 A CN 202410240151A CN 117831135 B CN117831135 B CN 117831135B
- Authority
- CN
- China
- Prior art keywords
- image
- pixel point
- noise
- frame image
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 26
- 238000001514 detection method Methods 0.000 title claims abstract description 24
- 238000012549 training Methods 0.000 claims abstract description 9
- 230000003321 amplification Effects 0.000 claims description 31
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 31
- 230000000694 effects Effects 0.000 claims description 28
- 238000000034 method Methods 0.000 claims description 19
- 230000009471 action Effects 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 12
- 238000002372 labelling Methods 0.000 claims description 4
- 230000008859 change Effects 0.000 description 21
- 230000006399 behavior Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 5
- 230000029058 respiratory gaseous exchange Effects 0.000 description 5
- 206010041349 Somnolence Diseases 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000001815 facial effect Effects 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 210000001015 abdomen Anatomy 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 210000003128 head Anatomy 0.000 description 2
- 238000002955 isolation Methods 0.000 description 2
- 206010000117 Abnormal behaviour Diseases 0.000 description 1
- 208000032140 Sleepiness Diseases 0.000 description 1
- 206010044565 Tremor Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000739 chaotic effect Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 208000035475 disorder Diseases 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 210000001747 pupil Anatomy 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 230000037321 sleepiness Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 210000001364 upper extremity Anatomy 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Landscapes
- Image Processing (AREA)
Abstract
The invention relates to the technical field of image recognition, in particular to a human trace detection method based on image processing, which is used for acquiring a video image dataset of continuous multi-frame images of a target object; amplifying each frame of image in the video image data set to obtain an amplified image set; and training the pre-constructed network model by taking the amplified image set as input data and the label as output data, and completing the identification of the target object to be detected based on the trained network model. The invention can improve the definition of the video image and avoid the problem of distortion and blurring of the video image when detecting the micro-variation signal in the humanoid behavior.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to a human trace detection method based on image processing.
Background
The existing target detection algorithm for the behavior change of the person aims at the behavior action with larger amplitude, trains the constructed artificial neural network model by labeling the image or the video with the data set, and recognizes the behavior action of the person by using the artificial neural network model after training. However, for small change signals in the video image, such as color change of facial skin, abdomen undulation, head vibration, etc., the signals are often not accurately identified.
However, these minute change signals often also contain rich information, such as the frequency of color changes of the skin of the human face, which is related to heart rate and respiration rate; the fluctuation period of the abdomen is consistent with the breathing period of the human body; the period of vibration of the head is also related to heart rate and respiration rate. Thus, video amplification techniques have evolved. The video amplification technology is a technology for changing the small change amplitude in an image by utilizing image sequence space-time information processing, and can enhance the amplitude of a small change signal in the nature and realize the visualization of small changes which are difficult to perceive by naked eyes, so that important information and rules contained in the small changes in the nature are revealed.
The video amplification technology comprises Euler video amplification technology (Eulerian Video Magnification, EVM), wherein the video amplification technology adopts an idea method of Euler visual angles, analyzes the space-time variation characteristics of video content from a global angle, and senses tiny changes in an image sequence.
Therefore, for the identification of the micro-variation signal, the Euler video action amplification processing can be carried out on the video image in the data set, and then the model training of the artificial neural network is carried out, so that the detection of the micro-variation signal is realized.
However, in the video amplifying process, the Euler video amplifying algorithm amplifies not only the target in the image but also the noise in the image, so that the problem of distortion and blurring of the amplified video image is caused.
Therefore, it is particularly important how to avoid the distortion blur problem of the amplified video image when detecting a minute change signal in the human track behavior.
Disclosure of Invention
The invention aims to provide a human trace detection method based on image processing, which is used for solving the problem that an amplified video image is easy to generate distortion and blur when detecting a tiny change signal in human trace behaviors.
In order to solve the technical problems, the invention provides a human trace detection method based on image processing, which comprises the following steps:
acquiring a video image dataset of continuous multi-frame images of a target object;
Amplifying each frame of image in the video image data set to obtain an amplified image set;
training a pre-constructed network model by taking the amplified image set as input data and the label as output data, and completing identification of a target object to be detected based on the trained network model;
The acquisition process of the amplified image in the amplified image set comprises the following steps:
amplifying the brightness value of each pixel point in the current frame image by adopting an Euler video motion amplification algorithm to obtain an amplified brightness value; taking the difference value between the amplified brightness value and the brightness value as an amplifying effect;
Acquiring noise probability of each pixel point in a current frame image, wherein the noise probability is the average value of at least two noise indexes, and the noise indexes comprise a first noise index, a second noise index and a third noise index;
The noise probability of each pixel point is adopted to adjust the amplifying effect, the adjusted amplifying effect is overlapped with the brightness value to obtain an adjusted brightness value, and finally an amplified image is obtained;
The first noise index is determined by the gray difference value of each pixel point in the current frame image and the pixel points in the neighborhood of the pixel point and the entropy value of the gray difference value;
The second noise index is determined by the gray value of each pixel point in the difference image of the current frame image and the previous frame image and the gray value average value of all the pixel points in the corresponding neighborhood;
The third noise index is obtained through the following steps:
acquiring gray level sequences of pixel points at the same positions in a current frame image and continuous multi-frame images adjacent to the current frame image in front of and behind the current frame image;
Calculating the absolute value of the difference between the current gray level of the pixel point of the current frame image and the gray level with the largest occurrence number in the gray level sequence;
Counting the frequency of the current gray level in the gray level sequence; and obtaining a third noise index based on the frequency and the absolute value of the difference.
Optionally, the first noise indicator is:
Wherein, Is the first noise index,/>Representing the gray level difference value of the jth pixel point and the kth neighborhood pixel point of the pixel point in the ith frame image,/>For the frequency of occurrence of class t gray differences,/>The inverse number of entropy representing the gray difference value in the j-th pixel neighborhood, T is the total number of classes of different gray difference values, exp () is an exponential function based on a natural constant e.
Optionally, the second noise indicator is:
Wherein, Is the second noise index,/>Gray value of j pixel point in difference image of i-1 frame image and i frame image,/>The j-th pixel in the difference image is represented by the/>Gray value of each neighborhood pixel point,/>The total number of the neighborhoods of the jth pixel point in the difference image is equal to or greater than 1.
Optionally, when the noise probability is the average of the first noise indicator and the second noise indicator, the adjusted brightness value is:
Wherein, Representing the noise probability of the j-th pixel point in the i-th frame image,/>As an indicator of the first level of noise,Is the second noise index,/>Representing the brightness value of the j-th pixel point in the i-th frame image,/>Indicating an amplified brightness value obtained after the j pixel point in the i-th frame image is amplified by the European video motion amplifying algorithm,Indicating the magnification effect,/>And the adjusted brightness value of the j pixel point in the i-th frame image is represented.
Optionally, the third noise indicatorThe method comprises the following steps:
Wherein M represents the frequency of the same gray level in the gray level sequence as the gray level of the j-th pixel in the i-th frame image, Representing the gray level with the highest number of the same gray levels in the gray level sequence corresponding to the jth pixel point in the ith frame image,/>For the gray level of the jth pixel point in the ith frame image, exp () is an exponential function based on a natural constant e.
Optionally, when the noise probability is the average of the first noise indicator, the second noise indicator, and the third noise indicator, the adjusted brightness value is:
Wherein, Representing the noise probability of the j-th pixel point in the i-th frame image,/>Is the first noise index,/>Is the second noise index,/>Is the third noise index,/>Representing the brightness value of the j-th pixel point in the i-th frame image,/>Representing amplified brightness value obtained after amplification treatment of jth pixel point in ith frame image by European video action amplification algorithm,/>Indicating the magnification effect,/>And the adjusted brightness value of the j pixel point in the i-th frame image is represented.
Optionally, the network model employs yolov network model.
Optionally, the label is obtained by manually labeling each magnified image in the magnified image set.
The beneficial effects of the invention are as follows:
According to the scheme, the noise possibility is detected for each pixel point in each frame of image, at least two aspects of a single frame of image, two adjacent frames of images and continuous multi-frame images are comprehensively analyzed, the possibility that each pixel point is noise is obtained, in the process of adopting an Euler video motion amplification algorithm, the pixel points with high noise possibility are not amplified or the amplification degree is as small as possible, so that the pixel points with low noise possibility are amplified, the influence of noise on the amplified video image is reduced, and the image quality of the amplified video image is improved.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions and advantages of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are only some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 schematically illustrates a flow chart of steps of a method for human trace detection based on image processing of the present invention;
Fig. 2 schematically shows a flowchart of the steps for acquiring a third noise indicator in an image processing-based human track detection method according to the present invention.
Detailed Description
In order to further describe the technical means and effects adopted by the present invention to achieve the preset purpose, the following detailed description is given below of the specific implementation, structure, features and effects of the technical solution according to the present invention with reference to the accompanying drawings and preferred embodiments. In the following description, a different one or another embodiment is not necessarily the same embodiment. Furthermore, the particular features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.
When capturing micro motion change of a target object, an Euler video motion amplification algorithm is adopted to amplify noise in an image during motion amplification processing, so that the problem of distortion and blurring of the amplified video image is caused.
It should be noted that the target object may be a person; for example, it may detect abnormal behavior such as fatigue, distraction, drowsiness or other physical states for a video conference or for the status of a person during work; particularly, for some personnel in special working situations, such as a driver driving a coach, the fatigue state and abnormal physical condition of the driver can be detected and reminded to ensure safe driving; and for example, operators in factories and mines can detect the fatigue state, the breathing state and other small change actions of the operators when the dangerous machines are controlled to operate, so that the operators can find abnormality in time and carry out alarm reminding so as to improve the safety of personnel. Of course, the monitoring of the health state of the human body, such as the breathing state of the infant, tremor information of the human body and the like, can also be aimed at.
Of course, the target object may also be a device, that is, a device that detects a tiny vibration of the device during operation to determine whether the device has a fault, and the target object may be a desktop electric fan, for example, so that the collected target video image is the desktop electric fan during operation.
Fig. 1 schematically shows a flow chart of steps of a human trace detection method based on image processing according to the invention. Fig. 2 schematically shows a flowchart of the steps for acquiring a third noise indicator in an image processing-based human track detection method according to the present invention.
In this embodiment, an operator is taken as an example, and a human trace detection method based on image processing in this embodiment is specifically described.
As shown in fig. 1, the human trace detection method based on image processing of the present invention includes the following steps:
At step S1, a video image dataset of successive multi-frame images of a target object is acquired.
The method comprises the steps of shooting videos of sitting or standing states of operators in a period of time through cameras arranged in factories, offices or office areas, and identifying different body parts of a human body for each frame of images in the obtained videos to obtain local video image data.
Of course, as other embodiments, the video acquired in the present embodiment may be information of a local position of a human body, such as a face, a chest, or an upper limb, and since the minute change actions of different body parts respectively represent different body state information, the video of the local position of the body part may be acquired according to actual needs, and all frame images in the video of the local position may be used as a video image dataset.
In this embodiment, the detection of the working state of the operator in the factory, such as distraction and fatigue, is performed, so that, taking a partial image of a face as an example, image data of the face in a certain period of time is obtained, and gray scale processing is performed on each image data to obtain a video image dataset.
At step S2, each frame image in the video image data set is subjected to an enlargement process to obtain an enlarged image set.
In one embodiment, the acquisition process of the magnified image in the magnified image set is:
amplifying the brightness value of each pixel point in the current frame image by adopting an Euler video motion amplification algorithm to obtain an amplified brightness value; taking the difference value between the amplified brightness value and the brightness value as an amplifying effect;
Acquiring noise probability of each pixel point in a current frame image, wherein the noise probability is the average value of at least two noise indexes, and the noise indexes comprise a first noise index, a second noise index and a third noise index;
And adjusting the amplifying effect by adopting the noise probability of each pixel point, and superposing the adjusted amplifying effect and the brightness value to obtain an adjusted brightness value, thereby finally obtaining an amplified image.
The Euler video motion amplification algorithm utilizes a space-time processing method to enhance specific color changes, and can amplify specific color changes in video and amplify the effect of interesting motion changes. The core step of the Euler video motion amplification algorithm is that Laplacian pyramid decomposition is carried out on each frame of the video, filtering treatment is carried out, and each layer of filtered signal is multiplied by a specific amplification factorAnd adding the new pyramid with the original signal before the frequency domain filtering to obtain a final amplified video. Because the Euler video motion amplification algorithm is the prior art, the specific amplification process is not repeated here.
Since the Euler video motion amplification algorithm makes itself insignificant motion variations apparent by enhancing the brightness variation of each pixel. However, the algorithm also amplifies noise which cannot be filtered by the filtering process, so that the probability of noise is detected for each pixel point in each frame of image, corresponding noise probability is obtained, and the amplifying effect in the Euler video action amplifying algorithm is adjusted so as to reduce the influence of noise amplification on video.
It should be noted that, the noise probability in this embodiment may be at least two combinations of the first noise index, the second noise index, and the third noise index, that is, any two or three of the three noise indexes may evaluate the noise of the pixel points of each frame image.
The first noise index is determined by the gray level difference value between any pixel point in the current frame image and the pixel points in the neighborhood of the pixel point and the entropy value of the gray level difference value. This is because the noise distribution is random, and in a single frame image, if the difference between the gray value of one pixel and the gray value of a pixel in the neighborhood is large, it may be noise, or may be an edge of the image. When the point is noise, the range of the gray difference value of the pixel point in the neighborhood is similar to that of the gray difference value of the pixel point in the neighborhood, the chaotic degree is small, and the entropy of the neighborhood is small; when the point is an edge, the gray level difference value range between the point and the pixel point in the neighborhood is larger, the disorder is more caused, and the entropy of the neighborhood is larger. Therefore, the pixel point and the neighborhood gray level difference value and the entropy of the gray level difference value are utilized to measure the possibility that each pixel point in the single frame image is noise.
Specifically, the first noise indicator is:
Wherein, Representing the gray level difference value of the jth pixel point and the kth neighborhood pixel point of the pixel point in the ith frame image,/>For the frequency of occurrence of class t gray differences,/>The inverse number of entropy representing the gray difference value in the j-th pixel neighborhood, T is the total number of classes of different gray difference values, exp () is an exponential function based on a natural constant e.
Wherein,Is the normalized value of the gray difference value mean value of the current pixel point and the surrounding neighborhood, wherein 255 on the denominator is used for normalizing the gray difference value mean value, when/>The larger the difference between the gray value of the pixel point and the rest of the pixels in the neighborhood is, the greater the probability that the position is noise or edge is. /(I)An inverse number of entropy representing gray differences between the pixel and pixels in the surrounding neighborhood, where/(m) is the smaller the entropy isThe larger the difference value of gray scale is, the closer the difference value of gray scale is, the smaller the degree of confusion is, the less the pixel point is the edge, the more the pixel point is the noise, and the/>The larger.
The neighborhood of the pixel point can be eight neighborhood or four neighborhood. In this embodiment, eight neighborhoods are used for analysis of the relevant data, i.e., k=8.
Taking eight neighborhoods as an example, 8 gray scale differences can be obtained in the embodimentFor the obtained 8 gray differencesCounting the frequency of occurrence (the frequency is a natural number) to obtain the total number T of categories of different gray level difference values and the corresponding occurrence frequency, and recording the occurrence frequency of the gray level difference value of the T th category as/>Calculating the entropy of the gray difference value between the pixel point and the pixel point in the 8 neighborhood, and further obtaining the noise index/>, of the current pixel point in the single-frame image. It should be noted that, the number of T is determined based on 8 gray-scale differences, and when the same gray-scale difference exists in the 8 gray-scale differences, the value of T is necessarily smaller than 8.
Further, since noise is usually isolated and has no change rule, and the change of human motion usually causes the gray level change of pixels in a large range, the second noise index of each pixel of the current frame image is measured by acquiring the difference image of two adjacent frames and performing neighborhood analysis on the difference image, i.e. from the change of two adjacent framesThe method specifically comprises the following steps:
Wherein, The gray value of the j pixel point in the difference image of the i-1 frame image and the i frame image is represented,The j-th pixel in the difference image is represented by the/>Gray value of each neighborhood pixel point,/>The total number of the neighborhoods of the jth pixel point in the difference image is equal to or greater than 1.
In the above-mentioned formula(s),Representing the average value of gray values of all pixel points in the neighborhood of the jth pixel point in the difference image; and/>Is to measure the gray value difference between the pixel point and the surrounding neighborhood pixel points, when/>Far greater than/>Time,/>Close to 2,/>The closer to 1; when/>Much smaller than/>In the time-course of which the first and second contact surfaces,Close to 0,/>The closer to 1. I.e. when the gray value of the pixel point is greater than the difference between the surrounding neighborhood pixel points,/>The closer to 1, the more isolated the gray scale variation of the pixel, the greater the likelihood that the pixel will be noisy. When the gray value of the pixel is smaller than the surrounding, namely/>And/>The closer,/>The closer to 1,/>The closer to 0, the smaller the difference between the gray level change of the pixel point and the gray level change of the neighboring pixel points, the more likely the gray level change of the pixel point is that the human motion change causes the gray level change of the pixel point in a large range, and the less the possibility of noise.
It should be noted that, the difference image in the present embodiment is a difference between a subsequent frame and a previous frame in the adjacent frame images, where the first frame image does not perform the above operation, and further does not perform the enlarging process.
The pixel areas may be eight-neighborhood or four-neighborhood. In this embodiment, eight neighborhoods are used for analysis of related data, i.e.The value is 8.
It should be noted that, since the sizes of the current frame image, the adjacent frame image and the difference image are the same, the number of pixels is the same, so that the number of pixels in different images and the positions of the corresponding pixels are the same, and the difference is only the gray value; that is, j in the jth pixel point in the current frame image and j in the jth pixel point in the difference image in the present embodiment only represents the position or the number sequence of the pixel points on different images, and does not include the gray value of the pixel point.
Further, since the action behavior of the person is considered to have continuity in time, that is, the gray value change of the corresponding pixel point at the same position in the continuous frame image has continuity (gradation), and the noise is random and does not have continuity, the gray value of the pixel point at the same position and the frequency of occurrence of the gray value thereof are statistically analyzed in the continuous multi-frame image, so as to distinguish the behavior of the person from the noise.
Specifically, the third noise indicator is obtained by:
Firstly, counting gray values of corresponding pixel points at the same position in a current frame image and continuous multi-frame images adjacent to the current frame image and the current frame image, determining gray levels of the gray values, and sequencing the gray levels according to a time sequence to obtain a gray level sequence;
Secondly, calculating the absolute value of the difference value of the gray level with the largest number of the same gray levels in the gray level sequence and the gray level of the pixel point of the current frame image based on the gray level sequence;
Then, obtaining the frequency of the gray level which is the same as the gray level of the pixel point of the current frame image in the gray level sequence; based on the frequency and the absolute value of the difference, a third noise index is obtained, specifically:
Wherein M represents the frequency of the same gray level in the gray level sequence as the gray level of the j-th pixel in the i-th frame image, Representing the gray level with the highest number of the same gray levels in the gray level sequence corresponding to the jth pixel point in the ith frame image,/>For the gray level of the jth pixel point in the ith frame image, exp () is an exponential function based on a natural constant e.
Wherein whenThe smaller the gray value of the pixel point at the current position is, the less the frequency of the gray value appearing in the continuous multi-frame image is, the stronger the isolation of the gray value is, and the greater the possibility of noise is, at the moment/>The larger, thereby/>The larger the size of the container,The difference value of the gray level representing the pixel point at the current position and the gray level with the largest number of the same gray levels in the gray level sequence is larger, which means that the higher the isolation of the gray value of the pixel point at the current position in continuous frame images is, the higher the possibility that the pixel point at the current position is noise is, namely/>, the greaterThe larger.
Illustratively, counting gray values of pixel points at the same position of an ith frame image and n frames of images adjacent to the ith frame image, sorting all gray values according to time sequence to obtain a gray value sequence, and calculating gray levels of the gray values, specifically, dividing the gray values of 0-255 into 26 gray levels, namely, level 1 (gray values 0-10), level 2 (gray values 11-20), …, level 25 (gray values 241-250), level 26 (gray values 251-255), wherein the interval between adjacent levels is set to be 1; wherein n is greater than or equal to 3, that is, the value of n may be 3, 4, 5, 6 or more, and in this embodiment, the value of n is 5.
In this embodiment, according to 26 gray levels divided in advance, gray levels of each gray value in a gray value sequence of 11 frames of images are determined, a corresponding gray level sequence is obtained, and the number of occurrences of each gray level is counted, so as to obtain the frequency of each gray level, where when the number of occurrences of the gray level is smaller, the pixel corresponding to the gray level is more likely to be noise.
In one embodiment, when the noise probability is the average value of the first noise index and the second noise index, based on the noise probability, the amplifying effect of the euler video motion amplifying algorithm on amplifying each frame image is adjusted, so as to obtain an adjusted brightness value of each pixel point, which specifically is:
The amplified brightness value of each pixel point of each frame image obtained after the amplification processing of the Euler video motion amplifying algorithm is differenced with the brightness value of the pixel point at the same position of the original image, so as to obtain an amplifying effect, the amplifying effect is adjusted according to the noise probability of the pixel point at the position, and the adjusted amplifying effect and the brightness value are overlapped to obtain an adjusted brightness value, specifically as follows:
Wherein, Representing the noise probability of the j-th pixel point in the i-th frame image,/>As an indicator of the first level of noise,Is the second noise index,/>Representing the brightness value of the j-th pixel point in the i-th frame image,/>Indicating an amplified brightness value obtained after the j pixel point in the i-th frame image is amplified by the European video motion amplifying algorithm,Indicating the magnification effect,/>And the adjusted brightness value of the j pixel point in the i-th frame image is represented.
It should be noted that, when the noise probability is the first noise index and the third noise index or the second noise index and the third noise index, the calculation manner of the adjustment brightness value of each pixel point of the current frame image is the same as the calculation process in the above embodiment, and only the selected noise index is different.
As a most preferred embodiment, when the noise probability is the average of the first noise indicator, the second noise indicator, and the third noise indicator, the adjusted brightness value of each pixel point is:
Wherein, Representing the noise probability of the j-th pixel point in the i-th frame image,/>Is the first noise index,/>Is the second noise index,/>Is the third noise index,/>Representing the brightness value of the j-th pixel point in the i-th frame image,/>Representing amplified brightness value obtained after amplification treatment of jth pixel point in ith frame image by European video action amplification algorithm,/>Indicating the magnification effect,/>And the adjusted brightness value of the j pixel point in the i-th frame image is represented.
Wherein when the noise probabilityThe larger, i.e. the greater the likelihood that the pixel is noisy,The smaller the amplification effect on noise can be reduced; when/>The smaller the pixel is, i.e. the less likely the pixel is noise, the greater the likelihood of motion by a person,/>The larger the noise probability is, the more the motion of a person can be effectively amplified, so that the influence of the noise on the amplified video is reduced through the adjustment of the noise probability on the amplifying effect.
Thus, an enlarged image subjected to enlargement processing is obtained, and an enlarged image set is obtained.
At step S3, training a pre-constructed network model by using the amplified image set as input data and the label as output data, and completing recognition of the target object to be detected based on the trained network model; the label is obtained by labeling each amplified image in the amplified image set.
The network model previously constructed in this embodiment adopts yolov network model. Since the yolov network model is constructed in the prior art, redundant description is not provided here.
In this embodiment, through constructing yolov network model, each image in the amplified image set processed by the adjusted euler video action amplifying algorithm is manually labeled, so as to obtain corresponding labels, that is, when the amplified image set is a human face, the labeled labels are classified into three types of labels of normal state, fatigue state and sleepiness state. It should be noted that the fatigue state can be determined by the facial skin color of a person, for example, the facial skin color is whiter than that of the normal state, and can be marked as the fatigue state; the drowsiness state may be determined based on pupil changes and blink conditions of the person.
The training data set in the embodiment is composed of an amplified image set and a label, and the training data set is input to a yolov network model for training, wherein the amplified image set is used as input data, and the label is used as output data; and detecting the state of a person in the video image of the target object to be detected by using the trained yolov network model, wherein when the detection is carried out, the video image of the target object to be detected needs to be amplified firstly, and then the amplified video image of the target object to be detected is input into the trained yolov network model, wherein the specific amplification process of the video image of the target object to be detected is the same as the processing process of the amplified image in the amplified image set in the embodiment.
According to the scheme, the noise possibility is detected for each pixel point in each frame of image, and at least two aspects of a single frame of image, two adjacent frames of images and continuous multi-frame images are comprehensively analyzed to obtain the possibility that the pixel point at the position is noise, after the amplifying treatment of the European video motion amplifying algorithm, the amplifying brightness value of each pixel point is adjusted, the amplifying effect of the pixel point with high noise possibility is reduced, the adjusted amplifying effect is close to 0, and the amplifying effect of the pixel point with low noise possibility is kept unchanged as much as possible, so that the influence of noise on the amplified video image is reduced, and the image quality of the amplified video image is improved.
While various embodiments of the present invention have been shown and described herein, it will be obvious to those skilled in the art that such embodiments are provided by way of example only. Many modifications, changes, and substitutions will now occur to those skilled in the art without departing from the spirit and scope of the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in practicing the invention.
Claims (8)
1. The human trace detection method based on image processing is characterized by comprising the following steps of:
acquiring a video image dataset of continuous multi-frame images of a target object;
Amplifying each frame of image in the video image data set to obtain an amplified image set;
training a pre-constructed network model by taking the amplified image set as input data and the label as output data, and completing identification of a target object to be detected based on the trained network model;
The acquisition process of the amplified image in the amplified image set comprises the following steps:
amplifying the brightness value of each pixel point in the current frame image by adopting an Euler video motion amplification algorithm to obtain an amplified brightness value; taking the difference value between the amplified brightness value and the brightness value as an amplifying effect;
Acquiring noise probability of each pixel point in a current frame image, wherein the noise probability is the average value of at least two noise indexes, and the noise indexes comprise a first noise index, a second noise index and a third noise index;
The noise probability of each pixel point is adopted to adjust the amplifying effect, the adjusted amplifying effect is overlapped with the brightness value to obtain an adjusted brightness value, and finally an amplified image is obtained;
The first noise index is determined by the gray difference value of each pixel point in the current frame image and the pixel points in the neighborhood of the pixel point and the entropy value of the gray difference value;
The second noise index is determined by the gray value of each pixel point in the difference image of the current frame image and the previous frame image and the gray value average value of all the pixel points in the corresponding neighborhood;
The third noise index is obtained through the following steps:
acquiring gray level sequences of pixel points at the same positions in a current frame image and continuous multi-frame images adjacent to the current frame image in front of and behind the current frame image;
Calculating the absolute value of the difference between the current gray level of the pixel point of the current frame image and the gray level with the largest occurrence number in the gray level sequence;
Counting the frequency of the current gray level in the gray level sequence; and obtaining a third noise index based on the frequency and the absolute value of the difference.
2. The image processing-based humanoid trace detection method of claim 1, wherein the first noise index is:
Wherein, Is the first noise index,/>Representing the gray level difference value of the jth pixel point and the kth neighborhood pixel point of the pixel point in the ith frame image,/>For the frequency of occurrence of class t gray differences,/>The reverse number of entropy of gray level difference values in the neighborhood of the jth pixel point is represented, T is the total number of classes of different gray level difference values, exp () is an exponential function based on a natural constant e, and K represents the total number of the neighborhood of the jth pixel point in the current frame image.
3. The image processing-based humanoid trace detection method of claim 2, wherein the second noise index is:
Wherein, Is the second noise index,/>Gray value of j pixel point in difference image of i-1 frame image and i frame image,/>The j-th pixel in the difference image is represented by the/>Gray value of each neighborhood pixel point,/>The total number of the neighborhoods of the jth pixel point in the difference image is equal to or greater than 1.
4. The image processing-based humane trace detection method according to claim 3, wherein when the noise probability is the average of the first noise indicator and the second noise indicator, the adjusted brightness value is:
Wherein, Representing the noise probability of the j-th pixel point in the i-th frame image,/>Is the first noise index,/>Is the second noise index,/>Representing the brightness value of the j-th pixel point in the i-th frame image,/>Representing amplified brightness value obtained after amplification treatment of jth pixel point in ith frame image by European video action amplification algorithm,/>Indicating the magnification effect,/>And the adjusted brightness value of the j pixel point in the i-th frame image is represented.
5. The image processing-based humanoid trace detection method of claim 4, wherein the third noise indicatorThe method comprises the following steps:
Wherein M represents the frequency of the same gray level in the gray level sequence as the gray level of the j-th pixel in the i-th frame image, Representing the gray level with the highest number of the same gray levels in the gray level sequence corresponding to the jth pixel point in the ith frame image,/>For the gray level of the jth pixel point in the ith frame image, exp () is an exponential function based on a natural constant e.
6. The image processing-based humane detection method according to claim 5, wherein when the noise probability is the average of the first noise indicator, the second noise indicator, and the third noise indicator, the adjusted brightness value is:
Wherein, Representing the noise probability of the j-th pixel point in the i-th frame image,/>As an indicator of the first level of noise,Is the second noise index,/>Is the third noise index,/>Represents the luminance value of the j-th pixel point in the i-th frame image,Representing amplified brightness value obtained after amplification treatment of jth pixel point in ith frame image by European video action amplification algorithm,/>Indicating the magnification effect,/>And the adjusted brightness value of the j pixel point in the i-th frame image is represented.
7. The image processing-based humanoid trace detection method of claim 1, wherein the network model employs yolov network model.
8. The image processing-based humanoid trace detection method according to claim 1, wherein the label is obtained by manually labeling each magnified image in a set of magnified images.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410240151.8A CN117831135B (en) | 2024-03-04 | 2024-03-04 | Human trace detection method based on image processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410240151.8A CN117831135B (en) | 2024-03-04 | 2024-03-04 | Human trace detection method based on image processing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117831135A CN117831135A (en) | 2024-04-05 |
| CN117831135B true CN117831135B (en) | 2024-05-10 |
Family
ID=90509848
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410240151.8A Active CN117831135B (en) | 2024-03-04 | 2024-03-04 | Human trace detection method based on image processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117831135B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106408563A (en) * | 2016-09-30 | 2017-02-15 | 杭州电子科技大学 | Snow noise detection method based on variation coefficient |
| CN112241670A (en) * | 2019-07-18 | 2021-01-19 | 杭州海康威视数字技术股份有限公司 | Image processing method and device |
| CN112672072A (en) * | 2020-12-18 | 2021-04-16 | 南京邮电大学 | Segmented steady video amplification method based on improved Euler amplification |
| CN113221639A (en) * | 2021-04-01 | 2021-08-06 | 山东大学 | Micro-expression recognition method for representative AU (AU) region extraction based on multitask learning |
| CN115908154A (en) * | 2022-09-20 | 2023-04-04 | 盐城众拓视觉创意有限公司 | Video late-stage particle noise removing method based on image processing |
| CN116309579A (en) * | 2023-05-19 | 2023-06-23 | 惠州市宝惠电子科技有限公司 | Transformer welding seam quality detection method using image processing |
| WO2023134791A2 (en) * | 2022-12-16 | 2023-07-20 | 苏州迈创信息技术有限公司 | Environmental security engineering monitoring data management method and system |
| CN116723413A (en) * | 2023-06-26 | 2023-09-08 | 豪威科技(武汉)有限公司 | RAW domain image denoising method and shooting device |
| CN117474029A (en) * | 2023-12-28 | 2024-01-30 | 中科微点技术有限公司 | AI polarization enhancement chart code wave frequency acquisition imaging identification method based on block chain |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4816970B2 (en) * | 2005-11-30 | 2011-11-16 | 日本電気株式会社 | Image processing apparatus, display apparatus, image processing method, and program |
| CN113776165B (en) * | 2021-09-10 | 2023-03-21 | 西安建筑科技大学 | YOLOv5l algorithm-based multi-region artificial fog pipe network intelligent control method and system |
-
2024
- 2024-03-04 CN CN202410240151.8A patent/CN117831135B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106408563A (en) * | 2016-09-30 | 2017-02-15 | 杭州电子科技大学 | Snow noise detection method based on variation coefficient |
| CN112241670A (en) * | 2019-07-18 | 2021-01-19 | 杭州海康威视数字技术股份有限公司 | Image processing method and device |
| CN112672072A (en) * | 2020-12-18 | 2021-04-16 | 南京邮电大学 | Segmented steady video amplification method based on improved Euler amplification |
| CN113221639A (en) * | 2021-04-01 | 2021-08-06 | 山东大学 | Micro-expression recognition method for representative AU (AU) region extraction based on multitask learning |
| CN115908154A (en) * | 2022-09-20 | 2023-04-04 | 盐城众拓视觉创意有限公司 | Video late-stage particle noise removing method based on image processing |
| WO2023134791A2 (en) * | 2022-12-16 | 2023-07-20 | 苏州迈创信息技术有限公司 | Environmental security engineering monitoring data management method and system |
| CN116309579A (en) * | 2023-05-19 | 2023-06-23 | 惠州市宝惠电子科技有限公司 | Transformer welding seam quality detection method using image processing |
| CN116723413A (en) * | 2023-06-26 | 2023-09-08 | 豪威科技(武汉)有限公司 | RAW domain image denoising method and shooting device |
| CN117474029A (en) * | 2023-12-28 | 2024-01-30 | 中科微点技术有限公司 | AI polarization enhancement chart code wave frequency acquisition imaging identification method based on block chain |
Non-Patent Citations (1)
| Title |
|---|
| 一种图像序列的细微运动放大算法;訾玲玲;丛鑫;;小型微型计算机系统;20160515(第05期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117831135A (en) | 2024-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11514270B2 (en) | Speckle contrast analysis using machine learning for visualizing flow | |
| US8055018B2 (en) | Object image detection method | |
| CN106056079B (en) | A kind of occlusion detection method of image capture device and human face five-sense-organ | |
| Le Ngo et al. | Micro-expression motion magnification: Global lagrangian vs. local eulerian approaches | |
| CN113420624A (en) | Non-contact fatigue detection method and system | |
| EP3405110A1 (en) | Method and apparatus for estimating breathing rate | |
| CN107798279A (en) | Face living body detection method and device | |
| CN104077568A (en) | High-accuracy driver behavior recognition and monitoring method and system | |
| CN114898261A (en) | Sleep quality assessment method and system based on fusion of video and physiological data | |
| CN110737201A (en) | monitoring method, device, storage medium and air conditioner | |
| KR20190050551A (en) | Apparatus and method for recognizing body motion based on depth map information | |
| CN117831135B (en) | Human trace detection method based on image processing | |
| Kielty et al. | Neuromorphic sensing for yawn detection in driver drowsiness | |
| CN117475353A (en) | Video-based abnormal smoke identification method and system | |
| US11275947B2 (en) | Image processing system, image processing method, and image processing program | |
| Peres et al. | Automatic segmentation of digital images applied in cardiac medical images | |
| Kshirsagar et al. | Fatigue detection using artificial intelligence | |
| Farzad et al. | Recognition and classification of human behavior in Intelligent surveillance systems using Hidden Markov Model | |
| CN115995059A (en) | Construction site safety helmet wearing detection method and system | |
| CN106651689A (en) | Intelligent examination system | |
| Brunel et al. | Time to detect the difference between two images presented side by side | |
| Bhavani | Automated Attendance System and Voice Assistance using Face Recognition | |
| Akinci et al. | A video-based eye pupil detection system for diagnosing bipolar disorder | |
| WO2019098835A1 (en) | A computer-controlled method of and apparatus and computer program product for supporting visual clearance of physical content | |
| CN111724347A (en) | Chest radiography image anomaly detection method and system based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |