CN114639171A - Panoramic safety monitoring method for parking lot - Google Patents

Panoramic safety monitoring method for parking lot Download PDF

Info

Publication number
CN114639171A
CN114639171A CN202210535421.9A CN202210535421A CN114639171A CN 114639171 A CN114639171 A CN 114639171A CN 202210535421 A CN202210535421 A CN 202210535421A CN 114639171 A CN114639171 A CN 114639171A
Authority
CN
China
Prior art keywords
information
panoramic
data set
parking lot
motion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210535421.9A
Other languages
Chinese (zh)
Other versions
CN114639171B (en
Inventor
刘寒松
王永
王国强
刘瑞
翟贵乾
李贤超
焦安健
谭连胜
董玉超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sonli Holdings Group Co Ltd
Original Assignee
Sonli Holdings Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sonli Holdings Group Co Ltd filed Critical Sonli Holdings Group Co Ltd
Priority to CN202210535421.9A priority Critical patent/CN114639171B/en
Publication of CN114639171A publication Critical patent/CN114639171A/en
Application granted granted Critical
Publication of CN114639171B publication Critical patent/CN114639171B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/20Analysis of motion
    • G06T7/269Analysis of motion using gradient-based methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

The invention belongs to the technical field of parking lot safety monitoring, and relates to a parking lot panoramic safety monitoring method.

Description

Panoramic safety monitoring method for parking lot
Technical Field
The invention belongs to the technical field of parking lot safety monitoring, and relates to a parking lot panoramic safety monitoring method based on multi-mode significance region recommendation.
Background
With the development of society and economic progress, family cars gradually become indispensable to families, and some families even have a plurality of cars, but the parking problem gradually becomes the problem that needs to be considered for urban management, and if the parking problem can not be fundamentally solved, great inconvenience will be brought to urban management, so the construction and safety problems of parking lots are more and more emphasized.
The traditional monitoring camera for the parking lot can only shoot at a fixed visual angle, so that the defects that if safety accidents occur at other positions which cannot be shot in a monitoring mode, or illegal activities are carried out through monitoring dead corners, real-time monitoring and real-time safety alarming cannot be achieved, and because only events in a fixed range can be seen in the traditional monitoring mode, the comprehensive monitoring of the parking lot is very unfavorable; even if the monitoring probe capable of moving in a range of 360 degrees is adopted, the content to be watched can be seen only by continuously moving, a large amount of content which may need attention is easily missed in the moving process, and the camera still belongs to a single visual angle at a certain moment and cannot achieve comprehensive monitoring. Meanwhile, the monitoring of the parking lot is limited by the internal memory of computer hardware, so that the parking lot can not be stored permanently for saving cost, the parking lot needs to be cleaned at intervals, some parking lots can only store monitoring contents for 4 days recently, a large amount of important information is lost, and corresponding monitoring contents cannot be called out when the monitoring contents are needed in the later period, so that the monitoring effect cannot be achieved.
In summary, the existing monitoring method can only monitor a fixed range, cannot automatically find a suitable position according to the content in the parking lot, cannot perform full-range real-time monitoring, and meanwhile, the traditional monitoring is time-consuming, labor-consuming and expensive, along with the rapid development of the deep learning technology, the deep learning-based method is rapidly developed in various industries, and a deep learning-based recommendation algorithm is continuously proposed, so that great performance improvement is achieved.
Disclosure of Invention
The invention aims to overcome the defects in the prior art, and provides a parking lot panoramic safety monitoring method based on multi-mode salient region recommendation.
In order to achieve the purpose, the specific process for realizing the panoramic safety monitoring of the parking lot comprises the following steps:
(1) collecting a panoramic camera video segment, and constructing a parking lot panoramic safety monitoring video data set according to the collected panoramic camera video segment;
(2) detecting according to the panoramic monitoring video data set of the parking lot based on the motion information, the comparison information and the significance of the sound information respectively;
(3) finding out corresponding important objects based on the significance detection results of the motion information, the comparison information and the sound information, and ranking the importance of the found important objects;
(4) carrying out panoramic monitoring video interframe consistency smoothing on the ranked objects, and connecting the same objects among the frames to form a smooth window motion track;
(5) converting the points into planes according to the motion tracks of different objects formed in the step (4), and providing panoramic safety content of a single window (single object) or multiple windows (multiple objects) in a screen mode;
(6) and (5) dynamically adjusting and manually intervening and monitoring contents aiming at the window contents output in the step (5), if the window contents are not the contents which are wanted to be focused, only moving a mouse to an object which needs to be focused, clicking the object, forming a projection window aiming at the center as a projection center by taking a mouse mark point as the center, and providing subsequent monitoring based on the window.
Specifically, in the step (1), the parking lot panoramic safety monitoring video data set adopts a saliency detection data set as a training set, in the process of constructing the training set, the whole data set is trained firstly, and then a data set related to a vehicle is selected as fine tuning data, so that a model focuses on saliency areas such as vehicles, pedestrians and obstacles, and the saliency detection data set comprises a video saliency detection data set DAVSOD which is most consistent with human eye visual features and is the largest, an audio contrast saliency detection data set MSRA10K which is most consistent with human eye visual features and is the largest, and an audio saliency detection data set AVEDataset which is most consistent with human eye visual features and is the largest.
Specifically, the motion information significance (VSOD) in the step (2) obtains the speed of the motion of the object relative to the background region based on the displacement relationship between the previous and subsequent frames of the object in the monitoring video to judge the state of the object, and calculates two adjacent frames (I) through FlowNet21,I2) Obtaining motion information between videos, optical flow information
Figure 516339DEST_PATH_IMAGE001
To calculate the velocity difference between two pixels:
Figure 433479DEST_PATH_IMAGE002
Figure 213216DEST_PATH_IMAGE003
Figure 608425DEST_PATH_IMAGE004
Figure 485115DEST_PATH_IMAGE005
is the distance the pixel has moved along the x-axis and y-axis respectively,
Figure 307577DEST_PATH_IMAGE006
is pixel from I1To I2The time, x, y are the coordinate positions of the initial pixel points, and t is the movement
Figure 574610DEST_PATH_IMAGE004
Figure 773510DEST_PATH_IMAGE005
The time required; after obtaining the optical flow information, the significance detection network is perceived by designing the optical flow (
Figure 442389DEST_PATH_IMAGE007
) Adopting DAVSOD as a motion information perception training data set, and converting the optical flow characteristics into motion significance characteristics:
Figure 747337DEST_PATH_IMAGE008
flow2Color stands for converting the optical Flow graph into an RGB graph, different colors represent different moving directions, the shade represents the speed of movement,
Figure 501667DEST_PATH_IMAGE007
taking as an input the RGB-map,
Figure 238679DEST_PATH_IMAGE009
the motion significance detection result is output.
Specifically, the significance of the contrast information in the step (2) is based on the characteristics of the position, color, texture and the like of the object, the object is highlighted through the contrast with the background information and the object, the MSRA10K is adopted as a contrast information perception training data set, and the image-based significance (ISOD) detection algorithm (ContrastNet) outputs the scale contrast information of the perception object through different layer sides of the network layer,
Figure 762064DEST_PATH_IMAGE010
Figure 254225DEST_PATH_IMAGE011
respectively representing color information, texture information, edge information,
Figure 495851DEST_PATH_IMAGE012
represents the multi-scale information of the multi-scale information,
Figure 770974DEST_PATH_IMAGE013
taking as input the RGB map of MSRA10K,
Figure 414445DEST_PATH_IMAGE014
the graph is an output comparative significance detection result graph.
Specifically, the sound information significance detection in the step (2) is to use sound as an auxiliary of visual information, use AVEDataset as a sound information perception training data set, convert audio information into a spectrogram through fourier transform, input the spectrogram into a network, and determine whether the current input contains sound information, so as to determine the importance of the current frame, specifically:
Figure 828240DEST_PATH_IMAGE015
wherein,
Figure 557162DEST_PATH_IMAGE016
in order to be a fourier transform,
Figure 635976DEST_PATH_IMAGE017
taking the fourier transform map as an input,
Figure 133954DEST_PATH_IMAGE018
the sound significance detection result graph is output.
Specifically, the ranking of the importance of the object in the step (3) is realized by the following formula:
Figure 905600DEST_PATH_IMAGE019
Figure 856239DEST_PATH_IMAGE020
wherein
Figure 535482DEST_PATH_IMAGE021
Representing the ranking of the kth object, N being the number of pixels, Norm being the maximum and minimum normalization function of the matrix,
Figure 887966DEST_PATH_IMAGE022
as object confidence, object confidence
Figure 830514DEST_PATH_IMAGE022
And the object number k is obtained by object detection YoloV 5.
Specifically, in the step (4), the inter-frame consistency information is that the window coordinate changes within 5 pixels, the coordinate change beyond the range is regarded as the content beyond the motion range, and the formed motion track is as follows:
Figure 268449DEST_PATH_IMAGE023
Figure 689066DEST_PATH_IMAGE024
' an initial trajectory of a representative point,
Figure 473220DEST_PATH_IMAGE024
represents the smoothed track of points, smooths represents the mean smoothing of the track,
Figure 586669DEST_PATH_IMAGE025
is a point
Figure 511900DEST_PATH_IMAGE026
The surrounding c =5 field of view,
Figure 736208DEST_PATH_IMAGE027
is a point
Figure 797705DEST_PATH_IMAGE026
Pixel points in the surrounding c =5 field.
Specifically, in the step (5), the dots are converted into the plane by the following formula
Figure 144373DEST_PATH_IMAGE028
Wherein windows represents an output window,
Figure 556899DEST_PATH_IMAGE029
representing point-based trajectories
Figure 584898DEST_PATH_IMAGE024
The perspective projection.
Compared with the prior art, the invention has the beneficial effects that: the method is characterized in that a multi-mode salient region recommendation algorithm is adopted, an concerned region is enhanced, a non-concerned region reduces resolution, transmission is accelerated, storage space is saved, and cost is saved; the video significance adopted is mainly based on the human attention behavior to formulate a rule, and the priority ranking of monitoring content processing is formulated by combining the event importance, so that the purpose of real-time panoramic safety monitoring is achieved.
Drawings
Fig. 1 is a structural framework schematic diagram of a parking lot panoramic safety monitoring method based on multi-modal saliency area recommendation provided by the invention.
Detailed Description
The invention will now be further described by way of examples in connection with the accompanying drawings without in any way limiting the scope of the invention.
Example (b):
according to the embodiment, through collected video data, a multi-mode salient region recommendation algorithm based on motion information, contrast information and sound information is adopted to detect a human eye attention region based on a human eye attention mechanism, a monitoring content processing priority ranking is formulated by combining event importance weights, meanwhile, resource occupation is saved in a mode of reducing resolution ratio of a non-attention region, and therefore the purpose of real-time panoramic safety monitoring is achieved, and panoramic parking lot panoramic safety monitoring is formulated based on safety detection weights and monitoring interframe continuity, and the method specifically comprises the following steps:
(1) construction of a parking lot panoramic surveillance video dataset
Firstly, a panoramic camera video segment is collected to construct a parking lot panoramic safety monitoring video data set recommended based on a multi-modal saliency region, and since most parking lots do not have the equipment popularized on a large scale at present, and the device has less attention to the multi-modal information of the parking lot, which results in the lack of the current data set, the embodiment adopts the significance detection data set (the largest video significance detection data set davsodod which best conforms to the human eye visual characteristics, the largest sound contrast significance detection data set MSRA10K which best conforms to the human eye visual characteristics and the largest audio significance detection data set avedaast which best conforms to the human eye visual characteristics) as the training set, in the process of constructing a training set, pre-training is carried out on a complete data set, and then a data set related to a vehicle is selected as fine tuning data, so that the model focuses on salient regions such as vehicles, pedestrians and obstacles;
(2) motion information based saliency detection
Human eyes are very interested in moving objects because moving objects are often accompanied by the occurrence of events, and based on this mechanism, moving objects in parking lots are also very important, such as a traveling vehicle, a driver driving the vehicle, and an animal running around, if they can be captured by a camera in advance,the method can be used for processing the objects, such as clearing obstacles, driving away animals, and the like, so as to greatly avoid traffic accidents, the motion information significance (VSOD) obtains the speed of the object moving relative to a background area based on the displacement relation between the front frame and the rear frame of the object in a monitoring video to judge the state of the object, and in order to obtain the motion information between videos, the embodiment calculates two adjacent frames (I) through FlowNet21,I2) Optical flow information between, optical flow information
Figure 766481DEST_PATH_IMAGE001
To calculate the velocity difference between two pixels:
Figure 956154DEST_PATH_IMAGE030
Figure 855977DEST_PATH_IMAGE003
Figure 235137DEST_PATH_IMAGE004
Figure 536805DEST_PATH_IMAGE005
is the distance the pixel has moved along the x-axis and y-axis respectively,
Figure 897379DEST_PATH_IMAGE006
is pixel from I1To I2The time, x, y are the coordinate positions of the initial pixel points, and t is the movement
Figure 284498DEST_PATH_IMAGE004
Figure 654299DEST_PATH_IMAGE005
The time required; after optical flow information is obtained, converting the optical flow characteristics into motion saliency characteristics, and designing an optical flow perception saliency detection network (
Figure 544895DEST_PATH_IMAGE007
) Adopting DAVSOD as a motion information perception training data set, converting an optical flow graph into an RGB graph as network input, and outputting a significance detection picture as follows:
Figure 669846DEST_PATH_IMAGE008
flow2Color stands for converting the optical Flow graph into an RGB graph, different colors represent different moving directions, the shade represents the speed of movement,
Figure 278682DEST_PATH_IMAGE007
taking an RGB map as input;
(3) saliency detection based on contrast information
In addition to moving objects, in normal situations, cameras are in a still shooting state, but there are still many situations to be noticed, such as that a marker is present in the middle of a road and a vehicle object is lost, in these situations, the object is often still, and good effect cannot be obtained by using moving object detection, so that, in order to obtain contrast information of a video, a saliency (ISOD) detection algorithm (contrast net) based on picture RGB (from MSRA 10K) is used to obtain the contrast information of the video by highlighting the object through comparison with background information and between objects, MSRA10K is used as the contrast information perception training data to output scale contrast information of a perceived object through different layers of a network layer, while a light layer side outputs more attention to color, texture and edge contrast information, the network deep layer side characteristic concerns the high-layer semantic information comparison, and the output comparison significance detection result graph is as follows:
Figure 452174DEST_PATH_IMAGE010
Figure 462856DEST_PATH_IMAGE011
respectively, color information, texture information and edge information,
Figure 430812DEST_PATH_IMAGE012
represents the multi-scale information of the multi-scale information,
Figure 526944DEST_PATH_IMAGE013
taking an RGB map as input;
(4) saliency detection based on sound information
The occurrence of voice is often accompanied by the occurrence of events, and has a very strong interest in the sounding object human, because in most cases, the occurrence of traffic accidents is accompanied by the occurrence of voice, such as car accidents, vehicle alarm alarms and vehicle horn sounds, which has a very strong research value for the detection of vehicles in parking lots; because of the binaural effect, the person has the ability of recognizing the position by listening to sound, and mainly benefits from that the ear of the person can recognize the position according to the distance from the sound object to the ear, so that the embodiment uses the sound as the assistance of visual information, adopts AVEDataset as a sound information perception training data set, converts the audio information into a spectrogram through Fourier transform, inputs the spectrogram into the network, and judges whether the current input contains sound information, thereby determining the importance of the current frame, and the output sound significance detection result graph is as follows:
Figure 307990DEST_PATH_IMAGE015
wherein,
Figure 173178DEST_PATH_IMAGE016
in order to be a fourier transform,
Figure 312035DEST_PATH_IMAGE017
taking the Fourier transform map as an input;
(4) content body level importance ranking based on panoramic surveillance video
The corresponding important objects can be found through information such as motion, contrast and sound, but a plurality of important objects may exist in some casesIn this case, the importance of a plurality of objects needs to be ranked, because when different information is used as the leading information, the determined important objects are different, and the confidence of the objects is obtained through object detection (YoloV 5) (ii)
Figure 895464DEST_PATH_IMAGE022
) And an object number (k) which is determined by the significance value of the pixel as the degree of likelihood that the current object is significant, that is, the degree of importance of the current object, and the sound basically accompanies the occurrence of the event, wherein the weight of the event determined by the sound is 2 times the weight, and the obtained object is ranked as:
Figure 145179DEST_PATH_IMAGE019
Figure 192770DEST_PATH_IMAGE021
representing the ranking of the kth object, wherein N is the number of pixels, and Norm is the maximum and minimum normalization function of the matrix;
(5) smoothing based on panoramic surveillance video inter-frame consistency
After a plurality of objects are ranked, the same objects between frames need to be connected to form a smooth window motion track, in order to prevent the window position from generating serious jump caused by the object motion, continuous sliding between windows needs to be modeled as inter-frame consistency, the inter-frame consistency information is that the window coordinate change is within a certain range (5 pixels), the coordinate change beyond the range is regarded as the content beyond the motion range, and the obtained motion track of the point is as follows:
Figure 502528DEST_PATH_IMAGE031
Figure 573252DEST_PATH_IMAGE024
' an initial trajectory of a representative point,
Figure 626659DEST_PATH_IMAGE024
represents the smoothed track of points, smooths represents the mean smoothing of the track,
Figure 466439DEST_PATH_IMAGE025
is a point
Figure 947099DEST_PATH_IMAGE026
The surrounding c =5 field of view,
Figure 318169DEST_PATH_IMAGE027
is a point
Figure 175266DEST_PATH_IMAGE026
Pixel points in the surrounding c =5 field;
(6) panoramic monitoring content output based on saliency region recommendation
After the motion tracks of different objects are formed, the panoramic security content of a single window (single object) or multiple windows (multiple objects) needs to be provided in a screen manner, and therefore, points need to be converted into a plane so as to be displayed in a display, specifically:
Figure 869553DEST_PATH_IMAGE028
wherein windows represents an output window,
Figure 521114DEST_PATH_IMAGE029
representing point-based trajectories
Figure 566430DEST_PATH_IMAGE024
(ii) a perspective projection;
(7) dynamically adjusting and manually intervening monitoring content
The window obtained in the step (6) is a program recommended window, if the window for directly playing the program recommended window conforms to human eyes under most conditions, however, a part of the window still exists, which may not be the content really concerned by security personnel, for example, the security personnel may want to patrol the garage once to see whether the garage is safe or not, at this time, even if the garage does not work at this time, if the security personnel needs to convert the window view angle determination right into self control, the mouse needs to be moved to the object, and the object concerned is clicked, at this time, the mouse mark point is used as the center to form a projection window aiming at the center as the projection center, and subsequent monitoring is provided based on the window.
Algorithms and image processing processes not disclosed herein are common or prior art in the art.
It is noted that the present embodiment is intended to aid in further understanding of the present invention, but those skilled in the art will understand that: various substitutions and modifications are possible without departing from the spirit and scope of the invention and appended claims. Therefore, the invention should not be limited to the embodiments disclosed, but the scope of the invention is defined by the appended claims.

Claims (8)

1. A parking lot panoramic safety monitoring method is characterized by comprising the following steps:
(1) collecting a panoramic camera video segment, and constructing a parking lot panoramic safety monitoring video data set according to the collected panoramic camera video segment;
(2) detecting according to the panoramic monitoring video data set of the parking lot based on the motion information, the comparison information and the significance of the sound information respectively;
(3) finding out corresponding important objects based on the significance detection results of the motion information, the comparison information and the sound information, and ranking the importance of the found important objects;
(4) carrying out panoramic monitoring video interframe consistency smoothing on the ranked objects, and connecting the same objects among the frames to form a smooth window motion track;
(5) converting the points into planes according to the motion tracks of different objects formed in the step (4), and providing single-window single object) or multi-window panoramic safety content in a screen mode;
(6) and (5) carrying out dynamic adjustment and manual intervention on the content monitored by the window content output in the step (5), if the window content is not the content wanted to be watched, only moving a mouse to an object needing to be watched, clicking the object, forming a projection window taking a mouse mark point as the center at the moment, and providing subsequent monitoring based on the window.
2. The parking lot panoramic safety monitoring method according to claim 1, wherein the parking lot panoramic safety monitoring video data set in the step (1) adopts a saliency detection data set as a training set, in the process of constructing the training set, the whole data set is trained firstly, and then a data set related to a vehicle is selected as fine tuning data, so that the model focuses on the saliency areas of the vehicle, the pedestrian and the obstacle, and the saliency detection data set comprises a video saliency detection data set DAVSOD which best conforms to the visual characteristics of human eyes and is the largest, an audio contrast saliency detection data set MSRA10K which best conforms to the visual characteristics of human eyes and is the largest, and an audio saliency detection data set AVEDataset which best conforms to the visual characteristics of human eyes and is the largest.
3. The parking lot panoramic safety monitoring method according to claim 2, wherein the motion information significance in the step (2) is based on the displacement relation between the front frame and the rear frame of the object in the monitoring video, the speed of the object moving relative to the background area is obtained to judge the state of the object, and two adjacent frames (I) are calculated through FlowNet21,I2) Obtaining motion information between videos, optical flow information
Figure 205419DEST_PATH_IMAGE001
To calculate the velocity difference between two pixels:
Figure 703396DEST_PATH_IMAGE002
Figure 740622DEST_PATH_IMAGE003
Figure 691261DEST_PATH_IMAGE004
Figure 162649DEST_PATH_IMAGE005
is the distance the pixel has moved along the x-axis and y-axis respectively,
Figure 780712DEST_PATH_IMAGE006
is pixel from I1To I2The time, x, y are the coordinate positions of the initial pixel points, and t is the movement
Figure 723260DEST_PATH_IMAGE004
Figure 364457DEST_PATH_IMAGE005
The time required; after optical flow information is obtained, the significance detection network is perceived by designing the optical flow
Figure 785074DEST_PATH_IMAGE007
And adopting the DAVSOD as a motion information perception training data set to convert the optical flow characteristics into motion significance characteristics:
Figure 523223DEST_PATH_IMAGE008
flow2Color stands for converting an optical Flow graph into an RGB graph, different colors represent different motion directions, shades represent the speed of motion,
Figure 636672DEST_PATH_IMAGE007
taking the RGB-map as an input,
Figure 889799DEST_PATH_IMAGE009
the motion significance detection result is output.
4. The parking lot panoramic safety monitoring method according to claim 3, wherein the comparison information significance of step (2) is based on the position, color and texture characteristics of the object, the object is highlighted by comparing with the background information and the object, the MSRA10K is used as a comparison information perception training data set, the picture-based significance detection algorithm ContrastNet outputs the scale comparison information of the perception object through different layer sides of the network layer,
Figure 114107DEST_PATH_IMAGE010
Figure DEST_PATH_IMAGE011
respectively representing color information, texture information, edge information,
Figure 441183DEST_PATH_IMAGE012
represents the multi-scale information of the multi-scale information,
Figure 725534DEST_PATH_IMAGE013
taking as input the RGB map of MSRA10K,
Figure 341323DEST_PATH_IMAGE014
and the output comparative significance detection result is a graph.
5. The parking lot panoramic safety monitoring method according to claim 4, wherein the sound information significance detection in the step (2) is implemented by taking sound as an aid of visual information, adopting AVEDataset as a sound information perception training data set, converting audio information into a spectrogram through Fourier transform, inputting the spectrogram into a network, and judging whether the current input contains sound information, so as to determine the importance of the current frame, and specifically comprises the following steps:
Figure 634901DEST_PATH_IMAGE015
wherein,
Figure 816484DEST_PATH_IMAGE016
in order to be a fourier transform,
Figure 84785DEST_PATH_IMAGE017
taking the fourier transform map as an input,
Figure 984608DEST_PATH_IMAGE018
the sound significance detection result graph is output.
6. The parking lot panoramic safety monitoring method according to claim 5, wherein the object importance ranking in step (3) is realized by the following formula:
Figure 816298DEST_PATH_IMAGE019
wherein
Figure 117966DEST_PATH_IMAGE020
Representing the ranking of the kth object, N is the number of pixels, Norm is the maximum and minimum normalization function of the matrix,
Figure 681803DEST_PATH_IMAGE021
as object confidence, object confidence
Figure 68922DEST_PATH_IMAGE021
And the object number k is obtained by object detection YoloV 5.
7. The parking lot panoramic safety monitoring method according to claim 6, wherein in the step (4), the inter-frame consistency information is that the window coordinate changes within 5 pixels, the coordinate change beyond the range is regarded as the content beyond the motion range, and the formed motion trail is as follows:
Figure 704303DEST_PATH_IMAGE022
Figure 922794DEST_PATH_IMAGE023
' an initial trajectory of a representative point,
Figure 719849DEST_PATH_IMAGE023
representing the smoothed tracks of the points, Smooth represents the averaging smoothing of the tracks,
Figure 594264DEST_PATH_IMAGE024
is a point
Figure 767756DEST_PATH_IMAGE025
The surrounding area c =5 is the area,
Figure 981700DEST_PATH_IMAGE026
is a point
Figure 949656DEST_PATH_IMAGE025
Surrounding c =5 pixels within the field.
8. The parking lot panoramic safety monitoring method according to claim 7, wherein in the step (5), the points are converted into planes by the following formula:
Figure 311367DEST_PATH_IMAGE027
wherein windows represents an output window,
Figure 22971DEST_PATH_IMAGE028
representing point-based trajectories
Figure 888159DEST_PATH_IMAGE023
The perspective projection.
CN202210535421.9A 2022-05-18 2022-05-18 Panoramic safety monitoring method for parking lot Active CN114639171B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210535421.9A CN114639171B (en) 2022-05-18 2022-05-18 Panoramic safety monitoring method for parking lot

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210535421.9A CN114639171B (en) 2022-05-18 2022-05-18 Panoramic safety monitoring method for parking lot

Publications (2)

Publication Number Publication Date
CN114639171A true CN114639171A (en) 2022-06-17
CN114639171B CN114639171B (en) 2022-07-29

Family

ID=81952844

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210535421.9A Active CN114639171B (en) 2022-05-18 2022-05-18 Panoramic safety monitoring method for parking lot

Country Status (1)

Country Link
CN (1) CN114639171B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115410189A (en) * 2022-10-31 2022-11-29 松立控股集团股份有限公司 Complex scene license plate detection method
CN116228834A (en) * 2022-12-20 2023-06-06 阿波罗智联(北京)科技有限公司 Image depth acquisition method and device, electronic equipment and storage medium

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105303571A (en) * 2015-10-23 2016-02-03 苏州大学 Time-space saliency detection method for video processing
US20160086052A1 (en) * 2014-09-19 2016-03-24 Brain Corporation Apparatus and methods for saliency detection based on color occurrence analysis
CN106878674A (en) * 2017-01-10 2017-06-20 哈尔滨工业大学深圳研究生院 A kind of parking detection method and device based on monitor video
CN106951870A (en) * 2017-02-15 2017-07-14 重庆警察学院 The notable event intelligent detecting prewarning method of monitor video that active vision notes
CN110827193A (en) * 2019-10-21 2020-02-21 国家广播电视总局广播电视规划院 Panoramic video saliency detection method based on multi-channel features
CN113128344A (en) * 2021-03-19 2021-07-16 杭州电子科技大学 Multi-information fusion stereo video saliency detection method
US20210398294A1 (en) * 2019-05-27 2021-12-23 Tencent Technology (Shenzhen) Company Limited Video target tracking method and apparatus, computer device, and storage medium
CN113920468A (en) * 2021-12-13 2022-01-11 松立控股集团股份有限公司 Multi-branch pedestrian detection method based on cross-scale feature enhancement
CN114299744A (en) * 2021-12-22 2022-04-08 广东艾科智泊科技股份有限公司 Panoramic parking lot system and management scheme

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160086052A1 (en) * 2014-09-19 2016-03-24 Brain Corporation Apparatus and methods for saliency detection based on color occurrence analysis
CN105303571A (en) * 2015-10-23 2016-02-03 苏州大学 Time-space saliency detection method for video processing
CN106878674A (en) * 2017-01-10 2017-06-20 哈尔滨工业大学深圳研究生院 A kind of parking detection method and device based on monitor video
CN106951870A (en) * 2017-02-15 2017-07-14 重庆警察学院 The notable event intelligent detecting prewarning method of monitor video that active vision notes
US20210398294A1 (en) * 2019-05-27 2021-12-23 Tencent Technology (Shenzhen) Company Limited Video target tracking method and apparatus, computer device, and storage medium
CN110827193A (en) * 2019-10-21 2020-02-21 国家广播电视总局广播电视规划院 Panoramic video saliency detection method based on multi-channel features
CN113128344A (en) * 2021-03-19 2021-07-16 杭州电子科技大学 Multi-information fusion stereo video saliency detection method
CN113920468A (en) * 2021-12-13 2022-01-11 松立控股集团股份有限公司 Multi-branch pedestrian detection method based on cross-scale feature enhancement
CN114299744A (en) * 2021-12-22 2022-04-08 广东艾科智泊科技股份有限公司 Panoramic parking lot system and management scheme

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
XIAOQI ZHAO ET.AL: "Multi-Source Fusion and Automatic Predictor Selection for Zero-Shot Video Object Segmentation", 《PROCEEDINGS OF THE 29TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA》 *
卢晶等: "融合运动信息的三维视觉显著性算法研究", 《计算机工程》 *
杨涛涛: "基于全景拼接的视频防抖算法设计", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115410189A (en) * 2022-10-31 2022-11-29 松立控股集团股份有限公司 Complex scene license plate detection method
CN116228834A (en) * 2022-12-20 2023-06-06 阿波罗智联(北京)科技有限公司 Image depth acquisition method and device, electronic equipment and storage medium
CN116228834B (en) * 2022-12-20 2023-11-03 阿波罗智联(北京)科技有限公司 Image depth acquisition method and device, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN114639171B (en) 2022-07-29

Similar Documents

Publication Publication Date Title
CN114639171B (en) Panoramic safety monitoring method for parking lot
KR102129893B1 (en) Ship tracking method and system based on deep learning network and average movement
US8582816B2 (en) Method and apparatus for video analytics based object counting
CN101916383B (en) Vehicle detecting, tracking and identifying system based on multi-camera
CN101950426B (en) Vehicle relay tracking method in multi-camera scene
JP4919036B2 (en) Moving object recognition device
CN107133559B (en) Mobile object detection method based on 360 degree of panoramas
EP3970060A1 (en) Driver attention detection using heat maps
CN111860274B (en) Traffic police command gesture recognition method based on head orientation and upper half skeleton characteristics
KR102253989B1 (en) object tracking method for CCTV video by use of Deep Learning object detector
KR20080085837A (en) Object density estimation in vedio
JP2009271758A (en) Image-recognizing device
JP5106302B2 (en) Moving object tracking device
US11161456B1 (en) Using the image from a rear view camera in a three-camera electronic mirror system to provide early detection of on-coming cyclists in a bike lane
JP2005243019A (en) Method for detecting traffic event in compressed video
Zhang et al. Surveillance video anomaly detection via non-local U-Net frame prediction
JP2013008070A (en) Sign recognition device and sign recognition method
CN112084928A (en) Road traffic accident detection method based on visual attention mechanism and ConvLSTM network
CN112380905A (en) Abnormal behavior detection method based on histogram and entropy of surveillance video
JP2013093865A (en) Vehicle periphery monitoring device
Hara et al. Predicting appearance of vehicles from blind spots based on pedestrian behaviors at crossroads
Miller et al. Intelligent Sensor Information System For Public Transport–To Safely Go…
WO2021181861A1 (en) Map data generation device
CN117998039A (en) Video data processing method, device, equipment and storage medium
Hara et al. Predicting vehicles appearing from blind spots based on pedestrian behaviors

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant