WO2021036636A1 - Vibration detection method and apparatus for lifting device, server and storage medium - Google Patents

Abstract

A vibration detection method and apparatus for a lifting device, a server and a computer storage medium, the method comprising: acquiring video data that comprises a lifting device; analyzing the video data to obtain vibration data of the lifting device; according to the vibration data of the lifting device, determining whether the lifting device is working abnormally; and if so, then generating emergency prompt information and sending same to a maintenance terminal. Using a variety of algorithms to process the video data may reduce the burden of manual detection, the lifting device may be detected in real time, and possible failures of the lifting device are promptly predicted, thereby greatly reducing the probability of safety hazards caused by the failures of the lifting device.

Description

Vibration detection method, device, server and storage medium of lifting equipment Technical field

This application relates to the technical field of video recognition, in particular to a vibration detection method, device, server and computer storage medium for lifting equipment.

Background technique

With the development of technology, the elevator has become an indispensable infrastructure in the society, and the elevator needs to rely on structures such as hoists and wire ropes to operate. It is very important to ensure the safety of elevators. At present, the common way is to send professionals to inspect and maintain the elevators on a regular basis. However, sometimes the elevator will fail during one inspection period. For example, a wire rope carrying the elevator may be inspected and maintained by professionals. In the next day, there was a tendency to break. At this time, it is often impossible to detect and repair the elevator's wire ropes and winches in time, which greatly increases the probability of potential safety hazards caused by elevator failure.

Summary of the invention

Based on the above problems, the present application provides a vibration detection method, device, server and storage medium for lifting equipment, which can reduce the burden of manual detection, and can perform real-time detection of the lifting equipment, greatly reducing the probability of safety hazards caused by elevator failure.

The first aspect of the embodiments of the present application provides a vibration detection method for lifting equipment, the method including:

Acquiring video data including the lifting device, where the video data includes video frame data;

Extracting the brightness information of each pixel in the video frame data;

Performing Fourier transform on the brightness information of each pixel in the video frame data to obtain processed video frame data;

Using a motion zoom processing algorithm to perform motion zoom processing on the processed video frame data to obtain zoomed video frame data;

The amplified video frame data is subjected to phase correlation calculation, interpolation filtering processing, and inverse Fourier transform to obtain vibration data of the lifting device, where the vibration data includes vibration amplitude, vibration frequency, and vibration phase;

Judging whether the lifting equipment is working abnormally according to the vibration data of the lifting equipment;

If yes, generate an emergency prompt message and send it to the maintenance terminal.

A second aspect of the embodiments of the present application provides a vibration detection device for lifting equipment, and the vibration detection device includes:

A data acquisition unit for acquiring video data including the lifting equipment;

A brightness extraction unit for extracting brightness information of each pixel in the video frame data;

A brightness processing unit, configured to perform Fourier transform on the brightness information of each pixel in the video frame data to obtain processed video frame data;

An amplification processing unit, configured to perform motion amplification processing on the processed video frame data by using a motion amplification processing algorithm to obtain amplified video frame data;

The vibration acquisition unit is used to obtain the vibration data of the lifting device after phase correlation calculation, interpolation filtering processing, and inverse Fourier transform on the amplified video frame data, and the vibration data includes vibration amplitude, vibration frequency, Vibration phase

The state judgment unit is configured to judge whether the lifting equipment is working abnormally according to the vibration data of the lifting equipment;

The prompt sending unit is used to generate emergency prompt information and send it to the maintenance terminal when the lifting equipment works abnormally.

A third aspect of the embodiments of the present application provides a server. The server includes a processor, a communication interface, and a memory. The processor, the communication interface, and the memory are connected to each other. The memory is used to store a computer program. The computer program includes program instructions, and the processor is configured to invoke the program instructions to execute the method described in the first aspect of the embodiments of the present application.

The fourth aspect of the embodiments of the present application provides a computer-readable storage medium that stores a computer program, and the computer program includes program instructions that, when executed by a processor, cause the processor to Perform the method described in the first aspect of the embodiments of the present application.

By implementing the above application examples, the following beneficial effects can be obtained:

The vibration detection method, device, server and computer storage medium of the above-mentioned lifting equipment obtain the video data including the lifting equipment; analyze and process the video data to obtain the vibration data of the lifting equipment; The vibration data determines whether the lifting equipment is working abnormally; if it is, an emergency prompt message is generated and sent to the maintenance terminal. Using a variety of algorithms to process video data can reduce the burden of manual detection, and can perform real-time detection of lifting equipment, and timely predict possible failures of the lifting equipment, which greatly reduces the probability of safety hazards caused by the failure of the lifting equipment.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. Ordinary technicians can obtain other drawings based on these drawings without creative work.

Figure 1 is a system architecture diagram of a vibration detection method in an embodiment of the application;

FIG. 2 is an application scenario diagram of the vibration detection method in an embodiment of the present application;

FIG. 3 is a schematic flowchart of a vibration detection method in an embodiment of the application;

FIG. 4 is a schematic diagram of the flow of step 302 in an embodiment of this application

FIG. 5 is a schematic flowchart of another vibration detection method in an embodiment of the application;

Fig. 6 is a schematic structural diagram of a vibration detection device in an embodiment of the application;

FIG. 7 is a schematic structural diagram of another vibration detection device in an embodiment of the application;

FIG. 8 is a schematic structural diagram of a server in an embodiment of the application.

detailed description

In order to enable those skilled in the art to better understand the solutions of the application, the technical solutions in the embodiments of the application will be clearly and completely described below in conjunction with the drawings in the embodiments of the application. Obviously, the described embodiments are only It is a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by a person of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The terms "including" and "having" in the specification and claims of this application and the above-mentioned drawings and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally also includes Other steps or units inherent in a process, product, or equipment.

Reference to "embodiments" herein means that a specific feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art clearly and implicitly understand that the embodiments described herein can be combined with other embodiments.

The following describes the system architecture of the vibration detection method of the lifting equipment in the embodiment of the application in detail with reference to FIG. 1. FIG. 1 is a system architecture diagram of the vibration detection method in the embodiment of the application, which specifically includes a camera 110, a server 120, and a maintenance terminal 130. The above-mentioned camera 110 may be a plurality of cameras with night-time camera functions, which are scattered in the operating space of the lifting equipment, and the above-mentioned lifting equipment can be photographed from various directions. The above-mentioned camera 110 can be automatically zoomed or remotely controlled, and can be photographed in real time. The video data is uploaded to the server 120; the server 120 can obtain the video data captured by the camera 110 and perform vibration detection, remotely determine whether the lifting equipment is working abnormally in real time, and connect the maintenance terminal 130 to provide information when the lifting equipment is abnormal. The above-mentioned maintenance terminal 130 sends prompt information. The above-mentioned maintenance terminal 130 is a terminal of an organization or individual responsible for repairing the above-mentioned lifting equipment. The above-mentioned maintenance terminal 130 may include, but is not limited to, mobile phones, computers, tablets, and other smart devices with communication connection functions.

Optionally, the maintenance terminal 130 may interact with the server 120 bidirectionally, and the maintenance terminal 130 may send maintenance completion information to the server 120 or feed back feedback information that the server 120 judges the status of the lifting equipment incorrectly.

In order to explain the above system architecture more clearly, the actual application scenario of the embodiment of the present application will be described in detail with reference to FIG. 2. FIG. 2 is an application scenario diagram of the vibration detection method in the embodiment of the application. The application scenario is an elevator, and the elevator generally includes a car. Car body, traction rope, bearing system, in the embodiment of the present application, the above-mentioned bearing system includes a hoist that provides the operating power of the elevator, which is not shown in the figure; the above-mentioned car body makes vertical reciprocating motion in the elevator through the traction rope The video capture device can be set in each area of the elevator shaft to obtain the video data of the car body and the traction rope without blind spots. The video capture device can include a camera array with night vision function and a communication unit. The communication unit is used to The collected video data is sent to the server. The built-in trained state judgment model of the server can judge whether the elevator is working abnormally according to the video data. For example, if there are four traction ropes, it will be generated after one is broken. For abnormal vibration data, the server can discover the abnormality of the above-mentioned elevator based on the vibration data, and send the abnormal information to the maintenance terminal so that the maintenance personnel can go for maintenance.

Through the above system architecture, the burden of manual detection can be reduced, and the lifting equipment can be detected in real time, and the possible failures of the lifting equipment can be predicted in time, which greatly reduces the probability of potential safety hazards caused by the failure of the lifting equipment.

The following is a detailed description of the vibration detection method of the lifting equipment in the embodiment of the application with reference to FIG. 3. FIG. 3 is a schematic flowchart of the vibration detection method in the embodiment of the application, which specifically includes the following steps:

Step 301: The server obtains video data containing the lifting equipment.

Wherein, the above-mentioned lifting equipment includes a car body, a traction rope and a hoist, and the server can respectively obtain video frame data of the same time period including the car body and the traction rope of the lifting equipment, and the above-mentioned video data may be for For multiple videos shot at the same time in different areas of the above-mentioned lifting equipment, the server can obtain the video data containing the lifting equipment through a wired connection or a wireless connection. It should be noted that different lifting equipment can have different shapes and sizes. There are no specific restrictions on the car body, different numbers of traction ropes, and different power winches.

The video data containing the lifting equipment can be acquired through the server, and the vibration of the lifting equipment can be detected by the method of video detection, which can reduce the burden of manual detection, and can detect the lifting equipment in real time, and timely predict the possible occurrence of the lifting equipment Failure, greatly reducing the probability of safety hazards caused by failure of the lifting equipment.

Step 302: The server analyzes and processes the video data to obtain vibration data of the lifting equipment.

Wherein, the above-mentioned vibration data may be a numerical value obtained after calculation, which is used to indicate the vibration condition of the above-mentioned lifting equipment. For the steps of analyzing and processing the above-mentioned video data by the server, refer to FIG. Schematic diagram of the process, specifically including the following steps:

Step 401: The server obtains the towing rope in the video data and the video frame data of the car body.

Wherein, the above-mentioned server can split the video data of the towing rope and the area of the car body according to the frame to obtain the video frame data. It should be noted that the above-mentioned video data can be divided into video frames according to different settings as needed. For example, it can be set to one frame per second or one frame per two seconds, and the resolution of the split video frame data can be set. The split setting is not specifically limited here.

Obtaining the traction rope in the video data and the video frame data of the car body through the server can process the video data more conveniently and improve the processing accuracy.

Step 402: The server extracts the brightness information of each pixel in the video frame data.

Among them, the server can convert the above-mentioned video frame data from RGB color space to YIQ color space. The above-mentioned YIQ is the National Television Standards Committee (NTSC), Y is the luminance signal (Luminance) that provides black-and-white TV and color TV, I Represents In-phase, the color changes from orange to cyan, Q represents Quadrature-phase, and the color changes from purple to yellow-green. Specifically, the conversion relationship between RGB and YIQ is:

Y＝0.299*R+0.587*G+0.114*B;

I=0.596*R–0.275*G–0.321*B;

Q=0.212*R-0.523*G+0.311*B.

The subsequent steps are all based on the brightness information of the YIQ color space. Therefore, it is necessary to separate the brightness information and chrominance information of each pixel in the video frame data to extract the brightness information of each pixel in the video frame data.

The server extracts the brightness information of each pixel in the video frame data, which can reduce the calculation amount of the algorithm and improve the running speed of the algorithm.

Step 403: The server performs Fourier transform on the brightness information of each pixel in the video frame data to obtain processed video frame data.

Wherein, the server performs Fourier transform on the above-mentioned brightness information, converts the brightness change in the time domain into a phase change in the frequency domain, and obtains the processed video frame data.

Specifically, Fourier transform is performed on the Y channel images of the multiple YIQ images to obtain multiple Y channel images of the above-mentioned lifting device in the frequency domain space; the multiple Y channel images are spatially decomposed based on the complex steerable gold tower, Thus, a plurality of corresponding first sub-band image sets are obtained. Each first subband image set includes a plurality of subband images corresponding to a plurality of image resolutions, the scales and directions of any two subband images in the plurality of subband images are different from each other, and the plurality of first subband images The sub-band images in the image set form multiple sub-band image sequences corresponding to the foregoing multiple image resolutions, and any two sub-band images in each sub-band image sequence are from different first sub-band image sets.

The server performs Fourier transform on the brightness information of each pixel in the video frame data to obtain the processed video frame data. The brightness change can be converted into a phase change in the frequency domain that is more suitable for algorithm calculations, which improves the algorithm Operating efficiency.

Step 404: The server uses a motion zoom processing algorithm to perform motion zoom processing on the processed video frame data to obtain zoomed video frame data.

Among them, the above-mentioned amplification processing algorithm may include a motion amplification algorithm based on the Laplacian view, a video motion amplification algorithm based on Euler motion modulation, a video motion amplification algorithm based on Euler video modulation of complex phase, and a fast phase using RIESZ pyramid Any of the video motion zoom algorithms.

Specifically, the aforementioned motion amplification algorithm based on the Laplace perspective is:

Identify the feature points in the video frame data and cluster the trajectories of the feature points to obtain clusters of multiple sets of actions. Then, perform optical flow field interpolation on each pixel of each frame of image to obtain dense light Flow field. Track the movement trajectories of these points over time to obtain different action layers, fill the movement layers with texture, increase the movement amplitude of the selected action layers, and finally get the enlarged video frame data.

Specifically, the above-mentioned video motion amplification algorithm based on Euler motion modulation is:

Perform spatial pyramid decomposition of video frame data to obtain video frame data with different spatial resolutions; perform time-domain band-pass filtering on video frame data of different scales on the pyramid to filter out several frequency bands; linearly amplify the above several frequency bands, and Reorganize to obtain the enlarged video frame data.

Specifically, the foregoing video motion amplification algorithm based on Euler video modulation of complex phase is:

The video frame data is decomposed in the spatial domain of the complex manipulated pyramid, and the video frame data of different scales on the pyramid is subjected to time-domain band-pass filtering to filter out several frequency bands; the above several frequency bands are linearly amplified, and the amplified video frame is obtained by reorganization data. The algorithm can decompose the video frame data more controllably. Through the direction controllable filter, the video frame data can be decomposed into subbands in any direction without aliasing. Due to the translation invariance of the complex video frame pyramid, the distortion phenomenon in the video frame processing process can be effectively reduced.

Specifically, the above-mentioned fast phase video motion amplification algorithm using the RIESZ pyramid is:

Change the pyramid of the spatial decomposition step in the video motion amplification algorithm based on Euler motion modulation to the RIESZ pyramid. Can greatly improve the calculation speed.

By using the motion amplification processing algorithm to perform motion amplification processing on the processed video frame data, the amplified video frame data can be obtained, which can obtain images that cannot be recognized by human eyes, and improve the probability of obtaining correct vibration data of the lifting equipment.

In an optional embodiment, before performing the magnification processing, at least one subband image sequence used for magnification processing may be selected from the multiple subband image sequences according to a preset partition gray value selection strategy;

Among them, by identifying the gray value of each pixel in each sub-band image sequence, and by performing partition processing on each sub-band image in each sub-band image sequence, determine the pixel gray value of each area after the partition, thereby filtering At least one sub-band image sequence used for zoom-in processing can be generated, which can improve the processing efficiency of the original video, obtain more accurate zoom-in motion information, and avoid the need for increased calculations for zooming in higher-resolution images.

Wherein, the partition gray value screening strategy can be based on the proportion of the foreground image and the background image of each sub-band image sequence, and can also be based on the shape of the detected product, for example, the detected product is a circle. The shape can be divided from the center of the circle with concentric shapes with different radii.

Afterwards, the at least one sub-band image sequence after the enlargement processing is obtained by performing an enlargement process on the at least one sub-band image sequence;

Wherein, the amplification processing of the at least one sub-band image sequence set may be based on the gray value difference of the same pixel in each image sequence at different times, that is, amplifying the gray value change of a certain pixel. frequency.

In a specific implementation, it is also possible to perform partition enlargement processing, by identifying the foreground image and the background image, and amplifying the gray value difference of the pixels of the foreground image, so as to obtain at least one sub-band image sequence after the enlargement processing.

By performing fusion processing on the at least one sub-band image sequence after the enlargement processing and the sub-band image sequences of the plurality of sub-band images other than the at least one sub-band image sequence, a motion amplification effect is obtained For the target video, the motion zooming effect refers to zooming in on the image of the reciprocating motion of the detected product in the original video.

Specifically, in a possible example, the filtering out at least one sub-band image sequence for magnification from the multiple sub-band image sequences according to a preset partition gray value screening strategy includes: determining the multiple sub-band image sequences. A foreground image and a background image of a sub-band image sequence, the foreground image includes an image of the reciprocating motion area of the detected product, and the background image is an image other than the image of the detected product; The area proportion of the foreground image in the sub-band image; the number of sub-divisions of the foreground image is determined according to the area proportion and a preset division calculation formula, and the foreground image is divided into multiples according to the number of sub-divisions For each sub-band image sequence, perform the following operations (1)-(6) to obtain the gray value change frequency of each sub-band image sequence: (1) Determine the current processing sub-band image sequence The measured pixel points of each foreground sub-division of each sub-band image; (2) According to the gray value of the multiple sub-band images contained in the current sub-band image sequence of each measured pixel point, the measured pixel point is generated Waveform diagram of gray value time-domain changes; (3) Perform the following (a), (b), and (c) operations for each foreground sub-region: (a) According to the multiple measured pixels contained in the currently processed foreground sub-region The time-domain change waveform diagram of the gray value of a point to determine whether the currently processed foreground sub-region contains the measured pixel point whose gray value changes periodically; (b) if yes, mark the currently processed foreground sub-region as The selected foreground sub-region; (c) If not, mark the currently processed foreground sub-region as an unselected foreground sub-region; (4) For the multiple selected foreground sub-regions after marking, follow the region Correlation splices foreground sub-regions with adjacent relationships into vibrating reference areas; (5) determining a plurality of reference pixel points whose gray value periodically changes among a plurality of pixels in the vibrating reference area, and determining each The gray value change frequency of the reference pixel; (6) Weighted calculation of the gray value change frequency of the multiple reference pixels in the vibration reference area to obtain the gray value change of the currently processed subband image sequence Frequency: According to the gray value change frequency of each sub-band image sequence, at least one sub-band image sequence that meets the preset reference vibration frequency is screened out.

Wherein, the determination of the foreground image and the background image of the plurality of sub-band image sequences may be determined according to the pixel gray value of each image in the plurality of sub-band image sequences; and may also be determined according to the plurality of sub-band images The sequence is determined by the reference feature points in the relatively static state in the original video. Secondly, determine the area proportion of the foreground image in the sub-band image according to each sub-band image and the foreground image of each sub-band image in the plurality of sub-band image sequences, and according to the area proportion and preset calculations The formula calculates the number of foreground sub-regions of each foreground image, where the larger the area of the foreground image, the greater the number of foreground sub-regions.

In specific implementation, after the multiple foreground sub-regions are obtained, the measured pixel point of each foreground sub-region is determined according to a preset strategy, and the measured pixel point of each sub-band image in the sub-band image sequence is determined. The gray value change generates a waveform diagram of the time domain change of the gray value of a pixel.

Said determining whether the currently processed foreground sub-region contains the measured pixel point whose gray value periodically changes according to the gray value time-domain change waveform diagram of the plurality of measured pixel points contained in the currently processed foreground sub-region, When it is determined that the gray value time domain change waveform diagram has a periodically fluctuating waveform, it is determined that the gray value of the pixel point is periodically changed, then the foreground sub-region where the pixel point is located is selected and marked; it will be marked The multiple foreground sub-divisions of are spliced according to the regional relevance or image color space relevance to obtain the vibration reference area. According to the change of the gray value of the pixel in the reference vibration area, determine a plurality of pixels that change periodically in the reference vibration area, and calculate the gray value change frequency of each reference pixel according to the formula, for example: H _t1 represents Is the gray value of a certain pixel at t1, H _t2 represents the gray value of this point at t2, and the gray value change frequency of this point is

Perform weighted calculation on the gray value change frequency of multiple pixels in the reference vibration area, for example, the gray value change frequency of the reference vibration area

Then, the gray value change frequency of the currently processed sub-band image sequence is determined according to the gray value change frequency H; and at least one sub-band image sequence that meets the preset reference vibration frequency is obtained by screening. Or add the gray value change frequencies H of different time spans in at least one subband image sequence to obtain a plurality of different change frequency values, and select from the plurality of different change frequency values to comply with a preset With reference to the vibration frequency, at least one sub-band image sequence conforming to the preset reference vibration frequency is determined.

In specific implementation, the original video of the detected product can be further divided into multiple sub-band image sequences, multiple image sequences are divided into multiple image groups, and then the gray value of each grouped pixel is determined, according to the pixel The change of the gray value of the point determines at least one sub-band image sequence, and performs the enlargement process.

It can be seen that in this example, the image area for detecting the gray value of the pixel can be determined based on the area ratio of the foreground image in the sub-band image sequence, so as to improve the processing efficiency of the original video and avoid the enlargement of the higher resolution image. The need to increase the amount of computing.

In a possible example, the preset calculation formula is:

Among them, x is the area ratio, y is the number of sub-divisions, and x is greater than 0 and less than or equal to 1.

Wherein, the coefficient and base of the preset formula can be adjusted according to the size of the subband image.

For example, when the area accounts for 50%, the number of partitions is

The result can be rounded to the nearest whole number.

It can be seen that, in this example, the number of foreground sub-divisions I is obtained by the proportion of the foreground image and the preset calculation formula, and the foreground image is partitioned, which avoids the processing of pixels in the background image. The amount of calculations improves the efficiency of image processing.

In a possible example, the determination of the measured pixel points of each foreground sub-region of each sub-band image in the currently processed sub-band image sequence includes at least one of the following: edge pixels of each foreground sub-region Pixels in the middle area of each foreground sub-division; multiple pixels randomly selected from each of the foreground sub-divisions.

Wherein, the determining the pixel to be measured selects the edge pixel of each foreground sub-division, and the edge pixel may be all the pixels at the edge of each foreground sub-division, or a part of the pixel at the edge; the determining the pixel to be measured Point selection of the pixels in the middle area of each foreground sub-division can be the middle point of each sub-division, or the pixels of the preset area in the middle of each sub-division; or a random number of each foreground sub-division The multiple pixels are filtered out, and the multiple pixels are multiple pixels uniformly distributed in the foreground sub-region.

It can be seen that in this example, the foreground image of the tested product is partitioned, and the pixels of each sub-region are determined in a variety of ways, and then based on the change in the gray value of the pixel to be tested, the product that meets the reference vibration frequency of the product is selected. The image sequence realizes a variety of ways to confirm the pixels and improve the efficiency of data processing.

Step 405: The server uses phase correlation calculation, interpolation filtering processing, and inverse Fourier transform on the amplified video frame data to obtain vibration data of the lifting device.

Wherein, the aforementioned phase correlation calculation frame sequence adopts a phase correlation algorithm to calculate the cross cross power spectrum between the frame sequences. The phase correlation algorithm uses the following formula to calculate the cross cross power spectrum.

In the above formula, F. _A frame image of a Fourier transform,

Is the Fourier transformed conjugate signal of the b-frame image, and the lower side of the division formula is the modulus of the correlation product of the two Fourier transformed signals. R is the calculation result of this step cross cross power spectrum, the above cross cross power spectrum contains frequency domain noise; then an adaptive filter bank can be used to reconstruct the motion signal, and the filter bank can be adaptively selected for filtering according to the position of the correlation peak of R After filtering, perform inverse Fourier transform, and then perform phase comparison. At this time, the adaptive matching method of sliding window will be used to estimate and extract the vibration parameters to obtain the cross cross power spectrum R′ after filtering the frequency domain noise; The inverse Fourier transform is performed on the above-mentioned cross cross power spectrum R′, and the vibration data of the pixels in the video can be obtained by comparing phase by phase. The calculation formula is as follows:

r=F ^-1 {R'}

In the above formula, F ^-1 {R'} represents the inverse Fourier transform of the cross cross power spectrum, and the obtained r is the vibration data of the above-mentioned lifting equipment.

By analyzing and processing the video data by the server to obtain the vibration data of the lifting device, it can be determined that the working state of the lifting device is quantified in the form of vibration data, and the server can detect the vibration of the lifting device more accurately.

Step 303: The server judges whether the lifting device is working abnormally according to the vibration data of the lifting device.

The server may obtain the equipment parameters of the lifting equipment, and establish a state judgment model based on the equipment parameters of the lifting equipment and the vibration data. The state judgment model is used to judge the working status of the lifting equipment based on the vibration data of the lifting equipment. The working state of lifting equipment includes normal state and abnormal state.

Wherein, the above-mentioned equipment parameters may include the operating speed range, operating path, maximum load-bearing weight, etc. of the above-mentioned lifting equipment, and the above-mentioned state judgment model may be a ready-to-use state judgment model obtained after a large amount of vibration data is trained. The model can contain multiple status judgment modes, and can switch the corresponding status judgment modes according to different lifting equipment for status judgment.

Optionally, the server may obtain the current motion state data of the lifting device, the motion state data includes hovering state and running state; the motion state data and the vibration data are input into the state judgment model to determine whether the lifting device is working abnormally, The above-mentioned state judgment model includes a hovering judgment module and an operation judgment module.

Specifically, when the state of motion is a hovering state, the hovering judgment module determines whether the vibration amplitude, frequency, and phase of the traction rope and the car body exceed the hovering vibration threshold. The hovering vibration threshold is It can include hovering vibration amplitude threshold, hovering vibration frequency threshold, hovering vibration phase threshold, etc.;

When the state of motion is a hovering state, the hovering judgment module determines whether the vibration amplitude, frequency, and phase of the traction rope and the car body exceed the hovering vibration threshold. The hovering vibration threshold may include hovering. Hovering vibration amplitude threshold, hovering vibration frequency threshold, hovering vibration phase threshold, etc.;

If any one of the vibration amplitude, the vibration frequency, and the vibration phase exceeds the hovering vibration threshold, it is determined that the lifting equipment is working abnormally.

Among them, the difference between the above-mentioned hovering vibration threshold and the operating vibration threshold can be based on the motion state. For example, when the lifting equipment is in the hovering state, the vertical operating vibration amplitude threshold should be set relatively low, because At this time, the lifting equipment eliminates the vertical vibration interference during operation. If there is a large vibration amplitude in the vertical direction, it means that the probability of abnormality is greater; when the above lifting equipment is in operation, all items should be changed. The operating vibration threshold is set higher. Because there is already vibration during operation at this time, setting the various operating vibration thresholds higher can reduce the probability of misjudgment, and the specific values of the hovering vibration threshold and the operating vibration threshold There is no specific limitation here.

The above-mentioned state judgment model can judge whether the above-mentioned lifting equipment is working abnormally, and if yes, proceed to step 304; if it is working normally, continue to perform the above-mentioned step of obtaining video data containing the lifting equipment.

Step 304: The server generates emergency prompt information and sends it to the maintenance terminal.

Wherein, the above-mentioned maintenance terminal is a terminal corresponding to the maintenance personnel or organization responsible for repairing the above-mentioned lifting equipment. Optionally, the above-mentioned server can screen out the nearest maintenance terminal according to the position of the lifting equipment and send the emergency prompt information to the above-mentioned nearest terminal through wireless transmission. Repair terminal.

The emergency prompt information generated by the server and sent to the maintenance terminal can reduce the burden of manual detection, and can conduct real-time detection of the lifting equipment, and timely notify the relevant maintenance personnel, which greatly reduces the probability of potential safety hazards caused by elevator failure.

Hereinafter, another vibration detection method in an embodiment of the present application will be described in detail with reference to FIG. 5. FIG. 5 is a schematic flowchart of another vibration detection method in an embodiment of the present application, which specifically includes the following steps:

Step 501: The server obtains video data including the lifting device.

Step 502: The server analyzes and processes the video data to obtain vibration data of the lifting equipment.

Step 503: The server judges whether the lifting device is working abnormally according to the vibration data of the lifting device.

Wherein, when it is judged that the above-mentioned lifting equipment is working abnormally, step 504 is executed; when it is judged that the above-mentioned lifting equipment is working normally, the step of acquiring the video data containing the above-mentioned lifting equipment is re-executed.

Step 504: The server judges the abnormal state type of the lifting equipment according to the state judgment model.

Among them, the above-mentioned abnormal state types include fault state and dangerous state. The above-mentioned fault state may include abnormal speed of the lifting equipment, broken traction rope, abnormal shaking of the lifting equipment, etc. The above-mentioned dangerous state may include wear and deformation of the traction rope, abnormal power of the hoist, and friction of the traction rope. Excessive resistance, etc., is not specifically limited here. The server can determine which abnormal state or abnormal states the above-mentioned lifting equipment may be in according to the state judgment model.

The server judges the abnormal state type of the lifting equipment according to the state judgment model, and the model established by big data can be used to more accurately judge the abnormal state of the lifting equipment, which can reduce the burden of manual detection and greatly reduce the potential safety hazards caused by elevator failure The probability.

In step 505, the server generates an emergency prompt message and sends it to the maintenance terminal.

Wherein, the above-mentioned emergency prompt information may include an abnormal state identifier, location information of the lifting equipment, and an alternative maintenance plan. The abnormal state identifier may include a fault state identifier and a dangerous state identifier. The above-mentioned alternative maintenance plan may be one or one corresponding to the judgment result. Multiple alternative maintenance programs are used to provide reference for maintenance personnel.

The emergency prompt information generated by the server and sent to the maintenance terminal can reduce the burden of manual detection, and can detect the lifting equipment in real time, and predict the possible failures of the lifting equipment in time, which greatly reduces the probability of safety hazards caused by the failure of the lifting equipment.

Optionally, in step 506, the server obtains the feedback information of the maintenance terminal.

Wherein, the feedback information may be the actual status information of the lifting equipment, and the server may determine whether the previous status judgment result is wrong based on the actual status information. If the judgment is wrong, the status judgment model is corrected. If the judgment is correct, Then continue to use the above state judgment model for judgment.

Obtaining the feedback information of the maintenance terminal through the server can continuously evolve and improve the above state judgment model, better judge the working state of the lifting equipment based on the vibration data, timely predict the possible failures of the lifting equipment, and greatly reduce the failure of the lifting equipment. Probability of safety hazards.

Optionally, a camera and a microphone can be set in the car body, and the server can call the monitoring video in the car body to determine whether it is a human-made dangerous action that causes abnormal vibration. The human-made dangerous action can include jumping and other actions that can cause the car body to vibrate. If it is an abnormal vibration caused by man, an emergency prompt message can also be sent to the car body to prompt the person in the car body to stop the dangerous behavior by voice.

Optionally, the above-mentioned lifting equipment can be equipped with a gravity sensor, which can determine whether the current load-bearing exceeds the safe load-bearing threshold. For example, the safe load-bearing threshold of a passenger elevator can be 1500kg, and the safe load-bearing threshold of a cargo elevator can be 3000kg. Generally, when the load exceeds the safety When the load-bearing threshold is reached, the elevator will automatically alarm to indicate that it is overweight. You can determine whether the safety load-bearing threshold needs to be adjusted according to the vibration detection results. If the vibration detection meets the threshold adjustment condition, the above-mentioned safe load-bearing threshold can be automatically increased or decreased. The threshold adjustment conditions may include The result of the vibration detection, the result of the above vibration detection can be dangerous or safe. When the result of the vibration detection is dangerous, the safe load-bearing threshold needs to be lowered to reduce the load of the elevator. When the result of the vibration test is safe, the safe load-bearing threshold can be increased The above-mentioned increase or decrease of the safe load-bearing threshold can be increased according to a preset range, and the above-mentioned preset range can be set freely according to the situation, and no specific limitation is made here. It can be seen that this can improve the operating efficiency of the lifting equipment while ensuring the safety of the lifting equipment.

The vibration detection device 600 for lifting equipment in the embodiment of the present application will be described in detail below with reference to FIG. 6. FIG. 6 is a schematic structural diagram of a vibration detection device in an embodiment of the application, which specifically includes the following units:

The data acquisition unit 610 is configured to acquire video data including the lifting equipment;

The brightness extraction unit 620 is configured to extract brightness information of each pixel in the video frame data;

The brightness processing unit 630 is configured to perform Fourier transform on the brightness information of each pixel in the video frame data to obtain processed video frame data;

The magnification processing unit 640 is configured to use a motion magnification processing algorithm to perform motion magnification processing on the processed video frame data to obtain magnified video frame data;

The vibration acquisition unit 650 is configured to obtain the vibration data of the lifting equipment after phase correlation calculation, interpolation filtering processing, and inverse Fourier transform on the amplified video frame data, where the vibration data includes vibration amplitude and vibration frequency , Vibration phase;

The state judgment unit 660 is configured to judge whether the lifting equipment is working abnormally according to the vibration data of the lifting equipment;

The prompt sending unit 670 is configured to generate emergency prompt information and send it to the maintenance terminal when the lifting equipment works abnormally.

For specific implementations of related units in the embodiments of the present application, reference may be made to the methods described in FIG. 3 and FIG. 4, and details are not described herein again.

Hereinafter, another vibration detection device 700 for lifting equipment in an embodiment of the application will be described in detail with reference to FIG. 7. FIG. 7 is a schematic structural diagram of another vibration detection device in an embodiment of the application, which specifically includes the following units:

The data acquisition unit 710 is configured to acquire video data including the lifting equipment;

The data processing unit 720 is configured to analyze the video data to obtain vibration data of the lifting equipment;

The state determining unit 730 is configured to determine whether the lifting device is working abnormally according to the vibration data of the lifting device;

The prompt sending unit 740 is configured to generate emergency prompt information and send it to the maintenance terminal when the lifting equipment works abnormally.

In the embodiment of the present application, the lifting device includes a car body, a traction rope, and a hoist, and the data acquisition unit 710 is used to obtain video data containing the lifting device, specifically:

Obtain video frame data of the same time period including the elevator car body and the towing rope of the lifting equipment respectively.

In this embodiment of the application, the data processing unit 720 is configured to analyze the video data to obtain the vibration data of the lifting equipment, specifically:

Extracting the brightness information of each pixel in the video frame data;

Performing Fourier transform on the brightness information of each pixel in the video frame data to obtain processed video frame data;

Using a motion zoom processing algorithm to perform motion zoom processing on the processed video frame data to obtain zoomed video frame data;

The amplified video frame data is subjected to phase correlation calculation, interpolation filtering processing, and inverse Fourier transform to obtain vibration data of the lifting device, and the vibration data includes vibration amplitude, vibration frequency, and vibration phase.

Further, the motion amplification algorithm includes a motion amplification algorithm based on the Laplacian perspective, a video motion amplification algorithm based on Euler motion modulation, a video motion amplification algorithm based on Euler video modulation of complex phases, and a fast RIESZ pyramid. Any of the phase video motion amplification algorithms.

In the embodiment of the present application, the state determining unit 730 is configured to determine whether the lifting device is working abnormally according to the vibration data of the lifting device, specifically:

Acquiring current motion state data of the lifting equipment, where the motion state data includes a hovering state and an operating state;

The motion state data and the vibration data are input into a state judgment model to judge whether the lifting equipment is working abnormally, and the state judgment model includes a hovering judgment module and an operation judgment module.

Specifically, when the motion state is a hovering state, the hovering judgment module determines whether the vibration amplitude, vibration frequency, and vibration phase of the traction rope and the car body exceed the hovering vibration threshold. Hovering vibration threshold includes hovering vibration amplitude threshold, hovering vibration frequency threshold, and hovering vibration phase threshold;

If any one of the vibration amplitude, vibration frequency, and vibration phase exceeds the hovering vibration threshold, it is determined that the lifting equipment is working abnormally.

When the motion state is in operation, the operation judgment module determines whether the vibration amplitude, vibration frequency, and vibration phase of the traction rope and the car body exceed the operating vibration threshold, which includes the operating vibration threshold. Vibration amplitude threshold, operating vibration frequency threshold, operating vibration phase threshold;

If any one of the vibration amplitude, the vibration frequency, and the vibration phase exceeds the operating vibration threshold, it is determined that the lifting equipment is working abnormally.

In the embodiment of the application, an abnormality judgment unit 750 is further included, configured to judge the abnormal state type of the lifting equipment according to the state judgment model, and the abnormal state type includes a fault state and a dangerous state.

In the embodiment of the present application, the prompt sending unit 740 is configured to generate emergency prompt information and send it to the maintenance terminal when the lifting equipment works abnormally, specifically:

The emergency prompt information includes an abnormal state identifier, location information of the lifting equipment, and an alternative maintenance plan, and the abnormal state identifier includes a fault state identifier and a dangerous state identifier.

In the embodiment of the present application, a feedback obtaining unit 760 is further included, configured to obtain the feedback information of the above-mentioned maintenance terminal.

For the parts not described in detail above, reference may be made to the methods described in FIG. 3, FIG. 4, and FIG. 5, which will not be repeated here.

The structure of another server in the embodiment of the present application will be described in detail below with reference to FIG. 8. FIG. 8 is a schematic diagram of the structure of another server in the embodiment of the present application.

As shown in Fig. 8, the server 800 includes a processor 801, a communication interface 802, and a memory 803. The server 800 may also include a bus 804. The processor 801, the communication interface 802, and the memory 803 can be connected to each other via a bus 804, which can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) Bus and so on. The bus 804 can be divided into an address bus, a data bus, a control bus, and so on. For ease of presentation, only one thick line is used in FIG. 8, but it does not mean that there is only one bus or one type of bus. The memory 803 is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute all or part of the methods described in FIG. 3, FIG. 4, and FIG. 5 .

An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute all or part of the method steps in FIG. 3, FIG. 4, and FIG. 5.

The foregoing computer-readable storage medium may be an internal storage unit of the foregoing server in any of the foregoing embodiments, such as a hard disk or memory of the server. The aforementioned computer-readable storage medium may also be an external storage device of the aforementioned server, such as a plug-in hard disk equipped on the aforementioned server, a Smart Media Card (SMC), a Secure Digital (SD) card, and a flash memory card. (Flash Card) and so on. Further, the aforementioned computer-readable storage medium may also include both an internal storage unit of the aforementioned server and an external storage device. The aforementioned computer-readable storage medium is used to store the aforementioned computer program and other programs and data required by the aforementioned server. The aforementioned computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

A person of ordinary skill in the art may be aware that the units and algorithm steps of the examples described in the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both, in order to clearly illustrate the hardware and software Interchangeability, in the above description, the composition and steps of each example have been generally described in accordance with the function. Whether these functions are performed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.

Those skilled in the art can clearly understand that, for the convenience and conciseness of description, the specific working process of the server and unit described above can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed server and method may be implemented in other ways. For example, the terminal embodiments described above are merely illustrative. For example, the division of the above-mentioned units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or may be Integrate into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, terminals or units, and may also be electrical, mechanical or other forms of connection.

The units described above as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments of the present application.

In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of this application is essentially or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium. It includes a number of instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the above-mentioned methods of the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code . In the above-mentioned embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in an embodiment, reference may be made to related descriptions of other embodiments.

Claims (10)

1. A vibration detection method for lifting equipment, characterized in that the method includes:

   Acquiring video data including the lifting device, where the video data includes video frame data;

   Extracting the brightness information of each pixel in the video frame data;

   Performing Fourier transform on the brightness information of each pixel in the video frame data to obtain processed video frame data;

   Using a motion zoom processing algorithm to perform motion zoom processing on the processed video frame data to obtain zoomed video frame data;

   The amplified video frame data is subjected to phase correlation calculation, interpolation filtering processing, and inverse Fourier transform to obtain vibration data of the lifting device, where the vibration data includes vibration amplitude, vibration frequency, and vibration phase;

   Judging whether the lifting equipment is working abnormally according to the vibration data of the lifting equipment;

   If yes, generate an emergency prompt message and send it to the maintenance terminal.
2. The method according to claim 1, wherein the lifting equipment includes a car body, a traction rope, and a hoist, and the acquiring video data containing the lifting equipment includes:

   The video frame data of the same time period including the car body and the towing rope of the lifting equipment are respectively acquired.
3. The method according to claim 1, wherein the judging whether the lifting device is working abnormally according to the vibration data of the lifting device comprises:

   Acquiring current motion state data of the lifting equipment, where the motion state data includes a hovering state and an operating state;

   The motion state data and the vibration data are input into a state judgment model to judge whether the lifting equipment is working abnormally, and the state judgment model includes a hovering judgment module and an operation judgment module.
4. The method according to claim 3, wherein said inputting said motion state data and said vibration data into a state judgment model to judge whether said lifting equipment is working abnormally comprises:

   When the motion state is a hovering state, the hovering judgment module determines whether the vibration amplitude, vibration frequency, and vibration phase of the traction rope and the car body exceed the hovering vibration threshold, and the hovering vibration Thresholds include hovering vibration amplitude threshold, hovering vibration frequency threshold, and hovering vibration phase threshold;

   If any one of the vibration amplitude, vibration frequency, and vibration phase exceeds the hovering vibration threshold, it is determined that the lifting equipment is working abnormally.
5. The method according to claim 3, wherein said inputting said motion state data and said vibration data into a state judgment model to judge whether said lifting equipment is working abnormally comprises:

   When the motion state is in operation, the operation judgment module determines whether the vibration amplitude, vibration frequency, and vibration phase of the traction rope and the car body exceed the operating vibration threshold, which includes the operating vibration threshold. Vibration amplitude threshold, operating vibration frequency threshold, operating vibration phase threshold;

   If any one of the vibration amplitude, the vibration frequency, and the vibration phase exceeds the operating vibration threshold, it is determined that the lifting equipment is working abnormally.
6. The method according to claim 3, characterized in that, before generating the emergency prompt information and sending it to the maintenance terminal, the method further comprises:

   The abnormal state type of the lifting equipment is determined according to the state judgment model, and the abnormal state type includes a fault state and a dangerous state.
7. The method according to claim 6, wherein the emergency prompt information includes an abnormal state identifier, location information of the lifting equipment, and an alternative maintenance plan, and the abnormal state identifier includes a fault state identifier and a dangerous state identifier.
8. A vibration detection device for lifting equipment, characterized in that the vibration detection device comprises:

   A data acquisition unit for acquiring video data including the lifting equipment;

   A brightness extraction unit for extracting brightness information of each pixel in the video frame data;

   A brightness processing unit, configured to perform Fourier transform on the brightness information of each pixel in the video frame data to obtain processed video frame data;

   An amplification processing unit, configured to perform motion amplification processing on the processed video frame data by using a motion amplification processing algorithm to obtain amplified video frame data;

   The vibration acquisition unit is used to obtain the vibration data of the lifting device after phase correlation calculation, interpolation filtering processing, and inverse Fourier transform on the amplified video frame data, and the vibration data includes vibration amplitude, vibration frequency, Vibration phase

   The state judgment unit is configured to judge whether the lifting equipment is working abnormally according to the vibration data of the lifting equipment;

   The prompt sending unit is used to generate emergency prompt information and send it to the maintenance terminal when the lifting equipment works abnormally.
9. A server, characterized in that the server includes a processor, a communication interface, and a memory, the processor, the communication interface, and the memory are connected to each other, wherein the memory is used to store a computer program, and the computer program includes program instructions The processor is configured to call the program instructions to execute the method according to any one of claims 1-7.
10. A computer-readable storage medium, wherein the computer storage medium stores a computer program, the computer program includes program instructions, and when executed by a processor, the program instructions cause the processor to execute as claimed in claim 1. The method of any one of ~7.

Images