WO2023273219A1 - Glass curtain wall open window open state detection method and apparatus, device, and medium - Google Patents

Abstract

Disclosed in embodiments of the present application are a glass curtain wall open window open state detection method and apparatus, a device, and a medium. The method comprises: acquiring original images captured by an image acquisition device, and correcting the original images to obtain corrected images, the original images being panoramic images of the façade of a target glass curtain wall comprising all open windows; inputting the corrected images into a pre-trained target detection model to detect open windows in an open state, and according to the output of the target detection model, determining open window detection result images and the positions of target open windows in the open state; determining target room numbers corresponding to the positions of the target open windows, and sending the target room numbers and the open window detection result images to a control person side device corresponding to a building where the target glass curtain wall is located.

Description

Glass curtain wall opening window opening state detection method, device, equipment and medium

This application claims the priority of the Chinese patent application with application number 202110719434.7 submitted to the China Patent Office on June 28, 2021, and the entire content of the above application is incorporated by reference in this application.

technical field

The embodiments of the present application relate to the field of safety detection of building facilities, for example, to methods, devices, equipment and media for detecting the opening state of glass curtain wall opening windows.

Background technique

The glass curtain wall is a building envelope or decorative structure commonly used in modern high-rise buildings. In the event of strong winds or even strong typhoons, if the opening window of the glass curtain wall is not closed in time, when the wind pressure exceeds the maximum load when the opening window is opened, the opening window is connected. The hardware is extremely prone to instant damage, causing the open window to fall off at a high altitude, seriously endangering personal safety and public safety. Therefore, it is very necessary to detect the opening state of the opening window in windy or even strong typhoon weather, so as to close the opening window in time.

The opening and closing (opening or closing) state detection method of the opening window of the glass curtain wall in the related art is: when there is a strong wind, the sensor installed on each opening window senses the opening and closing state of the opening window, and through the cellular mobile communication Push the opening and closing status information of the open window to terminal devices such as mobile phones for alarm. However, for super high-rise buildings, the cost of deploying and maintaining a large number of sensor equipment is huge, and the frequent opening and closing of opening windows is likely to cause damage to the sensors, resulting in inaccurate sensing data. Or through manual inspection to detect the opening and closing status of the opening and closing windows of the glass curtain wall, so as to control it, but for a high-rise building with dozens or even hundreds of floors, it is very difficult to rely on manual inspection, and the real-time nature of control It is also difficult to guarantee.

There is no better method for detecting the opening status of glass curtain wall opening windows.

Contents of the invention

In the first aspect, the embodiment of the present application provides a method for detecting the opening state of an open window of a glass curtain wall, the method comprising:

Obtaining the original image taken by the image acquisition device, and correcting the original image to obtain a corrected image, wherein the original image is a panoramic image of the facade of the target glass curtain wall including all open windows;

The correction image is input into the pre-trained target detection model to detect the open window in the open state, and determine the position of the open window detection result image and the target open window in the open state according to the output of the target detection model;

Determine the target room number corresponding to the position of the target open window, and send the target room number and the image of the detection result of the open window to the management and control personnel side equipment corresponding to the building where the target glass curtain wall is located.

In the second aspect, the embodiment of the present application provides a device for detecting the opening state of an open window of a glass curtain wall, the device comprising:

The correction module is configured to obtain the original image taken by the image acquisition device, and correct the original image to obtain a corrected image, wherein the original image is a panoramic image of the facade of the target glass curtain wall including all open windows;

The detection module is configured to input the corrected image into the pre-trained target detection model to detect the open window in the open state, and determine the detection result image of the open window and the open target open window in the open state according to the output of the target detection model. the position of the window;

The sending module is configured to determine the target room number corresponding to the position of the target opening window, and send the target room number and the detection result image of the opening window to the management and control personnel corresponding to the building where the target glass curtain wall is located side device.

In a third aspect, an embodiment of the present application provides a computer device, which includes:

one or more processors;

a storage device configured to store one or more programs;

When the one or more programs are executed by the one or more processors, the one or more processors are made to implement the method for detecting the opening state of the glass curtain wall opening window described in any embodiment of the present application.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the method for detecting the opening state of a glass curtain wall opening window described in any embodiment of the present application is implemented.

Description of drawings

Fig. 1A is a flow chart of a method for detecting the opening state of a glass curtain wall opening window provided in Embodiment 1 of the present application;

FIG. 1B is a schematic diagram of an image of an open window detection result in the method provided in Embodiment 1 of the present application;

Fig. 1C is a schematic diagram of the correction process in the method provided in Embodiment 1 of the present application;

FIG. 1D is a schematic diagram of the process of selecting a preset number of reference points in the method provided in Embodiment 1 of the present application;

Fig. 2 is the flow chart of a kind of glass curtain wall opening window opening state detection method that the application embodiment two provides;

Fig. 3 is a schematic structural diagram of a device for detecting the opening state of a glass curtain wall opening window provided in Embodiment 3 of the present application;

FIG. 4 is a schematic structural diagram of a computer device provided in Embodiment 4 of the present application.

detailed description

The application will be described below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, only some structures related to the present application are shown in the drawings but not all structures.

Embodiment one

FIG. 1A is a flow chart of a method for detecting the opening state of a glass curtain wall opening window provided in Embodiment 1 of the present application. This embodiment can detect the opening state of a glass curtain wall opening window. The method for detecting the opening state of the opening window of the glass curtain wall provided in this embodiment can be executed by the device for detecting the opening state of the opening window of the glass curtain wall provided in the embodiment of the present application. in computer equipment.

Referring to Figure 1A, the method of this embodiment includes but is not limited to the following steps:

S110. Acquire the original image captured by the image acquisition device, and correct the original image to obtain a corrected image, wherein the original image is a panoramic image of the target glass curtain wall facade including all open windows.

Wherein, the image acquisition device can be understood as a device with an image acquisition function fixed at a certain position outside the target building, for example, a wide-angle camera or a video camera, and the position can be a first preset distance from the bottom of the target building, so that the wide-angle camera can shoot To the panoramic image of the target glass curtain wall facade including all open windows, the first preset distance can be set in advance, or it can be determined according to the specific situation, and the embodiment of the present application does not make specific limitations; the image acquisition device can also be a plurality of Camera or a plurality of cameras, each camera or camera is responsible for photographing a part of the target glass curtain wall, and the images taken by all cameras or all cameras are stitched together to form a panoramic image of the target glass curtain wall facade including all open windows. Or the deployment position of the camera is not specifically limited. The target building can be understood as the building where the target glass curtain wall is located. The target glass curtain wall can be understood as the glass curtain wall to which the open window to be detected belongs. The glass curtain wall can be understood as the building envelope or decorative structure that the supporting structure system has a certain displacement capacity relative to the main structure and does not share the effect of the main structure, and the glass curtain wall includes multiple opening windows, which can be opened or closed .

In order to detect the opening state of the opening window in windy or even strong typhoon weather, so as to close the opening window in time, it is necessary to obtain the original image taken by the image acquisition device. Since the original image may be taken by a wide-angle camera or camera deployed at the bottom of the target building, The higher the position of the target building, the farther the object distance is, and the smaller the target is, then the original image may be a distorted image, so the original image needs to be corrected, for example, it can be: Correct the original image to the normal proportion of the target glass curtain wall exterior surface image to obtain a corrected image. Wherein, the normal ratio can be understood as the ratio corresponding to the target glass curtain wall facade image without distortion or distortion. There may be multiple correction methods, for example, distortion correction based on perspective transformation technology, geometric correction, etc., which are not specifically limited in this embodiment of the present application.

S120, input the corrected image into the pre-trained target detection model to detect the open window in the open state, and determine the open window detection result image and the position of the open target open window according to the output of the target detection model.

Among them, the target detection model can be Faster-Region Convolutional Neural Networks (Faster-Region Convolutional Neural Networks, referred to as Faster-RCNN) or YOLO (You Only Look Once), etc. The target opening window can be understood as an opening window in an open state.

After obtaining the corrected image, input the corrected image into the pre-trained target detection model to detect the open windows in the open state, and according to the output of the target detection model, that is, which open windows in the corrected image are open, the open window can be determined The detection result image and the specific position of each target opening window in the open state in the correction image are convenient for subsequent determination of the target room number corresponding to the position of each target opening window, and the target room number and the opening window detection result image Send to the controller side device corresponding to the building where the target glass curtain wall is located.

In one embodiment, according to the output of the target detection model, it can also be found that there is currently no target open window, that is, all open windows are currently closed. Similarly, in this case, the image of the detection result of the open window can also be output, and the output position of the target open window in the open state is empty information, or it can be displayed as other information such as none or 0. Furthermore, the target room number corresponding to the position of the subsequent target opening window may also be empty information, or may be displayed as other information such as none or 0.

Exemplarily, FIG. 1B is a schematic diagram of an image of an open window detection result in the method provided in Embodiment 1 of the present application, and exemplarily provides an implementation manner, as shown in FIG. 1B .

In the open window detection result image shown in FIG. 1B , it can be clearly seen which target open windows are in the open state, which is convenient for subsequent management and control personnel to view.

S130. Determine the target room number corresponding to the position of the target open window, and send the target room number and the image of the detection result of the open window to the control personnel side device corresponding to the building where the target glass curtain wall is located.

Wherein, the target room number can be understood as the room number corresponding to the room corresponding to the position of the target opening window in the target building, that is, the room number to which the target opening window belongs. The management and control personnel can be understood as the personnel responsible for the management of the target building. The controller side device can be understood as the device used by the controller, such as a mobile phone or a computer.

After obtaining the position of each target opening window in the open state, it is possible to determine the target room number corresponding to the position of each target opening window, and communicate the target room number and the detection result image of the opening window through wired or wireless communication The method is sent to the management and control personnel side equipment corresponding to the building where the target glass curtain wall is located, so that the management and control personnel can dispose of or close each target opening window in the open state in time.

In this embodiment, first obtain the original image taken by the image acquisition device, and correct the original image to obtain a corrected image. The original image is a panoramic image of the facade of the target glass curtain wall including all open windows, and then input the corrected image to the pre-training The open window in the open state is detected in the target detection model of the target detection model, and the detection result image of the open window and the position of each target open window in the open state are determined according to the output of the target detection model, and finally the position of the open window corresponding to each target is determined The corresponding target room numbers, and send the target room number and the window opening detection result image to the management and control personnel side equipment corresponding to the building where the target glass curtain wall is located. Through the above-mentioned embodiment, it is possible to detect the opening state of the opening window of the glass curtain wall when the windy weather comes, and quickly locate the position of the opening window in the open state. In addition, the original image captured by the image acquisition device can be used for subsequent detection of the opening state of the opening window of the glass curtain wall. Compared with the method of installing sensors for each opening window in the related art, the cost of equipment deployment and maintenance can be reduced.

In some embodiments, the correcting the original image to obtain the corrected image may include: selecting a preset number of reference points in the original image and determining a standard reference point corresponding to each reference point ; According to the coordinates of the preset number of reference points and the coordinates of the standard reference point, determine the transformation matrix projected from the preset number of reference points to the corresponding standard reference point; The original image is corrected to obtain the corrected image.

Wherein, the preset number can be set in advance, for example, there can be 4 preset reference points, or it can be determined according to specific circumstances, and the embodiment of the present application does not make specific limitations.

For example, in order to correct the original image, select a preset number of reference points in the original image, for example, when the deployment position of the image acquisition device is fixed, select a relatively regular point as the reference point, and determine the relationship with each reference point point corresponding to the standard reference point, where the standard reference point can be determined manually, or according to the proportional relationship between the width of the target building and the width of the corrected image, and the proportional relationship between the height of the target building and the height of the corrected image. Definitely, the embodiment of the present application does not make specific limitations. After the reference point and the corresponding standard reference point are determined, according to the coordinates of the reference point and the coordinates of the standard reference point, a transformation matrix projected from each reference point to the corresponding standard reference point can be determined. The original image can be transformed into a normal scale corrected image through the transformation matrix.

In the embodiment of the present application, through the above-mentioned correction method, the original image is uniformly corrected into a normal-scale corrected image, which facilitates subsequent determination of the corresponding target room number by correcting the position of each target opening window in the open state in the corrected image; since the original image Some of the opening windows are small in size, which is not conducive to the detection of the subsequent target detection model. The normal ratio of the facade of the target glass curtain wall is restored through correction, and the size of the opening window with a small size is indirectly enlarged to highlight the characteristics of the opening window. It is conducive to improving the recognition accuracy of each target opening window in the open state; preventing the output result of the subsequent target detection model from being inaccurate due to image distortion, thereby affecting the determination of the target room number.

Exemplarily, FIG. 1C is a schematic diagram of the correction process in the method provided in Embodiment 1 of the present application, and an exemplary implementation is given, as shown in FIG. 1C:

The image on the left in Figure 1C is the original image, and the image on the right is the rectified image corrected to normal scale. Four reference points are selected from the original image, denoted by A, B, C and D respectively. A1, B1, C1 and D1 are the corresponding four standard reference points, where A1 corresponds to A, B1 corresponds to B, and C1 Corresponds to C, D1 corresponds to D. According to the coordinates of A, B, C, and D and the coordinates of A1, B1, C1, and D1, the transformation matrix M projected from the reference point to the corresponding standard reference point can be determined, and then the original image can be transformed into a normal image through the transformation matrix M Scale rectified image.

In some embodiments, the selecting a preset number of reference points in the original image may include: inputting the original image into a pre-trained semantic segmentation model to obtain two A valued image, wherein the white area in the binarized image is the area formed by the target glass curtain wall facade, and the black area is the background area; contour extraction is performed on the white area in the binarized image, Obtaining contour information corresponding to the white area; extracting edge straight line information in the contour information, and using the intersection points between the edge straight lines as a preset number of reference points selected in the original image.

Wherein, the semantic segmentation model may be a fully convolutional neural network (Fully Convolutional Networks, referred to as FCN), U-Net or DeepLab, etc., which are not specifically limited in the embodiment of the present application. The black area can be understood as the area formed by the part of the original image that does not contain the facade of the target glass curtain wall.

For example, when there is a slight change in the position of the image acquisition device, the original image is input into the pre-trained semantic segmentation model, and a binary image corresponding to the original image can be obtained, and the outline of the white area in the binary image can be extracted , such as the Canny operator extraction method, or other extraction methods, which can obtain the contour information corresponding to the white area. The edge straight line information in the contour information can be extracted by using the Hough transform method, or other transform methods, which are not specifically limited in this embodiment of the present application. Finally, the intersection points between the edge straight lines are used as the reference points of the preset number selected in the original image.

Exemplarily, FIG. 1D is a schematic diagram of the process of selecting a preset number of reference points in the method provided in Embodiment 1 of the present application, and an exemplary implementation is given, as shown in FIG. 1D:

The first picture from left to right in Figure 1D is the original image, the second picture is the binarized image corresponding to the original image, the third picture is the image obtained after contour extraction, and the fourth picture is the extracted contour The image obtained after the edge straight line information in the information, the intersection points A2, B2, C2 and D2 between the edge straight lines in the fourth figure are the four reference points selected in the original image.

In the embodiment of the present application, the extraction of reference points through the above method based on deep learning semantic segmentation can still stably extract the position information of reference points when the image acquisition device has a slight position change, and at the same time can effectively exclude non-target glass curtain wall facades background information to improve the recognition accuracy of each target opening window that is in the open state.

In some embodiments, the semantic segmentation model can be obtained through the following training methods:

1. Data preparation: Use image acquisition equipment to look up from the bottom of the target building to shoot the facade of the target glass curtain wall to obtain panoramic image data. For example, the collected panorama image of the facade of the glass curtain wall should include the opened window as the training data sample of the subsequent semantic segmentation model; Panoramic image data under conditions.

Carry out data annotation on multiple panoramic image data collected, for example, artificially annotate the area and background area formed by the facade of the target glass curtain wall in the original image, and form a training set and a verification set based on the annotated data, the training set As training samples, the validation set is used as validation samples. The marked image may be a binarized mask image, the gray value corresponding to the area formed by the facade of the target glass curtain wall in the marked image may be 255, and the gray value corresponding to the background area may be 0.

2. Data enhancement: In the process of semantic segmentation model training, online data enhancement can be used to enrich training data samples. Among them, the ways of data enhancement may include but not limited to the following: scale change, random occlusion, perspective transformation, random rotation and horizontal flip, adding Gaussian noise and color change, etc.

3. Model training: During the training process, when the accuracy between the training results of the samples contained in the training set and the corresponding samples in the verification set reaches the preset threshold, it means that the verification results meet the end conditions. At this time, the training ends and the current model is saved. The trained parameters are used as the parameters of the semantic segmentation model. Wherein, the preset threshold may be set in advance, for example, 90%, or it may be determined according to specific circumstances, which is not specifically limited in this embodiment of the present application.

In the embodiment of the present application, the semantic segmentation model is obtained through the above training method, on the one hand, it can improve the accuracy of the model recognition result, and on the other hand, it is beneficial to obtain a more accurate transformation matrix later.

In some embodiments, determining the target room number corresponding to the position of each target opening window may include: inquiring about the preset correspondence between the positions of all opening windows on the facade of the target glass curtain wall and the room numbers ; Determine the target room number corresponding to the position of each target opening window according to the preset corresponding relationship.

For example, in order to facilitate the management and control personnel to quickly deal with the phenomenon of opening windows in windy weather, the corresponding room number can be found according to the position of each target opening window in the open state in the correction image. The position of the open window (including all open windows in the open and closed states) and the preset correspondence relationship with the room number, and then query the preset correspondence relationship between the position of all open windows on the facade of the target glass curtain wall and the room number, according to the preset The corresponding relationship can determine the target room number corresponding to the position of each target opening window.

In the embodiment of the present application, through the above query method, the target room number corresponding to the position of each target opening window can be quickly found, so that the management and control personnel can respond in time to prevent the window from being opened due to strong winds or even strong typhoons. If it is not closed in time, the open window will fall off at a high altitude, endangering personal safety and public safety, and causing unnecessary losses.

Embodiment two

FIG. 2 is a flow chart of a method for detecting the opening state of an open window in a glass curtain wall provided in Embodiment 2 of the present application. The embodiments of the present application are adjusted on the basis of the foregoing embodiments. This embodiment can explain the process of inputting the corrected image into the pre-trained target detection model to detect the open window in the open state, and determining the detection result image of the open window and the position of the target open window in the open state.

Referring to Fig. 2, the method of the present embodiment includes but not limited to the following steps:

S210. Acquire the original image captured by the image acquisition device, and correct the original image to obtain a corrected image, wherein the original image is a panoramic image of the exterior facade of the target glass curtain wall including all open windows.

S220, scanning and cropping the corrected image according to a window of a preset size and a preset step size to obtain a cropped image.

Wherein, the values corresponding to the preset size and the preset step are integer multiples of the maximum side length of the open window, and the value corresponding to the preset step is smaller than the value corresponding to the preset size.

For example, since the size of the corrected image corresponding to the target building may be large, if the corrected image of the original size is directly input to the target detection model for detection, the hardware conditions may be difficult to support, and the method of reducing the image is easy to lose key feature information in the corrected image , has an impact on the accuracy of the target detection model. Therefore, in the embodiment of the present application, the cropped image can be obtained by scanning and cropping the corrected image according to a window of a preset size, such as a window of n*n, and a preset step size, so that all the cropped images can be divided into Input to the pre-trained object detection model to detect the open window in the open state.

In the embodiment of the present application, the above cropping method can not only ensure that each open window can be completely included in the cropped image, but also relieve the pressure on the hardware.

In one embodiment, the window with a predetermined size may be a square.

In one embodiment, by setting the value corresponding to the preset step length to be smaller than the value corresponding to the preset size, the window can be cropped by overlapping scanned images, ensuring that each open window can be completely cropped out, and avoiding opening the window It may be partially cropped, which will affect the subsequent recognition accuracy.

In one embodiment, the predetermined size may be a side length of the window.

S230. Input all the cropped images into the pre-trained target detection model to detect the open window in the open state, and obtain the first open window detection result image corresponding to each cropped image and each cropped image. The first position coordinates of the rectangular detection frame corresponding to each first target opening window in the open state in the image of .

Wherein, the first open window detection result image can be understood as an image obtained by marking open windows in an open state with a rectangular detection frame in a cropped image. The size of the rectangular detection frame can be a rectangular frame that just frames the open window, for example, the size of the rectangular detection frame can be slightly larger than the size of the open window, or slightly smaller than the size of the open window. The first position coordinates may be the position coordinates of the center point of the rectangular detection frame corresponding to each first target opening window, or the position coordinates of other vertices. When determining the coordinates of the first position, the origin may be selected from the vertex in the upper left corner of a corresponding cropped image, or other points, which are not specifically limited in this embodiment of the present application.

Input all the cropped images into the pre-trained target detection model to detect the open window in the open state, and the first open window detection result image corresponding to each cropped image can be obtained, and by establishing the coordinate system , it is possible to determine the first position coordinates of the rectangular detection frame corresponding to each first target open window in the open state in each cropped image, for example, the center of the rectangular detection frame corresponding to each first target open window Point position coordinates, for example, can use (cxi, cyi, wi, hi) to represent a certain first position coordinates, wherein, cxi is the abscissa of the first position coordinates, cyi is the ordinate of the first position coordinates, wi is the width of the rectangular detection frame corresponding to the first position coordinates, hi is the height of the rectangular detection frame corresponding to the first position coordinates, so that all the first The image of the detection result of the open window is mapped to the corrected image to obtain the image of the detection result of the open window.

In some embodiments, the target detection model can be obtained through the following training methods:

1. Data preparation: scan and crop multiple corrected images according to the preset size of the window and the preset step size to obtain the cropped image; perform data labeling on each cropped image, and the labeling result can be just framed to open the window The rectangular frame can be represented by (cx, cy, w, h), where (cx, cy) is the coordinate of the center point of the rectangular frame, cx is the abscissa of the center point of the rectangular frame, and cy is the ordinate of the center point of the rectangular frame. w is the width of the rectangular frame, h is the height of the rectangular frame, divide the marked image into a training set and a verification set, the training set is used as a training sample, and the verification set is used as a verification sample.

For example, the window is a square, the preset size is the side length of the square, the preset step size is the sliding distance of the window (in pixels), and the window can be slid horizontally and vertically.

2. Data enhancement: In the process of target detection model training, online data enhancement can be used to enrich training data samples. Among them, the methods of data enhancement can include but not limited to the following: scale change, perspective transformation, random rotation and horizontal flip, histogram equalization, Gaussian noise, hue saturation brightness (Hue, Saturation, Value, referred to as HSV) Spatial color transformation and random occlusion, etc.

Data enhancement, on the one hand, can increase the amount of training data and improve the generalization ability of the model. On the other hand, noise data can be added to improve the robustness of the model.

3. Model training: During the training process, when the accuracy between the training results of the samples contained in the training set and the corresponding samples in the verification set reaches the preset threshold, it means that the verification results meet the end conditions. At this time, the training ends and the current model is saved. The trained parameters are used as the parameters of the object detection model. Wherein, the preset threshold may be set in advance, for example, 92%, or it may be determined according to specific circumstances, which is not specifically limited in this embodiment of the present application.

S240. Based on the position of each cropped image relative to the corrected image, map all first open window detection result images to the corrected image to obtain an open window detection result image.

Since each cropped image is obtained by scanning and cropping the corrected image, based on the position of each cropped image relative to the corrected image, all the first window detection result images are mapped to the corrected image, and each The first open window detection result images respectively correspond to the initial images, and at least one initial image is spliced according to the corresponding position to obtain the open window detection result image.

S250. Based on the position of each cropped image relative to the corrected image, map all the rectangular detection frames with the first position coordinates to the corrected image to obtain the position coordinates of the center point of the rectangular detection frame in the corrected image.

Based on the position of each cropped image relative to the corrected image, all the rectangular detection frames of the first position coordinates are mapped to the corrected image, and the position coordinates of the center points of multiple rectangular detection frames in the corrected image can be obtained, for example, Suppose (cxi, cyi, wi, hi) represent the coordinates of a first position, (cxj, cyj, wj, hj) are the coordinates of the center point of the first position coordinates mapped in the corrected image, and cxj is the position of the center point The abscissa of , cyj is the ordinate of the center point position, wj is the width of the rectangular detection frame corresponding to the first position coordinate, hj is the height of the rectangular detection frame corresponding to the first position coordinate.

S260. Determine the position of each target opening window according to the position coordinates of the center point.

After obtaining the center point position coordinates of all the first position coordinates of the rectangular detection frame in the corrected image, the position of the target opening window corresponding to each center point position coordinate can be determined according to each center point position coordinate, This facilitates the subsequent determination of the target room number corresponding to the position of each target open window, and sends the target room number and the detection result image of the open window to the management and control personnel side equipment corresponding to the building where the target glass curtain wall is located.

In some embodiments, after obtaining the center point position coordinates of the rectangular detection frame in the corrected image, it may further include: verifying the center point position coordinates, if the center point position coordinates correspond to the first One abscissa is greater than the second abscissa and less than the third abscissa, and the first ordinate corresponding to the center point position coordinate is greater than the second ordinate and less than the third ordinate, then the verification result is passed, wherein, The second abscissa and the third abscissa are selected from the abscissas corresponding to all vertices in the corrected image of the target open window corresponding to the center point position coordinates, the second ordinate and the The third ordinate is selected from the ordinates corresponding to the target opening window corresponding to the center point position coordinates in the corrected image; when the verification result is passed, obtain the coordinates corresponding to the center point position coordinates The target room number corresponding to the target opening window of .

For example, since the second abscissa and the third abscissa are selected from the abscissa corresponding to all vertices in the corrected image from the target opening window corresponding to the center point position coordinates, then assuming that the origin is the vertex in the upper left corner of the corrected image, the target can be opened The abscissa corresponding to the upper left vertex of the window in the corrected image is determined as the second abscissa, which is assumed to be represented by x1; the abscissa corresponding to the lower right vertex of the target opening window in the corrected image can be determined as the third abscissa, assuming Expressed by x2. Since the second ordinate and the third ordinate are selected from the ordinates corresponding to all vertices in the corrected image from the target open window corresponding to the center point position coordinates, then assuming that the origin is the vertex in the upper left corner of the corrected image, the target open window can be set at The ordinate corresponding to the vertex in the upper left corner in the corrected image is determined as the second ordinate, which is assumed to be represented by y1; the ordinate corresponding to the vertex in the lower right corner of the target opening window in the corrected image can be determined as the third ordinate, which is assumed to be y2 express.

After obtaining the position coordinates of the center point of the rectangular detection frame in the corrected image, verify the position coordinates of the center point, assuming that the position coordinates of the center point are (cxj, cyj, wj, hj), and the origin is the vertex in the upper left corner of the corrected image, If the first abscissa cxj corresponding to the position coordinates of the center point is greater than the second abscissa x1 and smaller than the third abscissa x2, that is: x1<cxj<x2, the first ordinate cyj corresponding to the center point position coordinates is greater than the second If the vertical coordinate y1 is smaller than the third vertical coordinate y2, namely: y1<cyj<y2, then the verification result is passed. When the verification result is passed, the target room number corresponding to the target opening window corresponding to the position coordinates of the center point is obtained.

In the embodiment of the present application, the above method is used to verify the position coordinates of the center point, which can ensure the accuracy of the position coordinates of the center point, avoid mistakes, and improve the accuracy rate.

In some embodiments, after obtaining the position coordinates of the center point of the rectangular detection frame in the corrected image, the range parameter of the rectangular detection frame can also be directly bound to the corresponding target room number, so that the target room number can be determined more quickly , and send the target room number and the image of the detection result of the open window to the management and control personnel side equipment corresponding to the building where the target glass curtain wall is located.

Among them, the range parameters of the rectangular detection frame can be represented by (x0, y0, x3, y3), x0 is the abscissa coordinate of the top left corner of the rectangle detection frame, y0 is the vertical coordinate of the top left corner of the rectangle detection frame, x3 is the right side of the rectangle detection frame The abscissa of the vertex of the lower corner, y3 is the ordinate of the vertex of the lower right corner of the rectangular detection frame.

S270. Determine the target room number corresponding to the position of the target open window, and send the target room number and the image of the detection result of the open window to the management and control personnel side device corresponding to the building where the target glass curtain wall is located.

In this embodiment, the original image captured by the image acquisition device is first obtained, and the original image is corrected to obtain a corrected image, and then the corrected image is scanned and cropped according to a preset window size and a preset step size to obtain a cropped image, and the All cropped images are input to the pre-trained target detection model to detect the open window in the open state, and the first open window detection result image corresponding to each cropped image and each cropped image are obtained. The first position coordinates of the rectangular detection frame corresponding to each first target open window in the open state, and then based on the position of each cropped image relative to the corrected image, map all the first open window detection result images to the corrected In the image, the image of the detection result of the open window is obtained, and based on the position of each cropped image relative to the corrected image, all the rectangular detection frames of the first position coordinates are mapped to the corrected image, and the center of the rectangular detection frame in the corrected image is obtained Point position coordinates, determine the position of each target opening window according to the center point position coordinates, and finally determine the target room number corresponding to the position of each target opening window, and send the target room number and the opening window detection result image to the target For the management and control personnel side equipment corresponding to the building where the glass curtain wall is located, through the above embodiment, the corrected image is cropped and then input to the pre-trained target detection model for detection, which not only ensures that each open window can be completely included in the cropped In the image, it can relieve the pressure on the hardware, improve the accuracy of the position of each target opening window determined subsequently, and can detect the opening state of the opening window when the windy weather comes, and quickly locate the opening window in the open state s position.

Embodiment Three

Fig. 3 is a schematic structural diagram of a device for detecting the opening state of a glass curtain wall opening window provided in Embodiment 3 of the present application. As shown in Fig. 3 , the device may include:

The correction module 310 is configured to obtain the original image taken by the image acquisition device, and correct the original image to obtain a corrected image, wherein the original image is a panoramic image of the target glass curtain wall facade including all open windows;

The detection module 320 is configured to input the corrected image into the pre-trained target detection model to detect the open window in the open state, and determine the detection result image of the open window and the target in the open state according to the output of the target detection model the position of the opening window;

The sending module 330 is configured to determine the target room number corresponding to the position of the target open window, and send the target room number and the image of the detection result of the open window to the control room corresponding to the building where the target glass curtain wall is located. Equipment on the personnel side.

In this embodiment, first obtain the original image taken by the image acquisition device, and correct the original image to obtain a corrected image. The original image is a panoramic image of the facade of the target glass curtain wall including all open windows, and then input the corrected image to the pre-training The open window in the open state is detected in the target detection model of the target detection model, and the detection result image of the open window and the position of the open window of the target in the open state are determined according to the output of the target detection model, and finally the target corresponding to the position of the target open window is determined Room number, and send the target room number and the image of the detection result of the open window to the management and control personnel side equipment corresponding to the building where the target glass curtain wall is located. Through the above-mentioned embodiment, it is possible to detect the opening state of the opening window of the glass curtain wall when the windy weather comes, and quickly locate the position of the opening window in the open state.

In some embodiments, the above-mentioned correction module 310 may include: a selection unit, configured to acquire the original image taken by the image acquisition device, and select a preset number of reference points in the original image and determine the number of reference points corresponding to the reference points the standard reference point; the first determination unit is set to determine the transformation matrix projected from the reference point to the corresponding standard reference point according to the coordinates of the reference point and the coordinates of the standard reference point; the first correction unit is set to The original image is corrected based on the transformation matrix to obtain a corrected image, wherein the original image is a panoramic image of a target glass curtain wall facade including all open windows.

In some embodiments, the above selection unit can be configured to: acquire the original image taken by the image acquisition device, and input the original image into the pre-trained semantic segmentation model to obtain the binarization corresponding to the original image image, wherein the white area in the binarized image is the area formed by the target glass curtain wall facade, and the black area is the background area; the white area in the binarized image is contour extracted to obtain the The contour information corresponding to the white area; extract the information of the edge straight lines in the contour information, and use the intersection points between the edge straight lines as the reference points of the preset number selected in the original image and determine the relationship with each The reference point corresponds to the standard reference point.

In some embodiments, the detection module 320 may include: a first detection unit and a second determination unit.

The first detection unit may be configured to: scan and crop the corrected image according to a window of a preset size and a preset step size to obtain a trimmed image, wherein the preset size and the preset step size The corresponding values are all integer multiples of the maximum side length of the open window, and the value corresponding to the preset step size is smaller than the value corresponding to the preset size; input all the cropped images into the pre-trained target detection model respectively for The open window in the open state is detected, and the first open window detection result image corresponding to each cropped image and the rectangular detection frame corresponding to the first target open window in the open state in each cropped image are obtained. The coordinates of the first location.

The second determination unit may be configured to: based on the position of each cropped image relative to the corrected image, map all the first window opening detection result images to the corrected image to obtain the open window detection Result image; based on the position of each cropped image relative to the corrected image, map the rectangular detection frame of all first position coordinates to the corrected image, and obtain the rectangular detection frame in the corrected image The position coordinates of the center point in ; determine the position of the target opening window according to the position coordinates of the center point.

In some embodiments, the above-mentioned device for detecting the opening state of the opening window of the glass curtain wall may further include: a verification module, configured to, after obtaining the position coordinates of the center point of the rectangular detection frame in the correction image, verify the center point Position coordinates are verified, if the first abscissa corresponding to the center point position coordinates is greater than the second abscissa and smaller than the third abscissa, the first ordinate corresponding to the center point position coordinates is greater than the second ordinate, and If it is smaller than the third ordinate, it means that the verification result is passed, wherein, the second abscissa and the third abscissa correspond to all vertices in the corrected image of the target opening window corresponding to the center point position coordinates selected from the abscissa, the second ordinate and the third ordinate are selected from the ordinates corresponding to all vertices in the corrected image of the target opening window corresponding to the position coordinates of the center point; correspondingly, The above-mentioned sending module 330 can be set to: when the verification result is passed, obtain the target room number corresponding to the target opening window corresponding to the position coordinates of the center point, and combine the target room number and the opening window The detection result image is sent to the control personnel side equipment corresponding to the building where the target glass curtain wall is located.

In some embodiments, the above-mentioned sending module 330 can also be configured to: inquire about the preset correspondence between the positions of all open windows on the facade of the target glass curtain wall and the room numbers; The target room number corresponding to the position of the target opening window, and the target room number and the detection result image of the opening window are sent to the management and control personnel side equipment corresponding to the building where the target glass curtain wall is located.

The device for detecting the opening state of the opening window of the glass curtain wall provided in this embodiment can be applied to the method for detecting the opening state of the opening window of the glass curtain wall provided in any of the above embodiments, and has corresponding functions and beneficial effects.

Embodiment four

FIG. 4 is a schematic structural diagram of a computer device provided in Embodiment 4 of the present application. As shown in FIG. 4 , the computer device includes a processor 410 and a storage device 420; the number of processors 410 in the computer device can be one or more In FIG. 4, a processor 410 is taken as an example; the processor 410 and the storage device 420 in the computer device may be connected through a bus or in other ways. In FIG. 4, the connection through a bus is taken as an example.

The storage device 420, as a computer-readable storage medium, can be used to store software programs, computer-executable programs and modules, such as the module corresponding to the method for detecting the opening state of glass curtain wall opening windows in the embodiment of the present application (for example, for glass curtain wall The correction module 310, the detection module 320 and the sending module 330 in the window open state detection device). The processor 410 executes various functional applications and data processing of the computer equipment by running the software programs, instructions and modules stored in the storage device 420 , that is, realizes the above-mentioned method for detecting the opening state of the open window of the glass curtain wall.

The storage device 420 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application program required by at least one function; the data storage area may store data created according to the use of the terminal, and the like. In addition, the storage device 420 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage devices. In some examples, the storage device 420 may further include memories that are located remotely relative to the processor 410, and these remote memories may be connected to computer equipment through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

A computer device provided in this embodiment can be used to execute the method for detecting the opening state of an open window of a glass curtain wall provided in any of the above embodiments, and has corresponding functions and beneficial effects.

Embodiment five

Embodiment 5 of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the method for detecting the opening state of a glass curtain wall opening window in any embodiment of the present application is implemented. The method includes :

Obtaining the original image taken by the image acquisition device, and correcting the original image to obtain a corrected image, wherein the original image is a panoramic image of the facade of the target glass curtain wall including all open windows;

The correction image is input into the pre-trained target detection model to detect the open window in the open state, and determine the position of the open window detection result image and the target open window in the open state according to the output of the target detection model;

Determine the target room number corresponding to the position of the target open window, and send the target room number and the image of the detection result of the open window to the management and control personnel side equipment corresponding to the building where the target glass curtain wall is located.

A storage medium containing computer-executable instructions provided in an embodiment of the present application, the computer-executable instructions are not limited to the method operations described above, and can also perform the detection of the opening state of the glass curtain wall opening window provided in any embodiment of the present application Related operations in the method.

The storage medium may be a non-transitory storage medium.

Through the above description about the implementation, those skilled in the art can clearly understand that the present application can be realized by means of software and necessary general-purpose hardware, and of course it can also be realized by hardware, but in many cases the former is a better implementation . Based on this understanding, the essence of the technical solution of this application or the part that contributes to related technologies can be embodied in the form of software products, and the computer software products can be stored in computer-readable storage media, such as computer floppy disks, Read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), flash memory (FLASH), hard disk or optical disc, etc., including several instructions to make a computer device (which can be a personal computer, A server or a network device, etc.) executes the methods described in multiple embodiments of the present application.

It is worth noting that in the above-mentioned embodiment of the detection device for the opening state of the opening window of the glass curtain wall, the various units and modules included are only divided according to the functional logic, but are not limited to the above-mentioned divisions, as long as the corresponding functions can be realized. Yes; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the present application.

The above descriptions are only part of the embodiments of the present application, and are not intended to limit the present application. For those skilled in the art, various modifications and changes may be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the concept of the present invention shall be included within the protection scope of the present application.

Claims (14)

1. A method for detecting the opening state of an opening window of a glass curtain wall, comprising:

   Obtaining the original image taken by the image acquisition device, and correcting the original image to obtain a corrected image, wherein the original image is a panoramic image of the facade of the target glass curtain wall including all open windows;

   The corrected image is input into the pre-trained target detection model to detect the open window in the open state, and the position of the open window detection result image and the open target window in the open state are determined according to the output of the target detection model;

   Determine the target room number corresponding to the position of the target open window, and send the target room number and the image of the detection result of the open window to the management and control personnel side equipment corresponding to the building where the target glass curtain wall is located.
2. The method according to claim 1, wherein said correcting said original image to obtain a corrected image comprises:

   selecting a preset number of reference points in the original image and determining standard reference points corresponding to the reference points;

   determining a transformation matrix projected from the reference point to the corresponding standard reference point according to the coordinates of the reference point and the coordinates of the standard reference point;

   Correcting the original image based on the transformation matrix to obtain a corrected image.
3. The method according to claim 2, wherein said selecting a preset number of reference points in said original image comprises:

   Inputting the original image into a pre-trained semantic segmentation model to obtain a binarized image corresponding to the original image, wherein the white area in the binarized image is formed by the facade of the target glass curtain wall area, the black area is the background area;

   performing contour extraction on the white area in the binarized image to obtain contour information corresponding to the white area;

   Extracting information of a plurality of edge straight lines in the contour information, and using intersection points among the plurality of edge straight lines as a preset number of reference points selected in the original image.
4. The method according to claim 1, wherein said inputting said corrected image into a pre-trained target detection model to detect an open window in an open state comprises:

   The corrected image is scanned and cropped according to a window of a preset size and a preset step to obtain a cropped image, wherein the values corresponding to the preset size and the preset step are respectively the maximum side length of the open window an integer multiple, the value corresponding to the preset step size is smaller than the value corresponding to the preset size;

   Input all the cropped images into the pre-trained target detection model to detect the open window in the open state, and obtain the first open window detection result image corresponding to each cropped image and each cropped image The first position coordinates of the rectangular detection frame corresponding to the first target opening window in the open state.
5. The method according to claim 4, wherein said determining the open window detection result image and the position of the open target open window according to the output of the target detection model comprises:

   Based on the position of each cropped image relative to the corrected image, mapping all first window-opened detection result images to the corrected image to obtain an open-window detection result image;

   Based on the position of each cropped image relative to the corrected image, map the rectangular detection frames of all the first position coordinates to the corrected image, and obtain the rectangular detection frames of all the first position coordinates in the corrected image. The position coordinates of the center point in the image;

   The position of the target opening window is determined according to the position coordinates of the center point.
6. The method according to claim 5, after obtaining the center point position coordinates of the rectangular detection frames of all first position coordinates in the corrected image, further comprising:

   Verifying the position coordinates of each center point, in response to the first abscissa corresponding to the position coordinates of each center point being greater than the second abscissa and smaller than the third abscissa, the first ordinate corresponding to the position coordinates of the center point The coordinate is greater than the second ordinate and less than the third ordinate, and the verification result is passed, wherein the second abscissa and the third abscissa are from the target opening window corresponding to each center point position coordinate selected from the abscissas corresponding to all vertices in the correction image, the second ordinate and the third ordinate are all vertices in the correction image in the target opening window corresponding to the position coordinates of each center point Select from the corresponding ordinate;

   In response to the verification result being passed, the target room number corresponding to the target opening window corresponding to the position coordinates of the center point is acquired.
7. The method according to any one of claims 1-6, wherein said determining the target room number corresponding to the position of the target opening window comprises:

   Query the preset corresponding relationship between the positions of all open windows on the facade of the target glass curtain wall and the room number;

   The target room number corresponding to the position of the target opening window is determined according to the preset corresponding relationship.
8. A device for detecting the opening state of an opening window of a glass curtain wall, comprising:

   The correction module is configured to obtain the original image taken by the image acquisition device, and correct the original image to obtain a corrected image, wherein the original image is a panoramic image of the facade of the target glass curtain wall including all open windows;

   The detection module is configured to input the corrected image into the pre-trained target detection model to detect the open window in the open state, and determine the detection result image of the open window and the open target open window in the open state according to the output of the target detection model. the position of the window;

   The sending module is configured to determine the target room number corresponding to the position of the target opening window, and send the target room number and the detection result image of the opening window to the management and control personnel corresponding to the building where the target glass curtain wall is located side device.
9. The device according to claim 8, wherein the correction module comprises:

   The selection unit is configured to obtain the original image taken by the image acquisition device, and select a preset number of reference points in the original image and determine a standard reference point corresponding to the reference point;

   The first determination unit is configured to determine a transformation matrix projected from the reference point to the corresponding standard reference point according to the coordinates of the reference point and the coordinates of the standard reference point;

   The first correction unit is configured to correct the original image based on the transformation matrix to obtain a corrected image, wherein the original image is a panoramic image of a target glass curtain wall facade including all open windows.
10. The device according to claim 9, wherein the selection unit is configured to obtain the original image taken by the image acquisition device, and input the original image into the pre-trained semantic segmentation model to obtain The binarized image, wherein, the white area in the binarized image is the area formed by the facade of the target glass curtain wall, and the black area is the background area; outline the white area in the binarized image Extracting to obtain the contour information corresponding to the white area; extracting the information of a plurality of edge straight lines in the contour information, and using the intersection points between the plurality of edge straight lines as the preset number selected in the original image and determine the standard reference points corresponding to each reference point.
11. The device according to claim 8, wherein the detection module comprises: a first detection unit;

    The first detection unit is configured to scan and crop the corrected image according to a window of a preset size and a preset step to obtain a cropped image, wherein the preset size and the preset step correspond to The values are integer multiples of the maximum side length of the open window, and the value corresponding to the preset step size is smaller than the value corresponding to the preset size; input all the cropped images into the pre-trained target detection model respectively for those in the open The open window of the state is detected, and the first open window detection result image corresponding to each cropped image and the first rectangle detection frame corresponding to the first target open window in the open state in each cropped image are obtained. Position coordinates.
12. The device according to claim 11, wherein the detection module comprises: a second detection unit;

    The second detection unit is configured to map all first open window detection result images to the corrected image based on the position of each cropped image relative to the corrected image, to obtain an open window detection result image ; Based on the position of each cropped image relative to the corrected image, map the rectangular detection frames of all first position coordinates to the corrected image, and obtain the rectangular detection frames of all first position coordinates in the Correcting the position coordinates of the center point in the image; determining the position of the target opening window according to the position coordinates of the center point.
13. A computer device comprising:

    one or more processors;

    a storage device configured to store one or more programs;

    When the one or more programs are executed by the one or more processors, the one or more processors are made to implement the method for detecting the opening state of the glass curtain wall opening window according to any one of claims 1-7.
14. A computer-readable storage medium, the computer-readable storage medium is stored with a computer program, and when the computer program is executed by a processor, the detection of the opening state of the glass curtain wall opening window according to any one of claims 1-7 is realized method.

Images