CN112449147A

CN112449147A - Video cluster monitoring system of photovoltaic power station and image processing method thereof

Info

Publication number: CN112449147A
Application number: CN201910807052.2A
Authority: CN
Inventors: 王达
Original assignee: Beijing Etechwin Electric Co Ltd
Current assignee: Beijing Etechwin Electric Co Ltd
Priority date: 2019-08-29
Filing date: 2019-08-29
Publication date: 2021-03-05
Anticipated expiration: 2039-08-29
Also published as: CN112449147B

Abstract

The video cluster monitoring system of the photovoltaic power station comprises a monitoring terminal and a plurality of photovoltaic power stations, wherein a plurality of camera devices are arranged in each photovoltaic power station, and the following operations are executed aiming at each camera device: acquiring a monitoring image of a preset monitored scene captured by any one camera device; determining a difference image of the captured monitoring image and a reference image of any one of the image pickup devices; determining at least one foreground sub-image based on the determined difference image; determining a similarity index of the captured monitoring image and the reference image based on the at least one foreground subgraph; and if the similarity index is smaller than the similarity threshold, sending the captured data information of the monitoring image to the monitoring terminal so as to present the monitoring image on the monitoring terminal. By adopting the photovoltaic power station video cluster monitoring system and the image processing method thereof, operation and maintenance personnel can remotely and timely know the conditions of each photovoltaic power station through monitoring images.

Description

Video cluster monitoring system of photovoltaic power station and image processing method thereof

Technical Field

The present invention generally relates to the field of image processing technologies, and in particular, to a video cluster monitoring system for a photovoltaic power station and an image processing method thereof.

Background

Solar energy is used as a novel energy source and has the characteristics of wide source, safety, reliability, no pollution and the like, so that the investment obtained by the construction of photovoltaic power stations is continuously increased, and the number of the built and put into use photovoltaic power stations shows a rapid growth trend. In order to know the production and operation conditions of the photovoltaic power station in time, the photovoltaic power station is usually provided with a local monitoring system to monitor, store, analyze and display the operating parameters and environmental parameters of the power station. Typical power plant operating parameters include the output voltage of the photovoltaic array, the input-output voltage, the input-output current and the output power of the inverter, the accumulated power generation of the power plant, and the like, while environmental parameters include the ambient temperature, the horizontal radiation intensity, the oblique radiation intensity, and the like. And the operation and maintenance personnel perform corresponding processing according to the change of the data on the local monitoring system, such as sending a control instruction or modifying various states and parameters and the like through the local monitoring system. With the increase of unattended or unattended demands of photovoltaic power stations, more and more photovoltaic power stations are provided with video monitoring systems, so that the functions of security protection or fire fighting are realized, and theft behaviors, fire accidents and the like in the photovoltaic power stations are monitored and recorded.

With the increase of the number of photovoltaic power stations controlled and managed by photovoltaic power station owners or professional operation and maintenance companies, the complexity and difficulty of operation and maintenance work of the photovoltaic power stations are obviously increased, and the management of a plurality of photovoltaic power stations distributed in various places at high efficiency and low cost becomes an important problem to be solved in the photovoltaic field.

The remote monitoring system based on the computer and the network technology provides a feasible way for the management of a plurality of photovoltaic power stations distributed in various places, supports the remote transmission and display of the operation data of each photovoltaic power station, and can store the historical data of the photovoltaic power stations for statistical analysis, accident prediction and the like. Through the transverse comparison of the operation conditions of a plurality of photovoltaic power stations, the defects and the defects of the original photovoltaic power station design are found, and the full scientific basis can be provided for the transformation of the existing photovoltaic power station and the design of the future photovoltaic power station.

However, the existing photovoltaic power station remote monitoring system almost only displays various operation data transmitted from a photovoltaic power station plant area in a remote mode, and does not have monitoring images about the photovoltaic power station plant area. The monitoring image stored by the local monitoring system is only used for operation and maintenance personnel to go to a photovoltaic power station factory for investigation when needed. The monitoring image information in the plant area of the photovoltaic power station is not fully utilized, the operation and maintenance personnel are difficult to comprehensively master the actual condition of the photovoltaic power station, and the remote management of the operation and maintenance personnel on the photovoltaic power station is weakened.

The remote transmission of monitoring images by a photovoltaic power station remote monitoring system has the following technical difficulties: firstly, the number of camera devices in a photovoltaic power station is large, and the amount of generated data is large. There may be tens of thousands of photovoltaic modules in large and medium-sized centralized photovoltaic power stations, and the floor area can reach several square kilometers, and it is generally necessary to adopt dozens of cameras to realize the full coverage of the key area of the photovoltaic power station. For a distributed photovoltaic power station, although the number of photovoltaic modules is small, the layout of the photovoltaic modules is scattered, so that the number of used image pickup devices is large. When all the monitoring images collected by all the camera devices of the photovoltaic power stations are transmitted to a computer where a remote monitoring system is located, great burden is caused on network transmission and computer processing. Secondly, in the case of a certain size of the monitored image capacity, there is a contradiction between the degree of clarity of image display and the rate of image update. Clear images are displayed on a remote monitoring system, and the monitoring images are subjected to fine sampling to embody rich details, which can prevent the transmission and display of other images. On the contrary, pursuing fast image updating rate will affect the definition of image display and the judgment of operation and maintenance personnel on the actual situation in the power station plant area.

At present, in order to enable the monitoring images collected by each camera device in the photovoltaic power station to be remotely transmitted and displayed, a cloud platform is proposed to receive and transmit the monitoring images of the photovoltaic power station collected by each camera device, and a terminal monitor downloads corresponding monitoring images for display. The core of the cloud technology lies in distributed storage and distributed computing, and the cloud technology is adopted for storing and displaying the monitoring images of the photovoltaic power station, so that the burden of a computer for operating a remote monitoring system can be reduced, and the processing speed of the monitoring images is improved.

In addition, aiming at the problem of overlarge capacity of the monitoring image in the remote transmission of the monitoring image, a video compression technology can be used for compressing the monitoring image so as to relieve the problem of data transmission delay. The existing monitoring image compression technology adopts the same compression algorithm for all images without distinction, and when the number of monitoring images is obviously increased, the monitoring image compression algorithm still cannot compress the capacity of all monitoring images to a particularly low level.

For the remote transmission and display problems of monitoring images of a photovoltaic power station, only a scheme of applying a cloud technology in a targeted manner is provided at present. Although other image processing technologies such as video compression and the like can also be applied to remote transmission and display of monitoring images of the photovoltaic power station, the characteristics of the monitoring images of the photovoltaic power station are not fully considered, so that the data volume of the monitoring images which need to be transmitted to photovoltaic power station owners or professional operation and maintenance companies is still large. In summary, the disadvantages of the prior art solutions are mainly reflected in the following aspects:

(1) the characteristic of high redundancy of monitoring images of the photovoltaic power station is not considered.

Photovoltaic power plant often builds in the top of waters such as rare people cigarette, the harsh barren mountain area of environment, rivers lake, sewage treatment plant, the roof of buildings such as resident house, factory building, and photovoltaic power generation adopts the fixed mounting mode more, the probability of taking place accidents such as conflagration is very low, consequently, the scene change in photovoltaic power plant camera device field is very little in most of the time, the long-range monitoring system who sends photovoltaic power plant owner or professional fortune dimension company via network transmission is mostly redundant monitoring image, because its repeatability, lead to its value of utilization very low.

(2) The characteristics of different areas in the monitoring image of the photovoltaic power station with different importance are not considered.

A normally operating photovoltaic power station consists of core equipment such as a photovoltaic array, an inverter, a combiner box, a transformer and the like, and can also comprise auxiliary equipment such as a photovoltaic cleaning robot and the like. Different equipment has different importance, different reliability and different failure probability for the photovoltaic power station. However, the existing video monitoring system does not distinguish the monitoring levels of different areas in the photovoltaic power station, which means that the attention degrees given to different areas in the monitoring images acquired by the camera device are the same.

(3) The characteristics of different importance of a plurality of monitoring images of different photovoltaic power stations are not considered.

A video monitoring system is installed on a photovoltaic power station, and a plurality of camera devices have different directions so as to realize the full coverage of a power station factory. Since different camera devices monitor different areas, the importance of the corresponding monitored images may differ to some extent. However, the attention given by the existing video monitoring system to a plurality of monitoring images of the same factory or a plurality of monitoring images of a plurality of factories is the same.

Disclosure of Invention

An object of an exemplary embodiment of the present invention is to provide a video cluster monitoring system for a photovoltaic power station and an image processing method thereof, which can solve the problem of remote transmission and display of monitoring images in the existing photovoltaic power station.

In one general aspect, an image processing method for a video cluster monitoring system of a photovoltaic power station is provided, where the video cluster monitoring system of the photovoltaic power station includes a monitoring terminal and a plurality of photovoltaic power stations, each photovoltaic power station has a plurality of cameras arranged therein, and the cameras are used to capture monitoring images of different monitored scenes in the corresponding photovoltaic power station, and the image processing method includes: performing the following for each of a plurality of image pickup devices arranged in each photovoltaic power plant: acquiring a monitoring image of a preset monitored scene captured by any one camera device aiming at the any one camera device; determining a difference image of the captured monitoring image and a reference image of the arbitrary one of the image pickup devices; determining at least one foreground sub-image based on the determined difference image, wherein the at least one foreground sub-image is used for reflecting image difference between the captured monitoring image and the reference image; determining a similarity index of the captured monitoring image to the reference image based on the at least one foreground sub-image; and if the similarity index is smaller than a similarity threshold value, sending the data information of the captured monitoring image to a monitoring terminal so as to present the monitoring image on the monitoring terminal.

Alternatively, the step of determining a difference image of the captured monitor image and the reference image of the arbitrary one of the image pickup devices may include: and performing difference operation on the captured monitoring image and the reference image to obtain a difference image, wherein the number of pixel points of the captured monitoring image and the reference image in the two-dimensional image space along the X-axis direction and the Y-axis direction is the same.

Optionally, the step of determining at least one foreground sub-picture based on the determined difference image may comprise: carrying out binarization processing on the determined difference image to obtain a binarization difference image; and determining an image formed by adjacent pixel points with the pixel value of 1 in the obtained binarization difference image as a foreground sub-image so as to obtain at least one foreground sub-image in the binarization difference image.

Optionally, based on the at least one foreground sub-graph, the step of determining a similarity index of the captured monitoring image to the reference image may comprise: calculating the area of each foreground subgraph; determining a similarity index of the captured monitoring image to the reference image based on the areas of all foreground subgraphs and the area of the captured monitoring image.

Optionally, the step of determining a similarity index of the captured monitoring image to the reference image based on the areas of all foreground subgraphs and the area of the captured monitoring image may comprise: calculating the sum of the areas of all the foreground subgraphs; calculating the ratio of the sum of the areas of all the foreground subgraphs to the area of the captured monitoring image; and determining the difference value of 1 and the calculated ratio as the similarity index of the captured monitoring image and the reference image.

Alternatively, the step of transmitting the data information of the captured monitoring image to the monitoring terminal may include: storing the data information of the captured monitoring image as a notification message into a sending queue, wherein at least one notification message is stored in the sending queue, and the at least one notification message is the data information of the monitoring image of which the similarity index determined by judging the similarity between the monitoring image captured by each camera device and the corresponding reference image is smaller than the similarity threshold value; and sending each notification message in the sending queue to the monitoring terminal according to the transmission priority of each monitoring image.

Alternatively, the transmission priority of any one of the monitoring images may be determined by: determining at least one designated area in the any one monitored image; determining the number of pixel points of at least one foreground subgraph corresponding to any one monitoring image falling into at least one designated area; setting a corresponding area correction coefficient for each designated area; and determining the transmission priority of any monitoring image according to the determined number of the pixel points falling into each designated area and the area correction coefficient corresponding to each designated area.

Optionally, the step of determining the transmission priority of any one of the monitoring images according to the determined number of the pixel points falling into each of the designated areas and the area correction coefficient corresponding to each of the designated areas may include: calculating the difference value between the sum of the number of all pixel points in any one monitoring image and the number of the pixel points falling into each designated area; aiming at each appointed region, calculating the product of the number of pixel points falling into the appointed region and a region correction coefficient corresponding to the appointed region; calculating the sum of products corresponding to all the designated areas; and determining the ratio of the sum of the products obtained by calculation to the sum of the difference values to the sum of the number of all pixel points in any one monitoring image as the transmission priority of any one monitoring image.

Optionally, the at least one designated area may be an area including different scenes in any one of the monitoring images, and each designated area is not overlapped with each other.

Optionally, the data information may include complete image information of the captured monitoring image, or data information of a minimum segmentation map in the captured monitoring image, where the minimum segmentation map may be a minimum image of a region where the at least one foreground sub-image is located, which is extracted from the captured monitoring image.

Optionally, the minimum segmentation map may be a minimum bounding rectangle containing an area where the at least one foreground sub-map is located, where the data information of the minimum segmentation map may include image information of the minimum segmentation map, location information, identification information, and a timestamp, the location information may indicate a location of the minimum segmentation map in the captured monitoring image, the identification information may indicate a camera device that captures the monitoring image to which the minimum segmentation map belongs, the timestamp may include a first timestamp and a second timestamp, the first timestamp may indicate a time when the monitoring image to which the minimum segmentation map belongs is captured, and the second timestamp may indicate a time when a reference image of the camera device that captures the monitoring image to which the minimum segmentation map belongs is captured.

Optionally, the image processing method may further include receiving, by the monitoring terminal, data information of the monitoring image from each of the plurality of photovoltaic power stations, respectively, and storing the data information of the monitoring image received from each of the photovoltaic power stations in a receiving queue; extracting data information of the monitoring images from the receiving queue according to the transmission priority of each monitoring image; and displaying the monitoring image corresponding to the extracted data information on the monitoring terminal according to the extracted data information of the monitoring image.

Optionally, the image processing method further includes: storing, by the monitoring terminal, the monitoring image presented at the monitoring terminal as a reference image, and storing a time at which the presented monitoring image is captured as a second timestamp, wherein the step of presenting, at the monitoring terminal, the monitoring image corresponding to the extracted data information according to the extracted data information of the monitoring image may include: acquiring a reference image corresponding to a second time stamp from the monitoring terminal according to the second time stamp in the data information of the monitoring image; zeroing pixel values of positions indicated by position information in the data information of the acquired reference image and the monitoring image to obtain a background image; and synthesizing the minimum segmentation image and the background image to obtain a monitoring image, and presenting the monitoring image obtained by synthesis at the monitoring terminal.

Optionally, the image processing method may further include: each photovoltaic power station sends the complete image information of the monitoring image captured by each camera device to the monitoring terminal in a predetermined period so as to store the monitoring image with the complete image information as a reference image, wherein the step of presenting the monitoring image corresponding to the extracted data information at the monitoring terminal according to the extracted data information of the monitoring image may further include: acquiring a reference image which is closest to a second timestamp at the moment when the monitoring image is captured from the monitoring terminal according to the second timestamp in the data information of the monitoring image at a preset time interval; zeroing pixel values of positions indicated by position information in the acquired data information of the reference image and the monitoring image to obtain a background image; and synthesizing the minimum segmentation image and the background image to obtain a monitoring image, and presenting the monitoring image obtained by synthesis at the monitoring terminal.

Optionally, the image processing method may further include: in each photovoltaic power station, storing complete image information of a monitoring image transmitted to a monitoring terminal and a time at which the transmitted monitoring image is captured, wherein the image processing method further comprises: and sending a feedback message to the photovoltaic power station sending the data information after the monitoring terminal presents the monitoring image corresponding to the extracted data information, wherein the feedback message comprises a second timestamp indicating the moment when the monitoring image serving as the reference image is captured and identification information indicating the camera device capturing the monitoring image serving as the reference image, and the photovoltaic power station sending the data information can determine the monitoring image captured by the camera device indicated by the identification information in the feedback message at the moment corresponding to the second timestamp as the reference image of the camera device based on the received feedback message for similarity comparison.

In another general aspect, there is provided a video cluster monitoring system for a photovoltaic power station, including a monitoring terminal and a plurality of photovoltaic power stations, each photovoltaic power station including: the system comprises a plurality of camera devices, a plurality of sub-processing units and a control unit, wherein the camera devices are used for capturing monitoring images of different monitored scenes in corresponding photovoltaic power stations, the sub-processing units respectively correspond to the camera devices, and each sub-processing unit is configured to execute the following processing aiming at the corresponding camera device: the method comprises the steps of obtaining a monitoring image of a preset monitored scene captured by a corresponding camera device, determining a difference image between the captured monitoring image and a reference image of the camera device, determining at least one foreground sub-image based on the determined difference image, wherein the at least one foreground sub-image is used for reflecting image difference between the captured monitoring image and the reference image, determining a similarity index between the captured monitoring image and the reference image based on the at least one foreground sub-image, and sending data information of the captured monitoring image to a monitoring terminal when the similarity index is smaller than a similarity threshold value so as to present the monitoring image on the monitoring terminal.

Alternatively, the process of determining a difference image of the captured monitor image and the reference image of the image pickup device may include: and performing difference operation on the captured monitoring image and the reference image to obtain a difference image, wherein the number of pixel points of the captured monitoring image and the reference image in the two-dimensional image space along the X-axis direction and the Y-axis direction is the same.

Optionally, the processing of determining at least one foreground subgraph based on the determined difference image may comprise: carrying out binarization processing on the determined difference image to obtain a binarization difference image; and determining an image formed by adjacent pixel points with the pixel value of 1 in the obtained binarization difference image as a foreground sub-image so as to obtain at least one foreground sub-image in the binarization difference image.

Optionally, the processing of determining a similarity index of the captured monitoring image to the reference image based on the at least one foreground sub-image may comprise: calculating the area of each foreground subgraph; determining a similarity index of the captured monitoring image to the reference image based on the areas of all foreground subgraphs and the area of the captured monitoring image.

Optionally, the processing of determining a similarity index of the captured monitoring image to the reference image based on the areas of all foreground subgraphs and the area of the captured monitoring image may comprise: calculating the sum of the areas of all the foreground subgraphs; calculating the ratio of the sum of the areas of all the foreground subgraphs to the area of the captured monitoring image; and determining the difference value of 1 and the calculated ratio as the similarity index of the captured monitoring image and the reference image.

Alternatively, the first transmitting unit may store the data information of the captured monitoring image as a notification message in a transmission queue, and transmit each notification message in the transmission queue to the monitoring terminal according to the transmission priority of each monitoring image, where at least one notification message may be stored in the transmission queue, and the at least one notification message is the data information of the monitoring image of which the similarity index determined by performing similarity determination on the monitoring image captured by each imaging device and the corresponding reference image is smaller than the similarity threshold.

Alternatively, the first sending unit may determine the transmission priority of any one of the monitoring images by: determining at least one designated area in the any one monitored image; determining the number of pixel points of at least one foreground subgraph corresponding to any one monitoring image falling into at least one designated area; setting a corresponding area correction coefficient for each designated area; and determining the transmission priority of any monitoring image according to the determined number of the pixel points falling into each designated area and the area correction coefficient corresponding to each designated area.

Optionally, the first sending unit may calculate a difference between a sum of the number of all the pixel points in the arbitrary monitoring image and the number of the pixel points falling into each designated area; aiming at each appointed region, calculating the product of the number of pixel points falling into the appointed region and a region correction coefficient corresponding to the appointed region; calculating the sum of products corresponding to all the designated areas; and determining the ratio of the sum of the products obtained by calculation to the sum of the difference values to the sum of the number of all pixel points in any one monitoring image as the transmission priority of any one monitoring image.

Optionally, the monitoring terminal may include: the first receiving unit is used for respectively receiving the data information of the monitoring image from each photovoltaic power station in the plurality of photovoltaic power stations and storing the data information of the monitoring image received from each photovoltaic power station into a receiving queue; the processor is configured to: extracting data information of the monitoring images from the receiving queue according to the transmission priority of each monitoring image, and acquiring the monitoring images corresponding to the extracted data information according to the extracted data information of the monitoring images; and a display unit displaying the monitoring image corresponding to the extracted data information.

Optionally, the monitoring terminal may further include: and a memory that stores the monitoring image presented at the monitoring terminal as a reference image and stores a time at which the presented monitoring image is captured as a second time stamp, wherein the process of obtaining the monitoring image corresponding to the extracted data information according to the extracted data information of the monitoring image may include: acquiring a reference image corresponding to a second time stamp from a memory according to the second time stamp in the data information of the monitoring image; zeroing pixel values of positions indicated by position information in the data information of the acquired reference image and the monitoring image to obtain a background image; and synthesizing the minimum segmentation image and the background image to obtain a monitoring image corresponding to the extracted data information.

Alternatively, the first transmitting unit may further transmit, to the monitor terminal, full image information of the monitor image captured by each of the image capturing apparatuses at a predetermined cycle, and the memory may further store the monitor image having the full image information as a reference image, wherein the process of obtaining the monitor image corresponding to the extracted data information according to the extracted data information of the monitor image may further include: acquiring a reference image which is closest to a second time stamp at the moment when the monitoring image is captured from a memory according to the second time stamp in the data information of the monitoring image at preset time intervals; zeroing pixel values of positions indicated by position information in the acquired data information of the reference image and the monitoring image to obtain a background image; and synthesizing the minimum segmentation image and the background image to obtain a monitoring image corresponding to the extracted data information.

Optionally, each photovoltaic power plant may further comprise: a plurality of storage units respectively corresponding to the plurality of camera devices, for storing complete image information of the monitoring image captured by the corresponding camera device and transmitted to the monitoring terminal and a captured time of the transmitted monitoring image, wherein the monitoring terminal may further include: a second transmitting unit that transmits a feedback message to the photovoltaic power stations that transmit the data information after the monitoring terminal presents the monitoring image corresponding to the extracted data information, wherein the feedback message may include a second timestamp indicating a time when the monitoring image as the reference image was captured and identification information indicating a camera that captured the monitoring image as the reference image, wherein each photovoltaic power station may further include: and the sub-processing unit in the photovoltaic power station sending the data information can determine the monitoring image captured by the camera device indicated by the identification information in the feedback message at the moment corresponding to the second timestamp as a reference image of the camera device for similarity comparison.

In another general aspect, there is provided a computer readable storage medium having stored thereon a computer program which, when being executed by a processor, implements the image processing method of the photovoltaic power plant video cluster monitoring system described above.

By adopting the photovoltaic power station video cluster monitoring system and the image processing method thereof in the exemplary embodiment of the invention, operation and maintenance personnel can timely know the conditions of each photovoltaic power station through the monitoring image, the storage space occupied by the monitoring image and the transmission network bandwidth can be reduced, and the transmission and display updating rates of important monitoring images are ensured.

Drawings

The above and other objects, features and advantages of exemplary embodiments of the present invention will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings which illustrate exemplary embodiments.

Fig. 1 shows a flow chart of an image processing method of a photovoltaic power plant video cluster monitoring system according to an exemplary embodiment of the invention;

FIG. 2 shows a flowchart of the steps of obtaining at least one foreground subgraph according to an exemplary embodiment of the invention;

FIG. 3 shows a flowchart of the steps of determining a transmission priority of any one of the monitoring images according to an exemplary embodiment of the present invention;

FIG. 4 illustrates a schematic diagram of extracting a minimal segmentation map from a monitored image according to an exemplary embodiment of the present invention;

FIG. 5 illustrates a flowchart showing steps of presenting a monitoring image at a monitoring terminal according to an exemplary embodiment of the present invention;

FIG. 6 is a flowchart illustrating steps of obtaining a monitoring image by image synthesis at a monitoring terminal according to an exemplary embodiment of the present invention;

fig. 7 shows a block diagram of a photovoltaic power plant video cluster monitoring system according to an exemplary embodiment of the present invention.

Detailed Description

Various example embodiments will now be described more fully with reference to the accompanying drawings, in which some example embodiments are shown.

The monitoring image of the photovoltaic power station has the characteristics of intuition and image, can be used for safety fields such as security protection and fire control, can also be used for guiding maintenance work such as cleaning, overhauling, installation and replacement, and provides important support for operation and maintenance management of the power station.

In the exemplary embodiment of the invention, aiming at the problems existing in the remote transmission and display of the monitoring image in the prior art, on the basis of the application of an image processing technology, the characteristics of strong redundancy and low real-time requirement of the monitoring image of the photovoltaic power station are combined, and an image processing method of a video cluster monitoring system suitable for the photovoltaic power station is provided.

The video cluster monitoring system for the photovoltaic power stations comprises a monitoring terminal and a plurality of photovoltaic power stations, wherein a plurality of camera devices are arranged in each photovoltaic power station and used for capturing monitoring images of different monitored scenes in the corresponding photovoltaic power stations.

Fig. 1 shows a flowchart of an image processing method of a photovoltaic power plant video cluster monitoring system according to an exemplary embodiment of the present invention. Here, the image processing method shown in fig. 1 may be executed for each of a plurality of image pickup devices arranged in each photovoltaic power plant, and an execution procedure of the image processing method of the photovoltaic power plant video cluster monitoring system will be described below by taking any one of the image pickup devices as an example.

Referring to fig. 1, in step S10, a monitoring image of a predetermined monitored scene captured by a camera is acquired.

In step S20, a difference image of the captured monitor image and the reference image of the image pickup device is determined.

Here, the monitoring terminal may display monitoring images captured by different cameras in the plurality of photovoltaic power stations in a split screen or multi-screen manner, and the reference image of the camera may refer to the monitoring image currently being displayed on the monitoring terminal and captured by the camera, that is, the reference images of different cameras are different.

For example, a difference image may be obtained by performing a difference operation on the captured monitor image and a reference image of the image pickup device. Here, it is necessary to perform the difference calculation on the premise that the number of pixel points in the two-dimensional image space between the captured monitor image and the reference image is the same as that in the X-axis direction and that in the Y-axis direction.

In an example, taking the example that the monitoring image captured by the camera device is a digital image, the captured monitoring image is composed of M × N discrete pixel points in a two-dimensional image space, M is the number of pixel points in the X-axis direction in the two-dimensional image space, and N is the number of pixel points in the Y-axis direction in the two-dimensional image space.

Suppose an imaging device C₁At t₁And t₂The monitoring images captured at the moment are respectively F_t1(x, y) and F_t2(x, y), wherein x is more than or equal to 1 and less than or equal to M, y is more than or equal to 1 and less than or equal to N, t₂＞t₁Will F_t2(x, y) and F_t1(x, y) performing difference operation to obtain a difference image D_t1→t2(x,y)，

D_t1→t2(x,y)＝|F_t2(x,y)-F_t1(x,y)| (1)

In the formula (1), D_t1→t2(x, y) is an image pickup device C₁At t₂The monitoring image captured at the moment and at t₁A difference image of the monitoring image captured at the moment.

Here, it should be understood that when imagingDevice C₁At t₁The monitoring image captured at the moment is the camera C₁The difference image of the captured monitor image and the reference image can be obtained using the above formula (1).

In step S30, at least one foreground sub-image is determined based on the determined difference image. Here, the at least one foreground sub-image is used to reflect image differences between the captured monitor image and the reference image.

The step of determining at least one foreground sub-picture is described below with reference to fig. 2. It should be understood that the manner of determining at least one foreground sub-image shown in fig. 2 is only a preferred example, and the present invention is not limited thereto, and other manners may be adopted to obtain a sub-image reflecting the image difference between two images.

Fig. 2 shows a flow chart of the steps of obtaining at least one foreground subgraph according to an exemplary embodiment of the invention.

Referring to fig. 2, in step S301, binarization processing is performed on the determined difference image, obtaining a binarized difference image.

For example, the determined difference image may be subjected to binarization processing based on a preset foreground threshold value. Here, the magnitude of the foreground threshold value may be empirically determined by those skilled in the art.

Foreground threshold T_DThe larger the numerical value of (2), the discrimination between the two monitoring images is reduced, so that the data volume of the monitoring images transmitted to the monitoring terminal by the photovoltaic power station can be reduced. However, the foreground threshold T_DIf the selection is too large, the number of monitoring images transmitted from the photovoltaic power station to the monitoring terminal is too small, and operation and maintenance personnel of the monitoring terminal may miss important information in the plant area of the photovoltaic power station. Foreground threshold T_DThe smaller the numerical value of the monitoring image is, the higher the discrimination between the two monitoring images is, so that operation and maintenance personnel in a monitoring room can know more actual conditions in a plant area of the photovoltaic power station. However, the foreground threshold T_DIf the selection is too small, the redundant monitoring images transmitted from the photovoltaic power station to the monitoring terminal are too large, so that network resources are wasted, and the monitoring images on the display equipment in the monitoring terminal are reducedThe update rate of.

In a preferred embodiment, in the experimental stage of the method applying the exemplary embodiment of the present invention, a suitable foreground threshold T may be selected according to the results of multiple experiments_DThe average value of (c) is used as the final usage value. The operation and maintenance personnel can also adjust the foreground threshold T according to specific conditions in the process of executing the image processing method_DThe size of (2). In addition, different foreground threshold values T can be selected for different camera devices of different photovoltaic power stations according to different actual conditions_D。

For example, the following formula may be utilized to base the foreground threshold T on_DFor difference image D_t1→t2(x, y) performing binarization:

in the formula (2), R_t1→t2(x, y) is an image pickup device C₁At t₂The monitoring image captured at the moment and at t₁And (4) a binarization difference image of the monitoring image captured at the moment.

By way of example, if for any x and y, there is R_t1→t2If (x, y) is 1, the binary difference image is a pure white image, and if there is R for any x and y_t1→t2If (x, y) ═ 0, the binary difference image is a solid black image.

In step S302, an image composed of adjacent pixel points with a pixel value of 1 in the obtained binarized difference image is determined as a foreground sub-image, so as to obtain at least one foreground sub-image in the binarized difference image.

At the binary difference image R_t1→t2There may be one or more foreground sub-images in (x, y), and for the case that the binarized difference image is a pure black image, there is no foreground sub-image in the binarized difference image.

Returning to fig. 1, in step S40, a similarity index of the captured monitoring image to the reference image is determined based on the at least one foreground sub-image.

For example, the similarity index may be determined by: and calculating the area of each foreground subgraph, and determining a similarity index of the captured monitoring image and the reference image based on the areas of all the foreground subgraphs and the area of the captured monitoring image.

For example, the binary difference image R may be_t1→t2The kth foreground subgraph in (x, y) is noted as: r_t1→t2,k(x, y), at which point the kth foreground sub-graph R may be considered_t1→t2,kThe number of all pixel points in (x, y) is called as the kth foreground subgraph R of the binary difference image_t1→t2,kThe area of (x, y) can be represented as S (R)_t1→t2,k(x,y))。

Here, the maximum area of the foreground sub-image of the binarized difference image is M × N, the minimum area of the foreground sub-image of the binarized difference image is 1, and the foreground sub-image of the binarized difference image is a black image with only one white dot.

As an example, the specific calculation process for determining the similarity index is as follows: calculating the sum of the areas of all the foreground subgraphs; calculating the ratio of the sum of the areas of all the foreground subgraphs to the area of the captured monitoring image; and determining the difference value of 1 and the calculated ratio as the similarity index of the captured monitoring image and the reference image.

For example, the similarity index may be calculated by the following formula:

in the formula (3), α is a similarity index, S (R)_t1→t2,k(X, Y)) is the area of the kth foreground subgraph of the binary difference image, K is more than or equal to 1 and less than or equal to K, K is the number of foreground subgraphs in the binary difference image, M and N are the number of pixel points in the two-dimensional image space along the X-axis direction and the Y-axis direction respectively, and M multiplied by N is the area of the captured monitoring image.

Here, since the difference image R_t1→t2(x, y) is a binarized difference image, and therefore, when specifically calculating the similarity index α, it may not be necessary to consider each previous oneThe shape of the scene graph, while only counting R_t1→t2And (x, y) counting the number of all pixel points with the pixel value of 1, and taking the counted number as the sum of the areas of all the foreground subgraphs.

It should be noted that although the image similarity is a numerical value, the two monitor images having different sizes cannot be compared in similarity because they cannot find a difference image. In other words, the image similarity is meaningful only for the same size of monitor image.

In an exemplary embodiment of the present invention, the degree of similarity between two monitored images is quantitatively expressed using a similarity index. It should be understood that the size of the similarity index of the two monitored images is related to the characteristics of the images, and also related to the size of the foreground threshold set when the difference image is subjected to the binarization processing.

In step S50, it is determined whether the determined similarity index is less than a similarity threshold.

Here, the monitoring image to be transmitted to the monitoring terminal may be selected based on the similarity threshold. Here, the similarity threshold T_αThe numerical value of (c) can be determined empirically by one skilled in the art.

For similarity threshold T_αThe selection of the value of (a) will affect the remote transmission and display of the monitoring image. Similarity threshold T_αThe larger the numerical value is, the fewer the selected monitoring images needing to be transmitted to the monitoring terminal are, so that operation and maintenance personnel may miss important information in a plant area of the photovoltaic power station, and the similarity threshold value T_αThe smaller the numerical value of the monitoring image is, the more the selected monitoring images need to be transmitted to the monitoring terminal, and the more the operation and maintenance personnel can know the actual situation in the plant area of the photovoltaic power station, but the excessive number of the redundant monitoring images transmitted to the monitoring terminal can also be caused.

If the determined similarity index is not less than (greater than or equal to) the similarity threshold, the process returns to step S10, and the monitoring image of the predetermined monitored scene captured at the next time is re-acquired.

If the determined similarity index is less than the similarity threshold, then step S60 is performed: and sending the data information of the captured monitoring image to the monitoring terminal so as to present the captured monitoring image at the monitoring terminal.

Here, it should be understood that in most cases, scene changes in a plant area of a photovoltaic power station are small, and a monitoring image of the photovoltaic power station is characterized in that in most cases, a difference between two previous and next frames of monitoring images acquired by the same camera device is small, and a similarity is high, so that great redundancy exists between the monitoring images. If the photovoltaic power station remotely transmits each frame of monitoring image captured by each camera device to the monitoring terminal of an owner or a professional operation and maintenance company through the network, a considerable amount of network bandwidth is occupied, and network resources are wasted. In view of the above problem of image transmission, in the exemplary embodiment of the present invention, only an image with a larger difference from the monitoring image displayed by the remote monitoring terminal needs to be selected as the monitoring image to be transmitted, so as to filter out the redundant monitoring image, thereby reducing the transmission amount of the monitoring image data.

In the exemplary embodiment of the invention, the similarity index of the image is used as a judgment basis for judging whether a monitoring image currently captured by a certain camera device in the photovoltaic power station is remotely transmitted to a monitoring terminal of an owner or a professional operation and maintenance company of the photovoltaic power station. The similarity index between the currently captured monitoring image and the monitoring image displayed by the monitoring terminal of the owner or the professional operation and maintenance company of the photovoltaic power station is greater than or equal to the similarity threshold T_αWhen the similarity index of the two images is smaller than the similarity threshold T, the currently captured monitoring image is not transmitted to the monitoring terminal_αAnd then, sending the currently captured monitoring image to a remote monitoring terminal. Therefore, operation and maintenance personnel in the remote monitoring room can know important conditions of the photovoltaic power station plant as far as possible, and the monitoring images cannot occupy too much storage space and transmission network bandwidth.

That is to say, the basic idea of the image processing method according to the exemplary embodiment of the present invention is to filter the monitoring images captured by the cameras in the plant area of each photovoltaic power station locally in the photovoltaic power station, and send only the monitoring image that is sufficiently different from the monitoring image displayed by the monitoring terminal of the owner of the photovoltaic power station or the professional operation and maintenance company to the remote monitoring terminal.

In a preferred embodiment, after the similarity determination is performed on the monitoring images captured by each camera device by the above image processing method, at this time, there may be a plurality of data information of the monitoring images that need to be transmitted to the monitoring terminal, in this case, the plurality of data information of the monitoring images that need to be transmitted to the monitoring terminal may be transmitted one by one according to the transmission priority of the monitoring images.

For example, the data information of the captured monitoring image may be stored as a notification message in a transmission queue, and each notification message in the transmission queue may be transmitted to the monitoring terminal according to the transmission priority of each monitoring image.

Here, at least one notification message including data information of a monitoring image of which a similarity index determined by performing similarity determination on the monitoring image captured by each image pickup device and a corresponding reference image is smaller than a similarity threshold value is stored in the transmission queue.

The step of determining the transmission priority of each monitored image is described below with reference to fig. 3.

Fig. 3 is a flowchart illustrating a procedure of determining a transmission priority of any one of the monitoring images according to an exemplary embodiment of the present invention.

Referring to fig. 3, in step S601, at least one designated area is determined in any one of the monitored images.

Preferably, the at least one designated area can be an area containing different scenes in any one monitoring image, and the influence degrees of the different scenes on the photovoltaic power station are different. By way of example, the scene may include, but is not limited to, birds or leaves falling on the photovoltaic panel, the entrance of foreign persons in the plant area of the photovoltaic power plant, and the occurrence of a fire accident.

By way of example, each designated area may be of any shape, preferably, an index one designated area may comprise a regular pattern, such as a rectangle or circle, to facilitate computer processing.

In an exemplary embodiment of the present invention, the monitoring images of the photovoltaic power station are divided by a designated area to distinguish the difference in transmission priority of the monitoring images due to the difference in importance. In one example, it is assumed that the at least one designated area may include, but is not limited to, an area where a photovoltaic panel is located, an area where an inverter is located. For example, assume that one monitoring image includes a large-area photovoltaic panel, and the other monitoring image includes an inverter. The loss of the photovoltaic power station caused by the fire at the position of the inverter is larger than the loss of the photovoltaic power station caused by the fire at the position of the photovoltaic panel on the same scale, because the fire at the photovoltaic panel can only cause one path of photovoltaic module not to generate power, and the fire at the inverter can cause a plurality of paths of photovoltaic modules connected with the inverter not to supply power. Therefore, the monitoring image containing the inverter fire should be remotely transmitted to the monitoring terminal of the owner or the professional operation and maintenance company of the photovoltaic power station more preferentially.

For example, a designated area may be selected from the two previous monitoring images, and the designated area includes an area where the inverter is located and an area where the photovoltaic panel is located, and accordingly, the area correction coefficient of the designated area including the inverter may be set to be greater than the area correction coefficient of the designated area including the photovoltaic panel, so that the transmission priority of the monitoring image including the inverter is higher than the transmission priority of the monitoring image including the photovoltaic panel.

For example, from any one of the monitoring images F_tAnd (x, y) selecting a sub-image, wherein the position of the sub-image in any monitoring image is called a designated area of the any monitoring image and can be marked as A.

Here, because the difference image R is binarized_t1→t2(x, y) from the monitor image F_t1(x, y) and F_t2(x, y) is obtained by algebraic operation between pixel points, and the algebraic operation aiming at the pixel points does not change the geometric elements of the image, so that the binary difference image R in a two-dimensional image space_t1→t2Coordinate values of respective points in the designated area of (x, y) and the monitor image F_t1(x, y) and F_t2And coordinate values of the (x, y) points are in one-to-one correspondence. In other words, the binary difference image R_t1→t2The specified region of (x, y) is also the monitor image F_t1(x, y) and F_t2(x, y) specified region.

In step S602, the number of pixel points of at least one foreground sub-image corresponding to any one monitored image falling into at least one designated area is determined.

In step S603, a corresponding area correction coefficient is set for each designated area.

According to different scenes contained in any one monitoring image, different area correction coefficients can be set for different designated areas of the any one monitoring image respectively. Each monitoring image can be provided with a plurality of designated areas, and one designated area is provided with a corresponding area correction coefficient. Moreover, each designated area is not overlapped with each other for regulation, namely, a plurality of designated areas cannot share a pixel point. Here, a unique area correction coefficient may be set for different designated areas, or the same area correction coefficient may be set for different designated areas.

Setting a monitoring image F_tOne designated area of (x, y) is A₁，λ(A₁) Representing a monitored image F_t(x, y) for the designated area A₁The area correction coefficient of (1). Preferably, when the value of λ is greater than 1, it indicates that the designated area is an important area in the monitored image. And when the value of the lambda is equal to 1, indicating that the appointed area is not selected in the monitoring image.

For at least one foreground subgraph of the binarization difference image, when the pixel point of the at least one foreground subgraph is in the designated area with the lambda value larger than 1, the monitoring image to be transmitted should obtain higher transmission priority.

In step S604, a transmission priority of any one of the monitoring images is determined according to the determined number of pixel points falling into each of the designated areas and the area correction coefficient corresponding to each of the designated areas.

As an example, a specific process of determining the transmission priority of any one monitoring image may be: calculating the difference value between the sum of the number of all pixel points in any monitoring image and the number of the pixel points falling into each designated area; aiming at each appointed region, calculating the product of the number of pixel points falling into the appointed region and a region correction coefficient corresponding to the appointed region; calculating the sum of products corresponding to all the designated areas; and determining the ratio of the sum of the products and the sum of the differences obtained by calculation to the sum of the number of all pixel points in any one monitoring image as the transmission priority of any one monitoring image.

For example, the transmission priority of any one of the monitoring images can be calculated using the following formula:

in the formula (4), β represents the transmission priority of any one monitored image, and assuming that there are L designated regions in the binarized difference image, each designated region is represented as a₁,A₂,…,A_L，λ(A_l) Indicates the l-th designated area A_lL is more than or equal to 1 and less than or equal to L, L is the number of designated areas, and the number of pixel points of at least one foreground subgraph falling into the L designated areas is respectively expressed as v₁,v₂,…,v_LAnd M and N respectively represent the number of pixel points along the X-axis direction and the Y-axis direction in a two-dimensional image space.

As an example, assuming no specified region exists in the binarized difference image, then for any L ∈ [1, L]All have lambda (A)_l) 1. In this case, the transmission priority of the monitored image at this time can be calculated based on the above equation (4) as:

the above calculation result may show that, when there is no designated area in the binarized difference image, the transmission priority of the monitored image to be transmitted is 1.

For the case that a specified region exists in the binary difference image, for any L ∈ [1, L ]]Having λ (A)_l) Is greater than 1. In this case, the transmission priority of the monitored image at this time can be calculated based on the above equation (4) as:

the above calculation result may show that, when the designated area exists in the binarized difference image, the transmission priority of the monitored image to be transmitted is greater than 1.

In the exemplary embodiment of the present invention, the whole image of the captured monitoring image may be sent to the monitoring terminal, and besides, in order to reduce the occupation of the network bandwidth and increase the rate of image update, only a part of the image in the captured monitoring image may be sent to the monitoring terminal.

In the first case, the data information of the monitoring image transmitted to the monitoring terminal may include complete image information of the captured monitoring image.

The transmission mode can ensure the integrity of the monitoring image displayed at the monitoring terminal, but can occupy more network bandwidth and influence the image updating rate.

In the second case, the data information of the monitoring image transmitted to the monitoring terminal may include data information of a minimum segmentation map in the captured monitoring image.

Here, since the scene covered by some cameras in the plant area of the photovoltaic power plant remains stationary for most of the time, the similarity of the monitoring images between different frames is high. Even if the selected similarity index is smaller than the similarity threshold T_αThe transmitted monitoring image is still large in redundant information, and the monitoring image is sent to the monitoring terminal.

With more and more photovoltaic power stations brought into visual operation and maintenance management, more monitoring images of the camera device need to be remotely transmitted to monitoring terminals of owners of the photovoltaic power stations or professional operation and maintenance companies, and the data volume of the monitoring images transmitted on the network is larger and larger.

The data information of the redundant monitoring images is remotely transmitted to a monitoring terminal of an owner or a professional operation and maintenance company by the photovoltaic power station, so that operation and maintenance personnel are not helped, network resources and computing resources of the power station and the monitoring terminal are wasted, and the transmission of the high-priority monitoring images is influenced.

In the exemplary embodiment of the invention, on the basis of screening the monitoring image to be transmitted based on the similarity index, the monitoring image to be transmitted is further divided, and only the data information of the minimum division graph of the monitoring image with larger difference with the monitoring image displayed by the remote monitoring terminal is sent to the monitoring terminal, so that the redundant information of the monitoring image to be transmitted is further reduced, and the data transmission quantity of the monitoring image is reduced.

In this case, the above calculation formula for calculating the transmission priority of the monitoring image is applied to both the entire monitoring image and the minimum segmentation map.

Here, the minimum segmentation map may be a minimum image including a region where at least one foreground sub-image is located, extracted from the captured monitoring image. For example, the largest minimum segmentation map in the monitored image is the monitored image itself, and the smallest minimum segmentation map is only one pixel point.

In other words, in this case, each photovoltaic power station only needs to transmit the data information of the minimum segmentation map, and does not need to transmit the complete image information of the monitoring image, so that the transmission rate can be effectively increased, and the bandwidth pressure is reduced.

Fig. 4 illustrates a schematic diagram of extracting a minimum segmentation map from a monitored image according to an exemplary embodiment of the present invention.

The minimum segmentation map may be a minimum image of the monitored image that contains all image differences between the captured monitored image and the reference image, and in a preferred embodiment, to facilitate processing of the image, the minimum segmentation map may be a minimum bounding rectangle containing a region in which the at least one foreground sub-image is located.

As shown in fig. 4, the smallest rectangle obtained by surrounding at least one foreground subgraph in two-dimensional space of the image with sides parallel to the X-axis and the Y-axis is called the smallest bounding rectangle, i.e. the smallest segmentation graph. Fig. 4 shows a minimum bounding rectangle a containing three foreground subgraphs.

Preferably, the minimum bounding rectangle does not allow the rectangle to be rotated in the image two-dimensional space to distinguish the minimum bounding rectangle of the image commonly used in the field of image processing.

Binarizing the difference image R in two-dimensional image space_t1→t2Coordinate values of each point in (x, y) and the monitoring image F_t1(x, y) and F_t2And coordinate values of each point in (x, y) are in one-to-one correspondence. Therefore, the spatial position of the minimum bounding rectangle containing at least one foreground subgraph in the binary difference image is also the monitoring image F_t2The spatial position of the smallest segmentation map to be transmitted in (x, y).

In the exemplary embodiment of the invention, the minimum segmentation map is determined as the to-be-transmitted area in the monitoring image, so that only the necessary partial area image in the to-be-transmitted monitoring image can be remotely transmitted to the monitoring terminal by the photovoltaic power station.

In a preferred embodiment, the minimal split subgraph can be obtained by: determining the minimum circumscribed rectangle of all foreground subgraphs in the binary difference image of the monitored image to be transmitted, determining the coordinates of the top left corner vertex and the top right corner vertex of the minimum circumscribed rectangle to determine the position of the minimum circumscribed rectangle in a two-dimensional image space, and intercepting an area on the monitored image to be transmitted according to the coordinates of the top left corner vertex and the top right corner vertex of the minimum circumscribed rectangle to obtain a minimum segmentation graph. Under the condition, the number of the pixels of all the foreground subgraphs in the binarization difference image of the monitoring image falling in the designated area is counted, and the transmission priority of the minimum segmentation image can be obtained according to the calculation formula of the transmission priority of the monitoring image.

For the case where the minimum segmentation map is extracted from the monitored image, the data information of the minimum segmentation map may include, but is not limited to, image information, location information, identification information, and a time stamp of the minimum segmentation map.

As an example, the position information may indicate a position of the minimum segmentation map in the captured monitoring image, the identification information may indicate a camera that captured the monitoring image to which the minimum segmentation map belongs, the time stamp may include a first time stamp that may indicate a time when the monitoring image to which the minimum segmentation map belongs was captured and a second time stamp that indicates a time when a reference image of the camera that captured the monitoring image to which the minimum segmentation map belongs was captured.

That is to say, before the captured monitoring image is sent out from the photovoltaic power station, image segmentation is firstly carried out, the minimum segmentation graphs with enough difference are used as the images to be transmitted as far as possible, and then the minimum segmentation graphs are arranged from high to low according to the transmission priority so as to ensure that the subgraph with the highest transmission priority is sent out firstly.

In the image processing method according to the exemplary embodiment of the present invention, after receiving the data information of the monitoring image from each photovoltaic power station, the monitoring terminal needs to recover the monitoring image based on the received data information of the monitoring image and display the data information on the monitoring terminal, so that the operation and maintenance personnel can remotely and timely know the situation of each photovoltaic power station through the monitoring image displayed on the monitoring terminal.

In the case where the minimum segmentation map is extracted from the monitoring image, after the data information of the minimum segmentation map is sent to the monitoring terminal of the owner of the photovoltaic power station or the professional operation and maintenance company, the monitoring terminal needs to perform image synthesis based on the received data information to obtain the monitoring image to be displayed, and notify the relevant photovoltaic power station through a feedback message, which monitoring image is captured by which camera device is now displayed. The process is continuously carried out, so that the monitoring images captured by the camera devices of the photovoltaic power stations can be remotely transmitted and displayed.

The steps of presenting the monitoring image at the monitoring terminal will be described with reference to fig. 5.

Fig. 5 is a flowchart illustrating a step of presenting a monitoring image at a monitoring terminal according to an exemplary embodiment of the present invention.

Referring to fig. 5, in step S70, data information of a monitoring image is received from each of a plurality of photovoltaic power stations, respectively, and stored in a receiving queue.

Preferably, since the data information of the monitoring images received by the monitoring terminal may be from a plurality of photovoltaic power stations, the data information of the monitoring images in the receiving queue also needs to be prioritized for transmission at the monitoring terminal, so that the monitoring images with high transmission priority can be preferentially displayed.

In step S80, the data information of the monitoring images is extracted from the reception queue in accordance with the transmission priority of each monitoring image.

For example, the data information of the monitoring images or the data information of the minimum segmentation map in the receiving queue may be sorted in the order of the transmission priority from high to low, with the highest transmission priority being arranged at the head of the queue and the lowest transmission priority being arranged at the tail of the queue.

As an example, the monitoring terminal may comprise a memory, which may take the form of a queue on a stored mathematical structure, preferably the maximum length of the queue being the sum of the lengths of the transmission queue of the photovoltaic power plant and the reception queue of the monitoring terminal.

In step S90, a monitoring image corresponding to the extracted data information is presented at the monitoring terminal based on the extracted data information of the monitoring image.

A monitor terminal in a monitor room of a proprietor of a photovoltaic power station or a professional operation and maintenance company is provided with a display, and can simultaneously display monitor images of different camera devices of a plurality of photovoltaic power stations in a split screen or multi-screen mode. When the monitoring image starts to be displayed on the display, the moment when the monitoring image is captured and the identification information are immediately sent to the corresponding photovoltaic power station, the photovoltaic power station determines the corresponding camera device based on the received identification information, determines the monitoring image as a reference image of the camera device, and determines the moment when the monitoring image is captured as a second time stamp of the reference image.

For example, assume that the image displayed on the display of the monitor terminal is F^1,1 _t1(x, y) corresponding to the monitoring image captured by the first camera of the photovoltaic power plant 1 at the moment t1, when the image F^1,1 _t1(x, y) is obtained by image synthesis based on the data information of the minimum segmentation chart, since it is locally stored with the monitoring image F of the photovoltaic power plant^1,1 _t1(x, y) are differentThus, the two monitoring images can be represented differently, for example, by F^1,1 _t1(x, y) represents a monitoring image displayed by the monitoring terminal, and F can be used^1,1 _t1(x, y) represents a monitoring image captured by a camera of the photovoltaic power plant to show the difference.

Aiming at the condition that the minimum segmentation graph is extracted from the monitoring image, the data information of the minimum segmentation graph stored in the receiving queue is sorted according to the transmission priority, so that the data information of the minimum segmentation graph with the high transmission priority is transmitted preferentially, operation and maintenance personnel of the photovoltaic power station can know the condition inside the photovoltaic power station through the displayed monitoring image outside the plant area of the photovoltaic power station, and the intelligentization level of the photovoltaic power station remote monitoring system and the management level of the operation and maintenance personnel are improved.

In this case, the monitoring image displayed on the monitoring terminal of the owner of the photovoltaic power station or the professional operation and maintenance company can be obtained by an image synthesis method. The steps of obtaining the monitoring image by the image synthesis at the monitoring terminal will be described with reference to fig. 6.

Fig. 6 is a flowchart illustrating a procedure of obtaining a monitoring image by an image synthesis manner at a monitoring terminal according to an exemplary embodiment of the present invention.

Referring to fig. 6, in step S901, a reference image corresponding to a second time stamp in the data information of the monitored image (i.e., the data information of the minimum segmentation map) is acquired from the monitoring terminal.

For example, the monitoring terminal may further perform the following steps: and storing the monitoring image presented at the monitoring terminal as a reference image, and storing the moment when the presented monitoring image is captured as a second timestamp.

Preferably, at the time of initialization, all monitoring images displayed by the monitoring terminal of the owner of the photovoltaic power station or the professional operation and maintenance company are set to be images of pure black, and accordingly, the reference image for comparison with the captured monitoring image in each photovoltaic power station is also set to be an image of pure black. In the method, any image is subjected to difference operation with a pure black image, and the original image is used, so that the first monitoring image captured by each camera device of the photovoltaic power station can be transmitted to a remote monitoring terminal.

In step S902, the pixel value of the position indicated by the position information in the data information of the monitored image in the acquired reference image is set to zero to obtain a background image.

For example, the background image may be obtained by multiplying the pixel value of the corresponding region in the reference image by 0 according to the position of the minimum segmentation map in the two-dimensional image space.

In step S903, the minimum segmentation map and the background image are synthesized to obtain a monitoring image, and the monitoring image obtained by the synthesis is presented at the monitoring terminal.

For example, the minimum segmentation map and the background image may be added according to the corresponding spatial positions to synthesize the monitoring image displayed on the monitoring terminal.

Here, the time stamp of the monitoring image obtained by the synthesis is the time at which the monitoring image is captured indicated in the data information of the monitoring image in the reception queue.

In the exemplary embodiment of the present invention, it is considered that a discontinuous phenomenon may occur in the monitoring image displayed by the monitoring terminal, so that the real situation of the monitoring image obtained after the synthesis and the monitored scene is different, resulting in an inaccurate monitoring result. This is caused by performing binarization processing on a difference image of the monitor image and the reference image, and selecting only a partial image (a minimum segmentation map) of the monitor image for remote transmission.

In order to overcome this phenomenon, in a preferred embodiment, each photovoltaic power station may transmit the complete image information of the monitoring image captured by each camera device to the monitoring terminal at a predetermined period, and the monitoring terminal may store the monitoring image with the complete image information as a reference image, or may directly display the monitoring image with the complete image information.

In this case, the monitoring terminal may synthesize the reference image with the data information based on the monitoring image at predetermined time intervals to obtain the monitoring image presented at the monitoring terminal.

For example, at predetermined time intervals, according to a second timestamp in the data information of the monitored image, a reference image at the moment when the monitored image is captured is closest to the second timestamp is acquired from the monitoring terminal; zeroing pixel values of positions indicated by position information in the acquired data information of the reference image and the monitoring image to obtain a background image; and synthesizing the minimum segmentation image and the background image to obtain a monitoring image, and presenting the monitoring image obtained by synthesis at the monitoring terminal.

Here, it should be understood that, preferably, the monitoring terminal continuously receives the data information of the minimum segmentation map from each photovoltaic power station, and the data information may be synthesized by using the reference image when a predetermined time interval is reached and synthesized by using the reference image at other times, so that the monitoring image obtained after synthesis better conforms to the real situation of the monitored scene.

Besides the image processing method for dealing with the special situation, the monitoring image captured by the appointed camera device of the appointed photovoltaic power station can be completely transmitted to the monitoring terminal, and the transmission of other monitoring images is temporarily shielded, so that operation and maintenance personnel can check the situation of the appointed photovoltaic power station plant area in time.

Further, the image processing method of a video cluster monitoring system according to an exemplary embodiment of the present invention may further include: in each photovoltaic power station, the complete image information of the monitoring image transmitted to the monitoring terminal and the time at which the transmitted monitoring image is captured are stored.

The image processing method of the video cluster monitoring system according to an exemplary embodiment of the present invention may further include: and sending a feedback message to the photovoltaic power station sending the data information through the monitoring terminal after the monitoring terminal presents the monitoring image corresponding to the extracted data information. As an example, the feedback message may include, but is not limited to, a second time stamp indicating a time at which the monitoring image as the reference image was captured and identification information indicating a camera device that captured the monitoring image as the reference image.

In this case, the photovoltaic power station that transmits the data information determines, based on the received feedback message, a monitoring image captured by the image pickup device at a time corresponding to the second timestamp, which is indicated by the identification information in the feedback message, as a reference image of the image pickup device for similarity comparison.

As shown in fig. 7, a video cluster monitoring system for a photovoltaic power plant according to an exemplary embodiment of the present invention includes: a plurality of photovoltaic power stations and a monitoring terminal 200. For example, the plurality of photovoltaic power plants may include photovoltaic power plant 100-1, photovoltaic power plant 100-2, … …, photovoltaic power plant 100-N, where N represents the number of the plurality of photovoltaic power plants.

Each photovoltaic power station includes a plurality of image pickup devices and a plurality of sub-processing units respectively corresponding to the plurality of image pickup devices. For example, the plurality of image pickup devices may include the image pickup device 11, the image pickup device 21, … …, and the image pickup device n1, and the plurality of sub-processing units may include the sub-processing unit 12, the sub-processing units 22, … …, and the sub-processing unit n2, where n denotes the number of the plurality of image pickup devices (also the number of the plurality of sub-processing units).

Suppose that an owner or a professional operation and maintenance company of a photovoltaic power station manages a plurality of photovoltaic power stations distributed in multiple places, and each photovoltaic power station is provided with a plurality of camera devices which are respectively arranged at different positions in the photovoltaic power station and used for monitoring the operation condition of a plant area of the photovoltaic power station. For example, the monitoring image captured by the camera device may be a digital image, which is saved locally in the photovoltaic power station, and the digital image may be in a common format such as BMP, GIF, JPEG, or TIFF, for example, and the captured monitoring image may be read by other software or programs.

The frame rate at which each camera captures a surveillance image may be the same or different. The monitor terminal of the owner or the professional operation and maintenance company of the photovoltaic power station is used for displaying the monitor image, and multi-screen or split-screen display of the monitor image can be achieved.

The multiple cameras are used for capturing monitoring images of different monitored scenes in the corresponding photovoltaic power stations, and each sub-processing unit is configured to execute the following processing aiming at the corresponding camera. Each sub-processing unit performs a similarity determination process for the monitoring image captured by the corresponding imaging device.

Here, the image processing method of the photovoltaic power plant video cluster monitoring system shown in fig. 1 may be executed in any of the sub-processing units shown in fig. 7, and it should be understood that the image processing method shown in fig. 1 is a flowchart for performing similarity determination on a monitored image captured by any of a plurality of image capturing apparatuses. Each sub-processing unit can execute the method shown in fig. 1 for the monitoring images captured by the corresponding camera device, so as to judge the similarity of the monitoring images of all the photovoltaic power stations.

Taking the photovoltaic power plant 100-1 as an example, the sub-processing unit 12 is configured to: the method comprises the steps of acquiring a monitoring image of a preset monitored scene captured by a corresponding camera device, determining a difference image of the captured monitoring image and a reference image of the camera device, determining at least one foreground subgraph based on the determined difference image, and determining a similarity index of the captured monitoring image and the reference image based on the at least one foreground subgraph. Here, the at least one foreground sub-image is used to reflect image differences between the captured monitor image and the reference image.

For example, the process of determining a difference image of the captured monitor image and the reference image of the image pickup device may include: the captured monitoring image and the reference image are subjected to a difference operation to obtain a difference image. Here, the number of pixel points in the X-axis direction and the Y-axis direction in the two-dimensional image space of the captured monitor image and the reference image is the same.

Based on the determined difference image, the process of determining at least one foreground subgraph may comprise: carrying out binarization processing on the determined difference image to obtain a binarization difference image; and determining an image formed by adjacent pixel points with the pixel value of 1 in the obtained binarization difference image as a foreground sub-image so as to obtain at least one foreground sub-image in the binarization difference image.

Based on the at least one foreground sub-graph, the process of determining a similarity index of the captured monitoring image to the reference image may comprise: calculating the area of each foreground subgraph; determining a similarity index of the captured monitoring image to the reference image based on the areas of all foreground subgraphs and the area of the captured monitoring image.

The process of determining a similarity index of the captured monitoring image to the reference image based on the areas of all foreground subgraphs and the area of the captured monitoring image may comprise: calculating the sum of the areas of all the foreground subgraphs; calculating the ratio of the sum of the areas of all the foreground subgraphs to the area of the captured monitoring image; and determining the difference value of 1 and the calculated ratio as the similarity index of the captured monitoring image and the reference image.

In a preferred embodiment, the photovoltaic power plant 100-1 may further include a first transmitting unit 30, which transmits the data information of the captured monitoring image to the monitoring terminal to present the monitoring image at the monitoring terminal when the determined similarity index is less than the similarity threshold.

It should be understood that the monitoring images of all the cameras arranged in the photovoltaic power station can be transmitted to a remote monitoring terminal through a network for storage and display.

In a preferred embodiment, the first transmitting unit 30 may store the data information of the captured monitoring image as a notification message in a transmission queue, and transmit each notification message in the transmission queue to the monitoring terminal according to the transmission priority of each monitoring image.

For example, at least one notification message is stored in the transmission queue, and the at least one notification message may include data information of a monitored image of which a similarity index determined by performing similarity determination on a monitored image captured by each image pickup device and a corresponding reference image is greater than a similarity threshold.

The first transmitting unit 30 may determine the transmission priority of any one of the monitored images by: determining at least one designated area in any one monitored image; determining the number of pixel points of at least one foreground subgraph corresponding to any monitoring image falling into at least one designated area; setting a corresponding area correction coefficient for each designated area; and determining the transmission priority of any monitoring image according to the determined number of the pixel points falling into each designated area and the area correction coefficient corresponding to each designated area.

Preferably, the at least one designated area is an area containing different scenes in any one of the monitored images, and each designated area is not overlapped with each other.

As an example, the first sending unit 30 may calculate a difference between the sum of the number of all the pixel points in any one of the monitoring images and the number of the pixel points falling into each of the designated areas; aiming at each appointed region, calculating the product of the number of pixel points falling into the appointed region and a region correction coefficient corresponding to the appointed region; calculating the sum of products corresponding to all the designated areas; and determining the ratio of the sum of the products obtained by calculation to the sum of the difference values to the sum of the number of all pixel points in any one monitoring image as the transmission priority of any one monitoring image.

In the exemplary embodiment of the present invention, the first transmitting unit 30 may transmit the entire image of the captured monitoring image to the monitoring terminal, and in addition, in order to reduce the occupation of the network bandwidth and increase the rate of image update, only a partial image of the captured monitoring image may be transmitted to the monitoring terminal.

In the first case, the data information may include complete image information of the captured monitoring image.

In the second case, the data information may include data information of a minimum segmentation map in the captured monitoring image.

Here, the minimum segmentation map may refer to a minimum image extracted from the captured monitoring image that contains a region where at least one foreground sub-image is located. Preferably, the minimum segmentation graph may be a minimum bounding rectangle containing a region in which the at least one foreground subgraph is located.

In this case, the data information of the minimum segmentation map may include, but is not limited to, image information, location information, identification information, and a time stamp of the minimum segmentation map.

For example, the position information may indicate a position of the minimum segmentation map in the captured monitoring image, the identification information may indicate a camera device that captured the monitoring image to which the minimum segmentation map belongs, the time stamp may include a first time stamp that may indicate a time when the monitoring image to which the minimum segmentation map belongs is captured and a second time stamp that may indicate a time when a reference image of the camera device used to capture the monitoring image to which the minimum segmentation map belongs is captured.

The monitoring terminal 200 in the photovoltaic power plant video cluster monitoring system according to the exemplary embodiment of the present invention may include a first receiving unit 50, a processor 60, and a display unit 70.

For example, the first receiving unit 50 receives data information of a monitoring image from each of a plurality of photovoltaic power stations, respectively, and stores the data information of the monitoring image received from each photovoltaic power station in a receiving queue.

The processor 60 may be configured to: and extracting the data information of the monitoring images from the receiving queue according to the transmission priority of each monitoring image, and acquiring the monitoring images corresponding to the extracted data information according to the extracted data information of the monitoring images.

The display unit 70 displays a monitoring image corresponding to the extracted data information under the control of the processor 60.

As an example, the display unit 70 may support a split-screen display function, for example, one small screen corresponding to displaying a monitoring image captured by one camera. However, the invention is not limited to this, and a portable device with a display screen, such as a smart phone or a tablet computer, can also be used as a display unit, so that operation and maintenance personnel can also view the monitoring image of the plant area of the photovoltaic power station outdoors at any time.

In a preferred embodiment, the monitoring terminal 200 in the photovoltaic power plant video cluster monitoring system according to an exemplary embodiment of the present invention may further include a memory 80 that stores the monitoring image presented on the display unit 70 as a reference image and stores the time at which the presented monitoring image is captured as a second time stamp.

For example, the processing of obtaining the monitoring image corresponding to the extracted data information from the extracted data information of the monitoring image may include: acquiring a reference image corresponding to a second time stamp from the memory 80 according to the second time stamp in the data information of the monitoring image; zeroing pixel values of positions indicated by position information in the data information of the acquired reference image and the monitoring image to obtain a background image; and synthesizing the minimum segmentation image and the background image to obtain a monitoring image corresponding to the extracted data information.

In a preferred embodiment, the first transmitting unit 30 in each photovoltaic power plant may further transmit the complete image information of the monitoring image captured by each camera to the monitoring terminal 200 at a predetermined cycle, and the memory 80 further stores the monitoring image with the complete image information as a reference image. In addition, after the first receiving unit 50 receives the monitoring image having the complete image information, the processor 60 may also control the display unit 70 to directly display the monitoring image.

For the above case of storing the reference image, the processor 60 may obtain, at predetermined time intervals, the reference image from the memory 80 at the time when the monitoring image is captured and closest to the second time stamp, according to the second time stamp in the data information of the monitoring image; zeroing pixel values of positions indicated by position information in the acquired data information of the reference image and the monitoring image to obtain a background image; and synthesizing the minimum segmentation image and the background image to obtain a monitoring image corresponding to the extracted data information.

In a preferred embodiment, each photovoltaic power station in the video cluster monitoring system for photovoltaic power stations according to the exemplary embodiment of the present invention may further include: a second receiving unit 40 and a plurality of storage units.

The plurality of storage units correspond to the plurality of camera devices respectively and are used for storing complete image information of the monitoring images which are captured by the corresponding camera devices and sent to the monitoring terminal and the capturing time of the sent monitoring images. For example, the plurality of memory cells may include memory cell 13, memory cell 23, … …, and memory cell n 3. The second receiving unit 40 is configured to receive a feedback message from the monitoring terminal 200.

For example, a buffer queue may be used to store each frame of monitoring image captured by a corresponding camera in a storage unit of the photovoltaic power plant, and one buffer queue may correspond to one camera. When a monitoring image enters the buffer queue, a first time stamp indicating the moment when the monitoring image is captured is attached. Since the frame rates at which the monitoring images are captured by different image pickup apparatuses are different, the number of monitoring images stored in different buffer queues at the same time may be different. For example, assuming that the sampling rate of the camera device 11 of the photovoltaic power plant 100-1 is the fastest and the sampling rate of the camera device 21 of the photovoltaic power plant 100-1 is the slowest, three monitoring images may exist in the buffer queue corresponding to the camera device 11, and only one monitoring image may exist in the buffer queue corresponding to the camera device 21.

When the feedback message is received from the remote monitoring terminal, the corresponding camera device is determined based on the identification information in the feedback message, and the monitoring image which is captured by the determined camera device and corresponds to the second timestamp is searched from the storage unit based on the second timestamp in the feedback message to be used as a reference image of the camera device for subsequent image difference operation.

It should be understood that the monitoring image stored in the storage unit is different from the monitoring image entered into the transmission queue in that the monitoring image stored in the storage unit is a complete monitoring image captured by the camera, and the monitoring image entered into the transmission queue may be a complete monitoring image or a minimum segmentation map of the monitoring image. As an example, the storage unit may take the form of a queue on the basis of a stored mathematical structure, and the maximum length of the queue is the sum of the lengths of the transmission queue of the photovoltaic power station and the receiving queue of the remote monitoring terminal.

In a preferred embodiment, the monitoring terminal 200 may further include: and a second transmitting unit 90 for transmitting a feedback message to the photovoltaic power station transmitting the data information after the display unit 70 presents the monitoring image corresponding to the extracted data information. Here, the feedback message may include, but is not limited to, a second time stamp indicating a time at which the monitoring image as the reference image was captured and identification information indicating the camera device that captured the monitoring image as the reference image.

At this time, the second receiving unit 40 in the photovoltaic power station sending the data information receives the feedback message, and the sub-processing unit in the photovoltaic power station sending the data information determines the monitoring image captured by the camera at the time corresponding to the second timestamp and indicated by the identification information in the feedback message as a reference image of the camera for similarity comparison.

There is also provided, in accordance with an exemplary embodiment of the present invention, a computer-readable storage medium storing a computer program. The computer readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the image processing method of the photovoltaic power plant video cluster monitoring system described above. The computer readable recording medium is any data storage device that can store data read by a computer system. Examples of the computer-readable recording medium include: read-only memory, random access memory, read-only optical disks, magnetic tapes, floppy disks, optical data storage devices, and carrier waves (such as data transmission through the internet via wired or wireless transmission paths).

According to the video cluster monitoring system of the photovoltaic power station and the image processing method thereof in the exemplary embodiment of the invention, a monitoring image remote transmission and display method supporting remote management of the photovoltaic power station is provided for the current situation that a monitoring terminal of an owner or a professional operation and maintenance company of the photovoltaic power station cannot display a monitoring image captured by a camera device arranged in a plant area of the photovoltaic power station.

By adopting the photovoltaic power station video cluster monitoring system and the image processing method thereof in the exemplary embodiment of the invention, remote transmission and display of the monitoring image are realized, the informatization level of the photovoltaic power station can be improved, and the working efficiency of operation and maintenance personnel and the management level of related management personnel can be improved on the basis of the informatization level of the photovoltaic power station.

In addition, by adopting the photovoltaic power station video cluster monitoring system and the image processing method thereof in the exemplary embodiment of the invention, under the condition of limited network bandwidth and transmission and display capacity, important monitoring images in each photovoltaic power station are displayed in a remote monitoring terminal within as short delay time as possible.

In addition, in order to reduce redundancy of monitoring image transmission data, the photovoltaic power station video cluster monitoring system and the image processing method thereof according to the exemplary embodiment of the present invention adopt two methods to reduce data transmission amount. Firstly, based on similarity judgment, sending a monitoring image which is captured by a camera device and has a larger image difference with a reference image to a monitoring terminal; and secondly, further dividing the monitoring image to be transmitted, and sending the minimum division image of the monitoring image to be transmitted to the monitoring terminal through the network.

In addition, in order to reduce the influence of network transmission delay and enable operation and maintenance personnel in a remote monitoring room to know the important situation occurring in the photovoltaic power station as early as possible, in the exemplary embodiment of the invention, the data information of the monitoring terminals in the sending queue of the local photovoltaic power station and the receiving queue of the monitoring terminal is sorted according to the transmission priority aiming at different situations, so that the data information of the monitoring image with high transmission priority can be sent or received preferentially.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as defined by the following claims.

Claims

1. An image processing method of a photovoltaic power station video cluster monitoring system comprises a monitoring terminal and a plurality of photovoltaic power stations, wherein each photovoltaic power station is provided with a plurality of camera devices for capturing monitoring images of different monitored scenes in the corresponding photovoltaic power station, and the image processing method comprises the following steps: performing the following for each of a plurality of image pickup devices arranged in each photovoltaic power plant:

acquiring a monitoring image of a preset monitored scene captured by any one camera device aiming at the any one camera device;

determining a difference image of the captured monitoring image and a reference image of the arbitrary one of the image pickup devices;

determining at least one foreground sub-image based on the determined difference image, wherein the at least one foreground sub-image is used for reflecting image difference between the captured monitoring image and the reference image;

determining a similarity index of the captured monitoring image to the reference image based on the at least one foreground sub-image;

and if the similarity index is smaller than a similarity threshold value, sending the data information of the captured monitoring image to a monitoring terminal so as to present the monitoring image on the monitoring terminal.

2. The image processing method according to claim 1, wherein the step of determining a difference image of the captured monitor image and the reference image of the arbitrary one of the image pickup devices comprises: performing a difference operation on the captured monitoring image and the reference image to obtain a difference image,

the number of pixel points of the captured monitoring image and the reference image in the two-dimensional image space along the X-axis direction and the Y-axis direction is the same.

3. The image processing method of claim 1, wherein the step of determining at least one foreground subgraph based on the determined difference image comprises:

carrying out binarization processing on the determined difference image to obtain a binarization difference image;

and determining an image formed by adjacent pixel points with the pixel value of 1 in the obtained binarization difference image as a foreground sub-image so as to obtain at least one foreground sub-image in the binarization difference image.

4. The image processing method of claim 3, wherein determining a similarity index of the captured monitoring image to the reference image based on the at least one foreground subgraph comprises:

calculating the area of each foreground subgraph;

determining a similarity index of the captured monitoring image to the reference image based on the areas of all foreground subgraphs and the area of the captured monitoring image.

5. The image processing method of claim 4, wherein the step of determining a similarity index of the captured monitoring image to the reference image based on the areas of all foreground subgraphs and the area of the captured monitoring image comprises:

calculating the sum of the areas of all the foreground subgraphs;

calculating the ratio of the sum of the areas of all the foreground subgraphs to the area of the captured monitoring image;

and determining the difference value of 1 and the calculated ratio as the similarity index of the captured monitoring image and the reference image.

6. The image processing method according to claim 1, wherein the step of transmitting the data information of the captured monitoring image to the monitoring terminal comprises:

storing the data information of the captured monitoring image as a notification message into a sending queue, wherein at least one notification message is stored in the sending queue, and the at least one notification message is the data information of the monitoring image of which the similarity index determined by judging the similarity between the monitoring image captured by each camera device and the corresponding reference image is smaller than the similarity threshold value;

and sending each notification message in the sending queue to the monitoring terminal according to the transmission priority of each monitoring image.

7. The image processing method according to claim 6, wherein the transmission priority of any one of the monitor images is determined by:

determining at least one designated area in the any one monitored image;

determining the number of pixel points of at least one foreground subgraph corresponding to any one monitoring image falling into at least one designated area;

setting a corresponding area correction coefficient for each designated area;

and determining the transmission priority of any monitoring image according to the determined number of the pixel points falling into each designated area and the area correction coefficient corresponding to each designated area.

8. The image processing method according to claim 7, wherein the step of determining the transmission priority of the arbitrary one of the monitor images based on the determined number of pixel points falling in each of the designated areas and the area correction coefficient corresponding to each of the designated areas comprises:

calculating the difference value between the sum of the number of all pixel points in any one monitoring image and the number of the pixel points falling into each designated area;

aiming at each appointed region, calculating the product of the number of pixel points falling into the appointed region and a region correction coefficient corresponding to the appointed region;

calculating the sum of products corresponding to all the designated areas;

and determining the ratio of the sum of the products obtained by calculation to the sum of the difference values to the sum of the number of all pixel points in any one monitoring image as the transmission priority of any one monitoring image.

9. The image processing method according to claim 7, wherein the at least one designated area is an area containing different scenes in the arbitrary monitored image, and each designated area is not overlapped with each other.

10. The image processing method according to claim 1, wherein the data information includes data information of a complete image of the captured monitoring image or a minimum segmentation map in the captured monitoring image,

wherein the minimum segmentation map is a minimum image extracted from the captured monitoring image and containing a region where the at least one foreground sub-image is located.

11. The image processing method according to claim 10, wherein the minimum segmentation map is a minimum bounding rectangle containing a region in which the at least one foreground sub-map is located,

wherein the data information of the minimum segmentation map includes image information, location information, identification information, and a time stamp of the minimum segmentation map,

the position information indicates a position of the minimum segmentation map in the captured monitoring image, the identification information indicates a camera device that captured the monitoring image to which the minimum segmentation map belongs, the time stamp includes a first time stamp indicating a time when the monitoring image to which the minimum segmentation map belongs is captured and a second time stamp indicating a time when a reference image of the camera device used to capture the monitoring image to which the minimum segmentation map belongs is captured.

12. The image processing method according to claim 11, further comprising:

respectively receiving data information of monitoring images from each photovoltaic power station in the plurality of photovoltaic power stations through the monitoring terminal, and storing the data information of the monitoring images received from each photovoltaic power station into a receiving queue;

extracting data information of the monitoring images from the receiving queue according to the transmission priority of each monitoring image;

and displaying the monitoring image corresponding to the extracted data information on the monitoring terminal according to the extracted data information of the monitoring image.

13. The image processing method according to claim 12, further comprising: storing, by the monitor terminal, a monitor image presented at the monitor terminal as a reference image, and storing a time at which the presented monitor image is captured as a second time stamp,

the step of presenting the monitoring image corresponding to the extracted data information at the monitoring terminal according to the extracted data information of the monitoring image comprises the following steps:

acquiring a reference image corresponding to a second time stamp from the monitoring terminal according to the second time stamp in the data information of the monitoring image;

zeroing pixel values of positions indicated by position information in the data information of the acquired reference image and the monitoring image to obtain a background image;

and synthesizing the minimum segmentation image and the background image to obtain a monitoring image, and presenting the monitoring image obtained by synthesis at the monitoring terminal.

14. The image processing method according to claim 13, further comprising: each photovoltaic power station transmits complete image information of the monitoring image captured by each camera device to the monitoring terminal at a predetermined period so as to store the monitoring image with the complete image information as a reference image,

wherein, the step of presenting the monitoring image corresponding to the extracted data information at the monitoring terminal according to the extracted data information of the monitoring image further comprises:

acquiring a reference image which is closest to a second timestamp at the moment when the monitoring image is captured from the monitoring terminal according to the second timestamp in the data information of the monitoring image at a preset time interval;

zeroing pixel values of positions indicated by position information in the acquired data information of the reference image and the monitoring image to obtain a background image;

15. The image processing method according to claim 13, further comprising: in each photovoltaic power station, storing complete image information of the monitoring image transmitted to the monitoring terminal and the time when the transmitted monitoring image is captured,

wherein the image processing method further comprises:

transmitting, by the monitoring terminal, a feedback message to the photovoltaic power station that transmits the data information after the monitoring terminal presents the monitoring image corresponding to the extracted data information, wherein the feedback message includes a second timestamp indicating a time at which the monitoring image as the reference image was captured and identification information indicating a camera that captured the monitoring image as the reference image,

and the photovoltaic power station sending the data information determines the monitoring image captured by the camera device at the moment corresponding to the second timestamp and indicated by the identification information in the feedback message as a reference image of the camera device based on the received feedback message, so as to be used for similarity comparison.

16. The utility model provides a photovoltaic power plant video cluster monitored control system which characterized in that, includes monitor terminal and a plurality of photovoltaic power plant, every photovoltaic power plant includes:

a plurality of camera devices for capturing monitoring images of different monitored scenes in corresponding photovoltaic power stations,

a plurality of sub-processing units respectively corresponding to the plurality of image pickup apparatuses, each sub-processing unit configured to execute the following processing for the corresponding image pickup apparatus:

acquiring a monitoring image of a predetermined monitored scene captured by the corresponding camera device,

determining a difference image of the captured monitoring image and a reference image of the camera device,

determining at least one foreground sub-graph based on the determined difference image, wherein the at least one foreground sub-graph is used to reflect image differences between the captured monitoring image and the reference image,

determining a similarity index of the captured monitoring image to the reference image based on the at least one foreground sub-image,

and the first sending unit is used for sending the data information of the captured monitoring image to the monitoring terminal when the similarity index is smaller than the similarity threshold value so as to present the monitoring image on the monitoring terminal.

17. The photovoltaic power plant video cluster surveillance system of claim 16, wherein the process of determining a difference image of the captured surveillance image and the reference image of the camera device comprises: performing a difference operation on the captured monitoring image and the reference image to obtain a difference image,

18. The photovoltaic power plant video cluster monitoring system of claim 16, wherein the process of determining at least one foreground sub-graph based on the determined difference image comprises:

19. The photovoltaic power plant video cluster monitoring system of claim 18, wherein the process of determining a similarity index of the captured monitoring image to the reference image based on the at least one foreground subgraph comprises:

calculating the area of each foreground subgraph;

20. The photovoltaic power plant video cluster monitoring system of claim 19, wherein the process of determining a similarity index of the captured monitoring image to the reference image based on the areas of all foreground subgraphs and the area of the captured monitoring image comprises:

calculating the sum of the areas of all the foreground subgraphs;

21. The photovoltaic power plant video cluster monitoring system of claim 16 wherein the first transmitting unit stores the data information of the captured monitoring image as a notification message in a transmitting queue, transmits each notification message in the transmitting queue to the monitoring terminal according to the transmission priority of each monitoring image,

at least one notification message is stored in the sending queue, wherein the at least one notification message is data information of the monitoring image of which the similarity index determined by performing similarity judgment on the monitoring image captured by each camera and the corresponding reference image is smaller than the similarity threshold.

22. The photovoltaic power plant video cluster monitoring system of claim 21 wherein the first sending unit determines the transmission priority of any one of the monitored images by:

determining at least one designated area in the any one monitored image;

setting a corresponding area correction coefficient for each designated area;

23. The photovoltaic power plant video cluster monitoring system of claim 22 wherein the first sending unit calculates the difference between the sum of the number of all pixels in any one of the monitored images and the number of pixels falling within each designated area; aiming at each appointed region, calculating the product of the number of pixel points falling into the appointed region and a region correction coefficient corresponding to the appointed region; calculating the sum of products corresponding to all the designated areas; and determining the ratio of the sum of the products obtained by calculation to the sum of the difference values to the sum of the number of all pixel points in any one monitoring image as the transmission priority of any one monitoring image.

24. The video cluster monitoring system for photovoltaic power plants according to claim 22, wherein said at least one designated area is an area containing different scenes in any one of said monitored images, and each designated area is not overlapped with each other.

25. The photovoltaic power plant video cluster monitoring system of claim 16 wherein the data information includes complete image information of the captured monitoring images, or data information of a minimal segmentation map in the captured monitoring images,

26. The photovoltaic power plant video cluster monitoring system of claim 25 wherein the minimum segmentation graph is a minimum bounding rectangle containing the area where the at least one foreground sub-graph is located,

27. The photovoltaic power plant video cluster monitoring system of claim 26, wherein the monitoring terminal comprises:

the first receiving unit is used for respectively receiving the data information of the monitoring image from each photovoltaic power station in the plurality of photovoltaic power stations and storing the data information of the monitoring image received from each photovoltaic power station into a receiving queue;

the processor is configured to: extracting data information of the monitoring images from the receiving queue according to the transmission priority of each monitoring image, and acquiring the monitoring images corresponding to the extracted data information according to the extracted data information of the monitoring images;

and a display unit displaying the monitoring image corresponding to the extracted data information.

28. The photovoltaic power plant video cluster monitoring system of claim 27 wherein the monitoring terminal further comprises: a memory that stores a monitoring image presented at the monitoring terminal as a reference image and stores a time when the presented monitoring image is captured as a second time stamp,

wherein, according to the data information of the extracted monitoring image, the processing of obtaining the monitoring image corresponding to the extracted data information comprises:

acquiring a reference image corresponding to a second time stamp from a memory according to the second time stamp in the data information of the monitoring image;

and synthesizing the minimum segmentation image and the background image to obtain a monitoring image corresponding to the extracted data information.

29. The photovoltaic power plant video cluster monitoring system of claim 28 wherein the first transmitting unit further transmits complete image information of the monitoring image captured by each camera to the monitoring terminal at a predetermined cycle, the memory further stores the monitoring image with the complete image information as a reference image,

wherein, according to the data information of the monitoring image obtained by extraction, the processing of obtaining the monitoring image corresponding to the extracted data information further comprises:

acquiring a reference image which is closest to a second time stamp at the moment when the monitoring image is captured from a memory according to the second time stamp in the data information of the monitoring image at preset time intervals;

30. The photovoltaic power plant video cluster monitoring system of claim 28 wherein each photovoltaic power plant further comprises: a plurality of storage units respectively corresponding to the plurality of image pickup devices for storing complete image information of the monitoring image transmitted to the monitoring terminal captured by the corresponding image pickup device and a time at which the transmitted monitoring image is captured,

wherein, monitor terminal still includes: a second transmission unit that transmits a feedback message to the photovoltaic power station that transmits the data information after the monitoring terminal presents the monitoring image corresponding to the extracted data information, wherein the feedback message includes a second timestamp indicating a time at which the monitoring image as the reference image was captured and identification information indicating an image pickup device that captured the monitoring image as the reference image,

wherein, every photovoltaic power plant still includes: a second receiving unit for receiving a feedback message from the monitoring terminal,

and the sub-processing unit in the photovoltaic power station sending the data information determines the monitoring image captured by the camera device indicated by the identification information in the feedback message at the moment corresponding to the second timestamp as a reference image of the camera device for similarity comparison.

31. A computer-readable storage medium storing a computer program which, when executed by a processor, implements the image processing method of a photovoltaic power plant video cluster monitoring system of any one of claims 1-15.