CN115170786A

CN115170786A - Electronic device and method for detecting movement of road protection object

Info

Publication number: CN115170786A
Application number: CN202210712947.XA
Authority: CN
Inventors: 侯彬; 郭颖; 刘琦; 崔淑铭; 王焕新; 杜昭; 陈石杰; 王海龙
Original assignee: Hisense TransTech Co Ltd
Current assignee: Hisense TransTech Co Ltd
Priority date: 2022-06-22
Filing date: 2022-06-22
Publication date: 2022-10-11

Abstract

The invention provides an electronic device and a method for detecting movement of a road protector, which relate to the technical field of intelligent public transport and comprise the following steps: responding to a monitoring starting instruction, and determining an early warning area in at least one target image containing the target road protection object according to the image position of the early warning area in the template image containing the target road protection object which is not deviated; determining target pixel points representing a target road protector in the early warning area of each target image according to the RGB value and the RGB threshold value of the pixel points in the early warning area of each target image; if the number of target pixel points in the early warning area of at least one target image meets a preset condition, determining that the position of the target road protector deviates; and informing the road protection personnel that the position of the target road protection object deviates. According to the embodiment of the invention, the problem of whether the road protection object in the shot image has deviation or not is obtained by comparing the shot image with the template image, and the processing speed is improved.

Description

Electronic device and method for detecting movement of road protection object

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to an electronic device and a method for detecting movement of a road protection object.

Background

With the increasing of urban vehicles, the road protectors such as guardrails and anti-collision barrels have increasingly obvious effects in road traffic safety, and after the road protectors such as the guardrails and the anti-collision barrels are deviated due to collision, if the road protectors cannot be repaired in time, the traffic problems that pedestrians pass through roads illegally and the buffering effect is weakened when vehicles collide can be caused, and the traffic safety and the traffic efficiency of roads are seriously threatened.

Usually, the road maintenance personnel regularly patrol on the road, discover the removal problem of road protection thing such as guardrail and anticollision bucket, however because road protection thing such as road guardrail and anticollision bucket is huge in quantity, often unable very first time discovery skew problem is patrolled in daily manual work, and the manual mode has the problem inefficiency, consuming time is many simultaneously, because can not in time restore road protection thing such as guardrail and anticollision bucket, just also can not play the effect of safety guarantee.

Disclosure of Invention

The invention provides an electronic device and a method for detecting the movement of a road protector, which can solve the problem of whether the road protector in a shot image has deviation or not by comparing the shot image with a template image, and improve the processing speed.

In a first aspect, an embodiment of the present invention provides an electronic device, including: a processor and a communication unit;

the processor is used for responding to the monitoring starting instruction and determining an early warning area in at least one target image containing the target road protector according to the image position of the early warning area in the template image containing the target road protector which does not deviate; the early warning area is the area around the area where the target road protection object is located in the target image;

determining target pixel points representing a target road protector in the early warning area of each target image according to the RGB value and the RGB threshold value of the pixel points in the early warning area of each target image; wherein the RGB threshold value is determined from RGB values of the target road guard;

if the number of target pixel points in the early warning area of at least one target image meets a preset condition, determining that the position of the target road protector deviates; the preset condition is determined according to the number of target pixel points in the early warning area of the template image;

and the communication unit is used for informing the road protection personnel that the position of the target road protection object deviates.

The electronic equipment can determine the early warning area in at least one target image according to the image position of the early warning area in the template image containing the target road protection object which does not deviate, find the target pixel points in the early warning area according to the RGB value and the RGB threshold value of the pixel points in the early warning area, meanwhile, refer to the number of the target pixel points in the template image, judge whether the position of the target road protection object deviates, inform road protection personnel if the position of the target road protection object deviates, determine whether the target road protection object deviates in the shot image according to the condition of the target road protection object in the template image, and improve the processing speed compared with manual inspection.

In one possible implementation, the processor is specifically configured to:

if the target road protector contains one color, searching pixel points of which the difference value between the RGB value and the RGB threshold value is within a first preset range from the early warning area of each target image, and taking the searched pixel points as target pixel points; wherein the RGB threshold value is determined according to an RGB value corresponding to one color of the target road protection object;

if the target road protection object contains multiple colors, searching a pixel point of which the difference value between the RGB value and any RGB threshold value is within a second preset range from the early warning area of each target image, and taking the searched pixel point as a target pixel point; wherein each RGB threshold is determined according to an RGB value corresponding to each color of the target road guard.

The electronic equipment can determine the target pixel points aiming at the target road protectors containing different RGB values, and the accuracy of determination is improved.

In one possible implementation, the processor is further configured to:

determining the ratio between target pixel points corresponding to each RGB threshold in the early warning area of at least one target image, wherein the difference value between the ratio and the second preset ratio is not in a third preset range; and determining the second preset ratio according to the ratio between target pixel points corresponding to each RGB threshold in the early warning area of the template image.

Above-mentioned electronic equipment can also compare through the ratio of the target pixel point that different RGB values correspond and predetermined ratio when having the target road protection thing of different RGB values, further confirms whether take place the skew to the accuracy of confirming has been improved.

In a possible implementation manner, the processor is specifically configured to:

determining the total number of target pixel points of the same image position of each target subregion in a plurality of target images; the target sub-region is a region at the same image position in an early warning region at the same image position in each target image;

determining the ratio of the total number corresponding to each target subregion to the total number of pixel points of each target subregion;

if the ratio of any one target subregion is not within the preset range corresponding to the target subregion, determining that the number of target pixel points in the early warning region of at least one target image meets a preset condition; the preset range corresponding to the target subregion is determined according to the ratio of the target pixel point of the subregion corresponding to the target subregion in the template image to the total number of the pixel points of the corresponding subregion in the template image.

Above-mentioned electronic equipment can be through being merging of target pixel point between the subregion, compares the ratio between the total number of pixel point of total number and subregion after it merges with the definite ratio of template image to confirm whether the target protection thing takes place the skew, thereby can more accurate confirm whether the protection thing takes place the skew the condition.

In one possible implementation, the processor is specifically configured to:

aiming at each target image, dividing an image position of the target image, which is the same as the image position of the early warning area of the template image, into an area serving as the early warning area in the target image; or alternatively

Aiming at each target image, taking an area out of the divided area at the image position of the target image, which is the same as the image position of the target road shield which does not deviate in the template image, as an early warning area in the target image; or

For each target image, dividing an area at the image position where the target image and the template image contain the target road protector which is not subjected to deviation, wherein the image position is the same as the image position where the target road protector is located, and taking the divided area as the area containing the target road protector in the target image;

and determining the early warning area in the target image according to the proportion between the area containing the target road protection object which is not deviated and the early warning area in the template image and the area containing the target road protection object in the target image.

The electronic equipment can determine the early warning area in the target image according to the position of the early warning area in the template image, or determine the early warning area in the target image according to the position of the target road protector in the template image, so that the early warning area can be determined more quickly when the template image and the target image have the same resolution, or determine the early warning area according to the ratio between the area of the target road protector which does not deviate and the early warning area in the template image, so that the early warning area can be determined more quickly when the template image and the target image have different resolutions.

In a second aspect, an embodiment of the present invention provides a method for detecting movement of a road shield, including:

responding to a monitoring starting instruction, and determining an early warning area in at least one target image containing the target road protection object according to the image position of the early warning area in the template image containing the target road protection object which is not deviated; the early warning area is the area around the area where the target road protection object is located in the target image;

determining target pixel points representing a target road protection object in the early warning area of each target image according to the RGB values and the RGB threshold values of the pixel points in the early warning area of each target image; wherein the RGB threshold value is determined from RGB values of the target road guard;

if the number of target pixel points in the early warning area of at least one target image meets a preset condition, determining that the position of the target road protector deviates; the preset condition is determined according to the number of target pixel points in an early warning area of a template image containing the target road protector which is not subjected to deviation;

and informing the road protection personnel that the position of the target road protection object deviates.

In a possible implementation manner, determining target pixel points representing a target road protection object in the early warning area of each target image according to the RGB values and the RGB threshold values of the pixel points in the early warning area of each target image includes:

if the target road protector contains multiple colors, searching pixel points of which the difference value between the RGB value and any one RGB threshold value is within a second preset range from the early warning area of each target image, and taking the searched pixel points as target pixel points; wherein each RGB threshold is determined according to the corresponding RGB value of each color of the target road protection.

In one possible implementation, before notifying the road protection personnel that the position of the target road protection object is offset, the method further includes:

In a possible implementation manner, it is determined that the number of target pixel points in the early warning region of at least one target image meets a preset condition by:

determining the total number of target pixel points of the same image position of each target subregion in a plurality of target images; the target sub-region is a region of the same image position in an early warning region of the same image position in each target image;

if the ratio of any one target subregion is not in the preset range corresponding to the target subregion, determining that the number of target pixel points in the early warning region of at least one target image meets a preset condition; the preset range corresponding to the target sub-region is determined according to the ratio of the target pixel point of the sub-region corresponding to the target sub-region in the template image to the total number of the pixel points of the sub-region corresponding to the template image.

In one possible implementation manner, determining the warning area in at least one target image containing the target road protection object according to the image position of the warning area in the template image containing the target road protection object without deviation comprises:

aiming at each target image, dividing an image position of the target image, which is the same as the image position of the early warning area of the template image, into an area serving as the early warning area in the target image; or

Aiming at each target image, taking an area out of the divided area at the image position of the target image, which is the same as the image position of the target road shield which does not deviate in the template image, as an early warning area in the target image; or alternatively

In a third aspect, embodiments of the present invention provide a storage medium, where when executed by a processor of an electronic device, instructions enable the electronic device to execute the method for detecting movement of a road guard according to any one of the second aspect.

In addition, for technical effects brought by any one implementation manner of the second aspect to the third aspect, reference may be made to technical effects brought by different implementation manners of the first aspect, and details are not described herein again.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.

Drawings

Fig. 1 is a schematic diagram of a scene of a road protection object according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of another example of capturing a scene of a road barrier according to an embodiment of the present invention;

fig. 3 is a flowchart of a method for detecting movement of a road protection object according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a template image including an unmoved target roadway shield according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of an image including moving target roadway shielding according to an embodiment of the present invention;

fig. 6 is a schematic diagram of an application of the movement detection method for a road protection object applied to a mobile phone according to an embodiment of the present invention;

fig. 7 is a schematic diagram of an application of a method for detecting movement of a road protection object applied to a device with a display function according to an embodiment of the present invention;

FIG. 8 is a schematic diagram illustrating interaction between a camera, a server, and a mobile phone of a road guard according to an embodiment of the present invention;

FIG. 9 is a schematic diagram of an early warning area and an area containing non-moving target roadway shields in a simulated image according to an embodiment of the present invention;

FIG. 10 is a schematic diagram of an early warning area and an area containing non-moving target roadway shields in another simulated image provided by an embodiment of the invention;

fig. 11 is a flowchart of a method for determining an early warning area in a target image according to an embodiment of the present invention;

FIG. 12 is a schematic diagram of an early warning area and an area including a target road protection object in a target image according to an embodiment of the present invention;

fig. 13 is a flowchart of a method for determining that the number of target pixel points in the early warning area of the target sub-area meets the preset condition according to the embodiment of the present invention;

FIG. 14 shows a picture F according to an embodiment of the present invention ₁ The sub-region searches the display of the target pixel pointAn intent;

fig. 15 is a schematic diagram illustrating comparison between pixel points in sub-regions of two target images according to an embodiment of the present invention;

fig. 16 is a schematic diagram illustrating comparison between pixel points in multiple sub-regions of two target images according to an embodiment of the present invention;

fig. 17 is a schematic diagram illustrating comparison between pixel points in sub-regions of two target images according to an embodiment of the present invention;

fig. 18 is a structural view of a movement detecting device for a road guard according to an embodiment of the present invention;

FIG. 19 is a block diagram of another alternative movement detection device for a road guard according to an embodiment of the present invention;

fig. 20 is a structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solution in the embodiments of the present application will be described in detail and removed with reference to the accompanying drawings. In the description of the embodiments herein, "/" means "or" unless otherwise specified, for example, a/B may mean a or B; "and/or" in the text is only an association relationship describing an associated object, and means that three relationships may exist, for example, a and/or B may mean: three cases of a alone, a and B both, and B alone exist, and in addition, "a plurality" means two or more than two in the description of the embodiments of the present application.

A road maintenance person may drive a vehicle, mount a camera 100 on the outside roof, and photograph a guard of the road, and as shown in fig. 1, the road maintenance person sits on the vehicle, and after seeing a guardrail 110 of the road, photograph the guard by the camera 100, and then perform image processing on the photographed image, to obtain whether the guardrail 110 is displaced. Similarly, as shown in fig. 2, when a road maintenance worker sits on a vehicle and sees the protection bucket 120 of the road, the road maintenance worker takes an image by using the camera 100, and then performs image processing on the taken image to obtain whether the protection bucket 120 is displaced.

An embodiment of the present invention provides an image processing method, and specifically, with reference to fig. 3, an embodiment of the present invention provides a method for detecting movement of a road shield, including:

s300: responding to a monitoring starting instruction, and determining an early warning area in at least one target image containing the target road protection object according to the image position of the early warning area in the template image containing the target road protection object which does not deviate;

the early warning area is the area around the area where the target road protection object is located in the target image; referring to fig. 4, a portion C1 is an area where a target road guard is located; the C2 portion (hatched portion) is an early warning region.

S301: determining target pixel points representing a target road protector in the early warning area of each target image according to the RGB value and the RGB threshold value of the pixel points in the early warning area of each target image; wherein the RGB threshold value is determined according to the RGB value of the target road protection object;

s302: if the number of target pixel points in the early warning area of at least one target image meets a preset condition, determining that the position of the target road protector deviates; the preset condition is determined according to the number of target pixel points in an early warning area of a template image containing a target road protector which is not subjected to deviation;

s303: and informing the road protection personnel that the position of the target road protection object is deviated.

If fig. 4 is a template image, compared with the template image, with reference to fig. 5, target pixel points representing the target road protector in the target image may be determined according to the RGB threshold, and then, according to the number of the target pixel points representing the target road protector in the early warning region of the target image, the number of the target pixel points in the early warning region of the template image (fig. 4) including the target road protector that has not undergone deviation is referred to, so that it is known that the target road protector is in a deviation state.

And starting the monitoring starting instruction after the set time is reached.

The method is exemplarily applied to a mobile phone, and as shown in fig. 6, mainly a road protection person may set a timing function on the mobile phone, when the road protection person sets 9 am on the mobile phone, the person drives to shoot images of a target road protection object, and after the images are shot, the images are transmitted to the mobile phone, when the time set by the road protection person is reached, target pixel points representing the road protection object in early warning areas in the images may be determined according to the transmitted images, based on the number of the target pixel points in the early warning areas in all the images, whether the road protection object on the road is shifted or not is determined, and if the road protection object is shifted, the road protection object is displayed. Certainly, when shooting is performed, names can be named for images of road guards of different sections of different roads, and when the road guards are judged to be offset, names and addresses of the offset road guards can be displayed, for example, 11 roads-road guard number 11201, the set time is 9 points, and after 9 points, if images are transmitted, processing can be performed to obtain a processing result and offset; the number 11203 of the 11-way road protection object is set to be 10 points, and after 10 points, if an image is transmitted, processing can be carried out to obtain a processing result without deviation; so that the road guard can process the 11-way road guard number 11201.

And triggering a monitoring starting instruction by shooting the image containing the road protection object.

Illustratively, referring to fig. 7, after clicking an oval button on the display-enabled device 700, the road protection staff controls the camera 100 to start capturing images of the road protection object, and then captures the images, and transmits the images to the display-enabled device 700, and determines target pixels representing the road protection object in the early warning areas in the images according to the transmitted images, and determines whether the road protection object on the road is shifted based on the number of the target pixels in the early warning areas in all the images, and if the target pixels are shifted, the road protection object is displayed. At this time, when the equipment with the display function displays whether the deviation result exists, the address of the road protection object does not need to be displayed because the shooting is finished for processing, so that the road protection personnel can process the road protection object in time. For example, a photograph is taken of a road protection object, and it is determined whether the road protection object is displaced, and if the road protection object is displaced, the road protection object can be handled by a road protection worker.

The user directly triggers the monitoring starting instruction.

Illustratively, referring to fig. 8, a camera 1 to a camera n, a server 800 and a plurality of mobile phones 1 to a mobile phone m of a road protection person are connected, the person drives to shoot an image of a target road protection object, when shooting, names can be given to images of road protection objects of different sections of different roads, then according to different names, each name represents a different road protection object, the image shot by the camera is uploaded to the server 800, the server 800 determines target pixel points representing the road protection object in an early warning area containing the image of the road protection object with the same name, and judges whether the image has an offset or not based on the number of the target pixel points in the early warning area, when the image has the offset, the server can send the offset result to the mobile phone of the road protection person corresponding to the name, and at the same time, send the name and address of the offset road protection object to the mobile phone of the road protection person corresponding to the name, so that the road protection person can process the road protection object according to the name and address of the road protection object.

Since the resolution of the template image and the resolution of the at least one target image may be the same or different, the early warning region in the target image and the early warning region in the template image have the same image position or different image positions, based on the above situation, the present invention proposes a manner of determining the early warning region in the at least one target image including the target road protection object according to the image position of the early warning region in the template image including the target road protection object that is not offset, and in the manner of 1, for each target image, the region divided at the image position in the target image that is the same as the image position where the early warning region of the template image is located is taken as the early warning region in the target image.

The resolution of the template image is the same as that of at least one target image, and the method 1 is more suitable, if the same coordinate system is established in the template image and the target image, the coordinate values of the pixel points at the same position are the same, so that the early warning area of the target image can be found at the same image position in the template image. With reference to fig. 9, a rectangular coordinate system is established at the lower left corner of the template image, the lower left corner of the template image is 0 point of the rectangular coordinate system, each pixel point in the template image has coordinate values of an x axis and a y axis, the coordinate values represent positions in the image, and if the transverse resolution of the template image is n and the longitudinal resolution is m, the image positions of the pixel points in the template image are:

the region containing the target road protector in the template image can be represented by coordinates of pixel points, and if the region containing the target road protector is rectangular, the region can be represented by coordinates of pixel points at four corners of the rectangle; for example, rectangles enclosed by (x 30, y 20), (x 50, y 20), (x 30, y 50), and (x 50, y 50) indicate regions of the target road protection object.

And when the early warning area in the template image is an area around the area containing the target road protection object, P1, P2, P3 and P4.

The image position of P1 is (x 20, y 20), (x 30, y 50), or (x 20, y 50);

the image position of P2 is (x 30, y 20), (x 30, y 10), (x 50, y 10), or (x 50, y 20);

the image position of P3 is (x 50, y 20), (x 60, y 50), or (x 50, y 50);

the image positions of P4 are (x 30, y 50), (x 30, y 60), (x 50, y 60), and (x 50, y 50).

In the case of the mode 1, at the same image position in the target image as the image position where the early warning region of the template image is located, a divided region is defined, that is:

a region composed of (x 20, y 20), (x 30, y 50) and (x 20, y 50) in the target image is a sub-region of an early warning region in the target image;

a region composed of (x 30, y 20), (x 30, y 10), (x 50, y 10) and (x 50, y 20) in the target image is a sub-region of an early warning region in the target image;

a region composed of (x 50, y 20), (x 60, y 50) and (x 50, y 50) in the target image is a sub-region of an early warning region in the target image;

the region composed of (x 30, y 50), (x 30, y 60), (x 50, y 60) and (x 50, y 50) in the target image is a sub-region of an early warning region in the target image.

In the case where the resolution of the template image and the at least one target image are the same, mode 2 is provided: and regarding each target image, taking the area outside the divided area at the image position of the target image, which is the same as the image position of the target road protection object without deviation in the template image, as an early warning area in the target image.

The area containing the target road protection in the template image is: (x 30, y 20), (x 50, y 20), (x 30, y 50), and (x 50, y 50).

As shown in fig. 10, the areas divided at the same image position as the image position where the target road guard is located in the template image without being shifted in the target image may be areas surrounded by (x 1, y 1), (x 1, ym), (xn, ym), and (xn, y 1), areas surrounded by (x 30, y 20), (x 50, y 20) and (x 50, y 50) except the areas surrounded by (x 30, y 20), (x 50, y 20), (x 30, y 50) and (x 50, y 50), and the remaining areas (i.e., diagonal portions) may be warning areas in the target image.

The resolution ratios of the template image and at least one target image are different, if the same coordinate system is established in the template image and the target image, the coordinate values of pixel points at the same position are different, and therefore, the early warning area of the target image cannot be directly found by using the same image position in the template image. Based on this, the embodiment of the present invention provides a mode 3, which is shown in fig. 11, and specifically:

s1100: for each target image, determining an area containing the target road protection object in the target image according to the resolution ratio of the template image and the target image and the area containing the target road protection object which is not subjected to deviation in the template image;

s1110: and determining the early warning area in the target image according to the proportion between the area containing the target road protection object which is not deviated and the early warning area in the template image and the area containing the target road protection object in the target image.

If the template image is 2 times the resolution of the target image, as shown in fig. 10 and 12, the area of the template image containing the target road guard that has not moved is:

(x 30, y 20), (x 50, y 20), (x 30, y 50), and (x 50, y 50);

then the area of the target image containing the target road protection material is 1/2 of the area of the template image containing the target road protection material which is not moved, namely:

(x 15, y 10), (x 25, y 10), (x 15, y 25), and (x 25, y 25).

The sub-regions in the early warning region of the template image are:

the image positions of P1 are (x 20, y 20), (x 30, y 50), (x 20, y 50), and the area of P1 is 10 × 30=300;

the image position of P2 is (x 30, y 20), (x 30, y 10), (x 50, y 20), and the area of P2 is 10 × 20=200;

the image position of P3 is (x 50, y 20), (x 60, y 50), (x 50, y 50), and the area of P3 is 10 × 30=300;

the image positions of P4 are (x 30, y 50), (x 30, y 60), (x 50, y 60), and (x 50, y 50), and the area of P4 is 10 × 20=200.

The area of the template image containing the target road protection object which does not move is as follows:

a region surrounded by (x 30, y 20), (x 50, y 20), (x 30, y 50), and (x 50, y 50) is 30 × 20=600;

the proportion of the area P1 to the area P3 to the area containing the target road protection object which does not move is 1/2;

the proportion of the area P2 to the area P4 to the area containing the target road protection object which does not move is 1/3;

the sub-regions of the early warning region shown in fig. 13 are Pa, pb, pc, and Pd.

The area of the target image containing the target road protection is as follows:

the areas of (x 15, y 10), (x 25, y 10), (x 15, y 25), and (x 25, y 25) are 150;

150, in which the areas of the regions Pa and Pc are 1/2, is 75;

150, which is 1/3 of the area of Pb and Pd, is 50;

then one side of the region according to Pa is 15 and the other side is 5, so that the four coordinate values of the region of Pa are:

(x10,y10)、(x15,y10)、(x15,y25)、(x10,y25)]；

then one side of the area according to Pb is 10 and the other side is 5, so that the four coordinate values of the area of Pb are:

(x15,y10)、(x15,y5)、(x25,y5)、(x25,y10)；

then one side length of the region according to Pc is 15 and the other side length is 5, so that the four coordinate values of the region of Pc are:

(x25,y10)、(x30,y10)、(x30,y25)、(x25,y25)；

then one side of the region according to Pd is 10 and the other side is 5, so that the four coordinate values of the region of Pd are:

(x15,y25)、(x25,y25)、(x25,y30)、(x15,y30)。

the guard rail has one color, such as red, white, gray and the like, and the protective barrel has two or three colors, such as red and white, or red, white, yellow and the like. For different road guards, an embodiment of the present invention provides a method for determining a target pixel point representing a target road guard in an early warning region of each target image according to an RGB value and an RGB threshold of the pixel point in the early warning region of each target image, including:

if the target road protection object contains one color, searching a pixel point of which the difference value between the RGB value and the RGB threshold value is within a first preset range from the early warning area of each target image, and taking the searched pixel point as a target pixel point; wherein the RGB threshold value is determined according to the RGB value corresponding to one color of the target road protection object.

Taking the example that the target road protection object contains one color, respectively setting the R channel range threshold as C according to the RGB three-channel range threshold of the pixel value _R C in T, R channel range threshold _R Is determined according to the R value of the RGB value corresponding to one color of the target road protection object, if the R value is white, if the R value is red, and the G channel range threshold value is C _G C in T, G channel range threshold _G Is determined according to the G value of the RGB value corresponding to one color of the target road protection object, if the target road protection object is white, the G value of the white is white, if the target road protection object is red, the G value of the red is red, and the threshold value of the B channel range is C _B C in T, B channel range threshold _B Is determined according to the B value of the RGB value corresponding to one color of the target road protection object, wherein if the B value is white, and if the B value is red, and T is an error value, as shown in the following table:

channel	R	G	B
				Threshold value	C _R ±T	C _G ±T	C _B ±T

And when the RGB value of each pixel point in the early warning area of each target image is compared with the RGB threshold value, if the difference value between the RGB value and the RGB threshold value is between-T and T, the pixel point is the target pixel point. Wherein the first preset range is-T to T. Specifically, the R value, the G value, and the B value of the RGB value of each pixel are compared with the corresponding R threshold, the corresponding G threshold, and the corresponding B threshold, and the difference values are determined to be between the error values in the table above, and if all are within the error range, it is determined that the pixel is the target pixel.

As shown in fig. 13, it is determined that the number of target pixel points in the early warning region of at least one target image satisfies the preset condition in the following manner:

s1300: determining the total number of target pixel points of the same image position of each target subregion in a plurality of target images; the target sub-region is a region at the same image position in an early warning region at the same image position in each target image;

s1310: determining the ratio of the total number corresponding to each target subregion to the total number of pixel points of each target subregion;

s1320: if the ratio of any one target subregion is not within the preset range corresponding to the target subregion, determining that the number of target pixel points in the early warning region of at least one target image meets a preset condition; the preset range corresponding to the target subregion is determined according to the ratio of the target pixel point of the subregion corresponding to the target subregion in the template image to the total number of the pixel points of the corresponding subregion in the template image.

Exemplary, snapping by electric police, bayonet video equipment [ T ] ₁ ,T ₂ ]Accumulating K pictures of the target road protection object in the time period range, and marking the pictures as F ₁ ,F ₂ ,...F _k . The F ₁ ,F ₂ ,...F _k Are all target images.

According to the content, K target images are divided into early warning areas, and F is used ₁ And starting processing, wherein the RGB values of the pixel points in the early warning area are expressed as follows:

as shown in fig. 9, the sub-regions of the early warning region are early warning region matrixes W ₁ ,W ₂ ,W ₃ ,W ₄ The structure is as follows:

wherein the element W _ij Representing the RGB values of the pixel points.

First, take W ₁ The (1, 1) th element W of ₁₁ RGB value W of _rgb And RGB three-channel range threshold C _rgb Making a comparison if | W _rgb -C _rgb If | < T, the element W is proved ₁₁ The RGB value of the corresponding pixel point is in a preset RGB three-channel range, namely the pixel point is a target pixel point, and therefore W is recorded ₁₁ Has a comparison result of H ₁₁ =1, otherwise H ₁₁ =0, continue comparison until (m, n) th element W _mn Record the comparison result H _mn . Get H ₁₁ ,H ₁₂ ,...H _mn Composition set H ₁ ＝(H ₁₁ ,H ₁₂ ,...H _mn ). Sequentially to the matrix W ₂ ,W ₃ ,W ₄ The respective elements are compared and finally the picture F is output ₁ Early warning area comparison result set H _f1 ＝(H ₁ ,H ₂ ,H ₃ ,H ₄ ) As shown in FIG. 14, H ₁ ＝(1001111001111)，H ₂ ＝(1001110101101)，H ₃ ＝(1001111011110)，H ₄ ＝(1011111001011)。

To F ₂ ～F _K After the above steps are completed, the comparison results of the four sub-regions shown in fig. 14 are obtained, that is, 1 represents a target pixel point, 0 represents that the target pixel point is not the target pixel point, and a picture F is output ₁ ,F ₂ ,...F _k Comparison result set L = (H) of early warning area pixel points _f1 ,H _f2 ,...H _fk )。

For F ₁ ～F _K In which the same sub-areas of the pre-warning area are combined, e.g. F ₁ In (d) a sub-region of Pa and F ₂ In (5), the Pa sub-regions are combined, and H is shown in FIG. 15 _f1 ,H _f2 Merging, taking H first _f1 ,H _f2 Respective subset H ₁ Is marked as a ₁₁ ,a ₁₂ ,....a _mn And b ₁₁ ,b ₁₂ ,....b _mn The same subscripts a and b are logically anded, and if a =1 and b =1, then a&b =1, otherwise a&b =0, i.e. when both pixels are target pixels, it is recorded as 1, otherwise it is recorded as 0 _f1 ,H _f2 After the combination, a (10011100) subset H is obtained ₁ Merging to generate a set V ₁ ＝(V ₁₁ ,V ₁₂ ,...V _mn ). Continue to adopt the same algorithm to F ₁ And F ₂ Middle H ₂ ,H ₃ ,H ₄ Merge, as shown in FIG. 16, and output H after completion _f1 ,H _f2 Combined result V of _x1 ＝(V ₁ ,V ₂ ,V ₃ ,V ₄ )。

Taking the result V of the previous step as shown in FIG. 17 _x1 And H _f3 Output result V is merged _x2 Then V _x2 Then with H _f4 Merging until completing with H _fk Output final result R = (R) after merging ₁ ,R ₂ ,R,R ₄ ) I.e. image F ₁ ～F _K Are merged to completion, where R ₁ ,R ₂ ,R ₃ ,R ₄ Respectively represent sub-regions W ₁ ,W ₂ ,W ₃ ,W ₄ And finally comparing the pixel points. R ₁ Is (11110110), R ₂ Is (00000011), R ₃ Is (01000101), R ₄ Is (00000011).

Subset R of the set R ₁ Statistical set R ₁ The percentage of the element with the median =1, that is, the percentage of the total number of target pixels to the total number of pixels in the sub-region, and if the percentage is the same as the percentage of the total number of pixels in the template imageAnd if the total number of the target pixel points of the corresponding subregion accounts for the percentage of the total number of the pixel points of the subregion, the content of the subregion of the target image is considered to be similar to the content of the corresponding subregion of the template image within a preset range. Specifically, since the image content of the corresponding sub-area in the template image is the content of the target road protector that is not shifted, the target image is similar to the template image, and it is indicated that the content of the sub-area of the target image is not shifted, at least in this sub-area. Wherein, the shape of the sub-region of the target image and the corresponding sub-region of the template image is as shown in fig. 9 and 12, if the sub-region of the target image is P1, then the corresponding sub-region of the template image is Pa.

For the contents of FIG. 17, for R1, G _％ 1+ 0)/8 + 100, G _％ =75 percentage of subareas corresponding to template image, e.g. P% compare if | G _％ -P _％ E is less than or equal to (E is a preset error value), and a comparison result S is recorded ₁ =0, otherwise S ₁ =1. Get set R one by one ₂ ,R ₃ ,R ₄ The percentage of the element with the statistical value =1 and the preset threshold value P _％ Row E comparison, final output set S = (S) ₁ ,S ₂ ,S ₂ ,S ₄ )。

If the elements of set S are summed, if S ₁ +S ₂ +S ₃ +S ₄ And if the position is more than 0, judging that the target road protection object has position deviation. That is, if any one of the sub-regions has a ratio calculated out of the preset range, it is considered that the deviation occurs.

Illustratively, when the target road protection object contains a plurality of colors, the process of determining the target pixel points of the target road protection object is as follows:

if the target road protector contains multiple colors, searching pixel points of which the difference value between the RGB value and any one RGB threshold value is within a second preset range from the early warning area of each target image, and taking the searched pixel points as target pixel points; wherein each RGB threshold value is determined according to the RGB value corresponding to each color of the target road protection object.

When the target road protection object comprises a plurality of colors, including red and white, then the threshold value of R, the threshold value of G, and the threshold value of B are determined by using the RGB values of red. And simultaneously, determining the threshold value of R, the threshold value of G and the threshold value of B by utilizing the white RGB values, wherein the 6 threshold values are the determined RGB threshold values when the RGB values corresponding to various colors in the target road protection object are obtained.

And calculating the difference value between the RGB value of each pixel point in the early warning area of each target image and the plurality of RGB threshold values, and if the difference value between any one of the RGB threshold values is within a second preset range, determining the pixel point as a target pixel point. Similarly, when comparing RGB thresholds, RGB is also compared separately.

Further, particularly when the target road protection object contains a plurality of colors, before notifying a road protection person that the position of the target road protection object is deviated, the method further includes:

determining the ratio between target pixel points corresponding to each RGB threshold in the early warning area of at least one target image, wherein the difference value between the ratio and the second preset ratio is not in a third preset range; and the second preset ratio is determined according to the ratio between target pixel points corresponding to each RGB threshold in the early warning area of the template image.

When the target road guard contains multiple colors, for example, the target road guard contains red and white, the pixel points in the early warning region of the target image are compared with the red RGB threshold to obtain target pixel points similar to the red, that is, target pixel points corresponding to the red, and then the pixel points in the early warning region of the target image are compared with the white RGB threshold to obtain target pixel points similar to the white, that is, target pixel points corresponding to the white. And then obtaining the ratio of the number of target pixel points corresponding to the red color and the number of target pixel points corresponding to the white color in the early warning area of at least one target image.

The method comprises the steps of comparing pixel points in an early warning area of a template image with red RGB threshold values to obtain target pixel points similar to red, namely target pixel points corresponding to red, then comparing the pixel points in the early warning area of the target image with white RGB threshold values to obtain target pixel points similar to white, namely target pixel points corresponding to white, and setting the ratio of the number of the target pixel points corresponding to red to the number of the target pixel points corresponding to white in the early warning area of the template image to be a second preset ratio.

And comparing the ratio of the number of target pixel points corresponding to the red color and the number of target pixel points corresponding to the white color in the early warning area of at least one target image with a second preset ratio to obtain whether the difference is within a third preset range.

If the target road protection object is not in the early warning area of the target image, the size of the area containing the target road protection object in the early warning area of the target image is different from the size of the area containing the target road protection object in the template image, and the target road protection object in the early warning area of the target image may be deviated due to the fact that the target road protection object in the template image is not deviated.

Meanwhile, the number of target pixel points in the early warning area of at least one target image is judged to meet the preset condition, and therefore whether the two points shift or not is comprehensively judged, and accuracy is improved.

As shown in fig. 18, the present invention also provides a movement detecting device for a road guard, comprising:

an identification module 1800, configured to respond to a monitoring start instruction, and determine an early warning region in at least one target image including a target road barrier according to an image position of the early warning region in a template image including the target road barrier that is not offset; the early warning area is the area around the area where the target road protection object is located in the target image;

a determining module 1810, configured to determine, according to an RGB value and an RGB threshold of a pixel in the early warning region of each target image, a target pixel representing a target road guard in the early warning region of each target image; wherein the RGB threshold value is determined from RGB values of the target road guard;

the first judging module 1820 is configured to determine that the position of the target road protector deviates if the number of target pixel points in the early warning region of at least one target image meets a preset condition; the preset condition is determined according to the number of target pixel points in an early warning area of a template image containing the target road protector which is not subjected to deviation;

a notification module 1830 configured to notify a road protection staff that the position of the target road protection object is shifted.

Optionally, the determining module 1810 is specifically configured to:

if the target road protection object contains multiple colors, searching a pixel point of which the difference value between the RGB value and any RGB threshold value is within a second preset range from the early warning area of each target image, and taking the searched pixel point as a target pixel point; wherein each RGB threshold is determined according to the corresponding RGB value of each color of the target road protection.

Optionally, as shown in fig. 19, the apparatus further includes:

the second judging module 1840 is configured to determine that a difference between a ratio between target pixel points corresponding to each RGB threshold in the early warning region of the at least one target image and a second preset ratio is not within a third preset range; and determining the third preset range according to the ratio of target pixel points corresponding to each RGB threshold in the early warning area of the template image.

Optionally, the first determining module 1820 is specifically configured to:

if the ratio of any one target subregion is not in the preset range corresponding to the target subregion, determining that the number of target pixel points in the early warning region of at least one target image meets a preset condition; the preset range corresponding to the target subregion is determined according to the ratio of the target pixel point of the subregion corresponding to the target subregion in the template image to the total number of the pixel points of the template image.

Optionally, the determining module 1810 is specifically configured to:

For each target image, taking an area outside the divided area at the image position, which is the same as the image position of the target road shield without deviation in the template image, in the target image as an early warning area in the target image; or alternatively

In addition, the method and apparatus for detecting movement of a road guard according to the embodiment of the present invention described with reference to fig. 1 to 19 may be implemented by an electronic device.

The electronic device includes: a processor and a communication unit;

Optionally, the processor is specifically configured to:

if the target road protection object contains one color, searching a pixel point of which the difference value between the RGB value and the RGB threshold value is within a first preset range from the early warning area of each target image, and taking the searched pixel point as a target pixel point; wherein the RGB threshold value is determined according to an RGB value corresponding to one color of the target road protection object;

Optionally, the processor is further configured to:

Optionally, the processor is specifically configured to:

if the ratio of any one target subregion is not within the preset range corresponding to the target subregion, determining that the number of target pixel points in the early warning region of at least one target image meets a preset condition; the preset range corresponding to the target sub-region is determined according to the ratio of the target pixel point of the sub-region corresponding to the target sub-region in the template image to the total number of the pixel points of the sub-region corresponding to the template image.

Optionally, the processor is specifically configured to:

Based on the above description, the electronic device structure of fig. 20 is exemplarily proposed.

Fig. 20 shows a schematic configuration diagram of the electronic apparatus.

The following specifically describes embodiments by taking an electronic device as an example. It should be understood that the electronic device shown in fig. 20 is only one example, and the electronic device may have more or fewer components than shown in fig. 20, may combine two or more components, or may have a different configuration of components. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

A block diagram of a hardware configuration of an electronic device according to an exemplary embodiment is exemplarily shown in fig. 20. As shown in fig. 20, the electronic apparatus includes: radio Frequency (RF) circuitry 2010, memory 2020, a display unit 2030, a Wireless Fidelity (Wi-Fi) module 2040, a processor 2050, a bluetooth module 2060, and a power supply 2070.

The RF circuit 2010 may be configured to receive and transmit signals during message transmission and reception or during a call, and may receive downlink data of a base station and then deliver the downlink data to the processor 2050 for processing; the uplink data may be transmitted to the base station. Typically, the RF circuitry includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like.

The memory 2020 may be used for storing software programs and data. The processor 2050 executes various functions of the electronic device and data processing by executing software programs or data stored in the memory 2020. The memory 2020 may include high-speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. The memory 2020 stores an operating system that enables the electronic device to operate. The memory 2020 may store an operating system and various applications and may store code that implements the methods described in embodiments of the present application.

The display unit 2030 may be used to receive input numeric or character information and generate signal input related to user settings and function control of the electronic device, and particularly, the display unit 2030 may include a touch screen 2031 provided on a front surface of the electronic device and collecting touch operations of a user thereon or nearby, such as clicking a button, dragging a scroll box, and the like.

The display unit 2030 may also be used for displaying a Graphical User Interface (GUI) for information input by or provided to the user and various menus of the electronic device. Specifically, the display unit 2030 can include a display screen 2032 provided on the front surface of the electronic apparatus. The display screen 2032 may be configured in the form of a liquid crystal display, a light emitting diode, or the like. Display unit 2030 may be used to display various graphical user interfaces described herein.

The touch screen 2031 may be covered on the display screen 2032, or the touch screen 2031 and the display screen 2032 may be integrated to implement the input and output functions of the electronic device, and after the integration, the touch screen may be referred to as a touch display screen for short. The display unit 2030 in the present application may display an application program and corresponding operation steps.

Wi-Fi belongs to a short-distance wireless transmission technology, and electronic equipment can help a user to receive and send emails, browse webpages, access streaming media and the like through a Wi-Fi module 2040, and provides wireless broadband internet access for the user.

The processor 2050, which is a control center of the electronic device, connects various parts of the entire terminal using various interfaces and lines, performs various functions of the electronic device and processes data by operating or executing software programs stored in the memory 2020 and calling data stored in the memory 2020. In some embodiments, processor 2050 may include one or more processing units; the processor 2050 may also integrate an application processor, which primarily handles operating systems, user interfaces, application programs, etc., and a baseband processor, which primarily handles wireless communications. It is to be appreciated that the baseband processor described above may not be integrated into the processor 2050. In the present application, the processor 2050 may run an operating system, an application program, a user interface display, and a touch response, as well as the processing method described in the embodiments of the present application. Further, the processor 2050 is coupled to a display unit 2030.

The bluetooth module 2060 is configured to perform information interaction with other bluetooth devices having a bluetooth module through a bluetooth protocol. For example, the electronic device may establish a bluetooth connection with a wearable electronic device (e.g., a smart watch) that is also equipped with a bluetooth module through the bluetooth module 2060, so as to perform data interaction.

The electronic device also includes a power supply 2070 (e.g., a battery) that powers the various components. The power supply may be logically coupled to the processor 2050 via a power management system to manage charging, discharging, power consumption, etc. via the power management system. The electronic device can also be provided with a power button for starting and closing the terminal, locking the screen and the like.

In addition, in combination with the electronic device in the above embodiments, an embodiment of the present invention may provide a storage medium, where instructions, when executed by a processor of the electronic device, enable the electronic device to perform the movement detection method of a road guard as described in any one of the above.

While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims

1. An electronic device, comprising: a processor and a communication unit;

2. The electronic device of claim 1, wherein the processor is specifically configured to:

if the target road protector contains multiple colors, searching pixel points of which the difference value between the RGB value and any one RGB threshold value is within a second preset range from the early warning area of each target image, and taking the searched pixel points as target pixel points; wherein each RGB threshold is determined according to an RGB value corresponding to each color of the target road guard.

3. The electronic device of claim 2, wherein the processor is further configured to:

4. The electronic device of claim 1, wherein the processor is specifically configured to:

5. The electronic device of any of claims 1-4, wherein the processor is specifically configured to:

For each target image, taking an area outside the divided area at the image position, which is the same as the image position of the target road shield without deviation in the template image, in the target image as an early warning area in the target image; or

6. A method for detecting movement of a road guard, comprising:

responding to a monitoring starting instruction, and determining an early warning area in at least one target image containing the target road protection object according to the image position of the early warning area in the template image containing the target road protection object which does not deviate; the early warning area is the area around the area where the target road protection object is located in the target image;

7. The method of claim 6, wherein determining the target pixel points representing the target road protection object in the early warning region of each target image according to the RGB values of the pixel points in the early warning region of each target image and the RGB threshold value comprises:

8. The method of claim 7, wherein prior to notifying a road guard of the offset in the position of the target road guard, the method further comprises:

9. The method according to claim 6, wherein the number of target pixel points in the early warning region of at least one target image is determined to meet the preset condition by:

determining the total number of target pixel points of the pixel points at the same image position of each target subregion in a plurality of target images; the target sub-region is a region at the same image position in an early warning region at the same image position in each target image;

10. The method according to any one of claims 6 to 9, wherein determining the early warning region in at least one target image containing the target road protection object according to the image position of the early warning region in the template image containing the target road protection object without deviation comprises: