CN112232111A

CN112232111A - Commercial concrete vehicle mixing drum steering monitoring method and system, electronic equipment and storage medium

Info

Publication number: CN112232111A
Application number: CN202010904583.6A
Authority: CN
Inventors: 陈一钦; 汪寒; 吕慧华
Original assignee: Hangzhou Hopechart Iot Technology Co ltd
Current assignee: Hangzhou Hopechart Iot Technology Co ltd
Priority date: 2020-09-01
Filing date: 2020-09-01
Publication date: 2021-01-15
Anticipated expiration: 2040-09-01
Also published as: CN112232111B

Abstract

The embodiment of the invention provides a method and a system for monitoring steering of a mixing drum of a commercial concrete truck, electronic equipment and a storage medium; the method comprises the following steps: acquiring an optical flow track of a characteristic point in a video image frame of a monitoring video of a commercial concrete vehicle mixing drum; determining steering indication information of the commercial concrete vehicle mixing drum at the current moment according to the information of the optical flow tracks of the characteristic points; the characteristic points are points which are easy to identify of the commercial concrete vehicle mixing drum. According to the method, the system, the electronic device and the storage medium for monitoring the steering of the commercial concrete vehicle mixing drum, the optical flow track is obtained from the video image frame of the monitoring video, and then the steering of the commercial concrete vehicle mixing drum is determined according to the flow direction of the optical flow track, so that the method and the system have the advantages of being more visual in detection result and better in visualization effect.

Description

Commercial concrete vehicle mixing drum steering monitoring method and system, electronic equipment and storage medium

Technical Field

The invention relates to the field of buildings, in particular to a method and a system for monitoring steering of a mixing drum of a commercial concrete truck, electronic equipment and a storage medium.

Background

With the rapid development of domestic construction and real estate industry for many years, cement concrete is constantly changing and developing as an important material of building structures. In consideration of various factors such as full utilization of construction land, improvement of labor conditions, reduction of environmental pollution and the like, 124 central cities in China have implemented regulations that prohibit concrete mixing on urban construction sites since 2004, and have begun to produce commercial concrete.

Commercial concrete transportation vehicles have come to be used to transport concrete to construction sites efficiently and safely. The commercial concrete transport vehicle is also called a mixer truck and a commercial concrete vehicle, is a special vehicle for transporting concrete in a long distance, and has double functions of transportation and mixing. Most of commercial concrete vehicles have a pear-shaped mixing drum which is obliquely arranged on a frame, the mixing drum rotates around the axis of the mixing drum, and materials enter and exit from a drum opening at the upper rear part. Two strip-shaped helical blades are symmetrically welded in the stirring cylinder from the cylinder opening to the cylinder bottom. When the mixing drum rotates, the two blades can do spiral motion, the concrete in the mixing drum can also move along the spiral direction of the spiral blades, and the concrete is continuously lifted and stirred in the process of turning. During feeding and transportation, the mixing drum rotates forwards, and the concrete moves inwards along the blades. When discharging, the concrete can be discharged along the blade only by reversing the mixing drum.

In the transportation of commercial concrete car and other working processes, in order to prevent concrete solidification or caking in the churn, keep the churn corotation all the time, otherwise the condition of stifle jar will appear, all can cause very big injury to jar body and car, and can hinder the construction progress, bring many unnecessary economic losses. In order to prevent the tank from being closed and prevent the conditions of material stealing and illegal discharging, the steering of the mixing drum of the commercial concrete truck needs to be monitored in real time.

In the prior art, a first magnetic pole and a second magnetic pole which are arranged on a tank body of a mixer truck are sensed through a magnetic sensor, so that the steering of a mixing drum of a commercial concrete truck is monitored. On the one hand, the mode is not direct enough, a worker can only know the steering of the mixing drum through a detection signal, the mixing drum cannot be directly observed, on the other hand, the magnetic sensor is also easily interfered by external factors, and the accuracy rate of a monitoring result is relatively low.

Disclosure of Invention

Aiming at the problems in the prior art, the embodiment of the invention provides a method and a system for monitoring steering of a mixing drum of a commercial concrete truck, electronic equipment and a storage medium.

The embodiment of the first aspect of the invention provides a steering monitoring method for a mixing drum of a commercial concrete truck, which comprises the following steps:

acquiring an optical flow track of a characteristic point in a video image frame of a monitoring video of a commercial concrete vehicle mixing drum;

determining steering indication information of the commercial concrete vehicle mixing drum at the current moment according to the information of the optical flow tracks of the characteristic points; wherein the content of the first and second substances,

the characteristic points are points which are easy to identify for the commercial concrete vehicle mixing drum.

In the above technical solution, the acquiring of the optical flow trajectory of the feature point in the video image frame of the monitoring video of the commercial concrete vehicle mixing drum includes:

determining the positions of the feature points in the video image frames of the monitoring video and storing position information;

and obtaining the optical flow track of the feature points according to the position information of the feature points.

In the above technical solution, the determining the position of the feature point in the video image frame of the monitoring video and storing the position information includes:

s1, selecting a video image frame from the monitoring video as a current video image frame, determining the position of the feature point in the current video image frame, storing position information, and initializing the count value of a counter;

s2, taking the next video image frame of the current video image frame in the monitoring video as the current video image frame, judging whether the counting value of the counter can be divided by a preset value K, if yes, executing the next step, otherwise, executing the step S4; wherein K is a positive integer;

s3, searching a new feature point in the current video image frame, and clearing the count value of the counter;

s4, judging whether the feature points exist in the current video image frame, if so, determining the positions of the feature points in the current video image frame, then executing the next step, if not, removing the nonexistent feature points, and then executing the next step;

s5, judging whether the position of the feature point in the current video image frame is abnormal or not, if not, saving the position information of the feature point in the current video image frame, simultaneously adding 1 to the count value of the counter, then executing the step S2 again, if so, clearing the feature point with the abnormality, adding 1 to the count value of the counter, and then executing the step S2 again.

In the above technical solution, the determining the steering indication information of the commercial concrete vehicle mixing drum at the current moment according to the information of the optical flow trajectory of the feature point includes:

determining the type of the optical flow locus of the feature point according to the information of the optical flow locus of the feature point; wherein the types of optical flow trajectories of the feature points include: a first type of long optical flow trajectory, a second type of long optical flow trajectory, and a short optical flow trajectory; the first type of long optical flow track is an optical flow track with the length larger than a first length threshold value, the displacement of the head and tail positions larger than a preset displacement threshold value, and the proportion of the head and tail positions pointing to the optical flow track to the adjacent position sequence of the displacement direction is larger than a preset first proportion; the second type of long optical flow track is an optical flow track with the length larger than a first length threshold value and the displacement of the head and tail positions smaller than or equal to a preset displacement threshold value; the short optical flow trajectory is an optical flow trajectory having a length less than or equal to a first length threshold and greater than a second length threshold;

and determining the steering indication information of the commercial concrete vehicle mixing drum at the current moment according to the type of the optical flow track of the characteristic point.

In the above technical solution, the determining the type of the optical flow trajectory of the feature point according to the information of the optical flow trajectory of the feature point includes:

step S11, counting the total number of optical flow tracks of the characteristic points, and recording the total number of the optical flow tracks as N; initializing the value of a parameter n to be 1, wherein the parameter n is used for representing the index of the processed optical flow trajectory;

step S12, judging whether N is larger than N, if so, ending the step of determining the type of the optical flow trajectory of the feature point, and if not, executing the next step;

step S13, determining whether the length of the nth optical flow trajectory is greater than a second length threshold S, if the length of the nth optical flow trajectory is less than or equal to the second length threshold S, recording an index n of the optical flow trajectory, and simultaneously making n equal to n +1, and then re-executing step S12; if the length of the nth optical flow track is larger than a second length threshold value s, executing the next step;

step S14, determining whether the length of the nth optical flow trajectory is greater than a first length threshold L, if the length of the nth optical flow trajectory is less than or equal to the first length threshold L, recording the nth optical flow trajectory as a short optical flow trajectory, making n equal to n +1, and then re-executing step S12; if the length of the nth optical flow track is greater than the first length threshold L, executing the next step;

step S15, determining whether the displacement of the nth optical flow trajectory in the vertical direction is greater than a displacement threshold W, if the displacement of the nth optical flow trajectory in the vertical direction is less than or equal to the displacement threshold W, recording the optical flow trajectory as a second type of long optical flow trajectory, making n equal to n +1, and then re-executing step S12; if the displacement of the nth optical flow track in the vertical direction is larger than the displacement threshold value W, executing the next step;

step S16, determining whether a ratio of an adjacent position sequence pointing to a relative displacement direction of head and tail positions of the optical flow trajectory in the nth optical flow trajectory in the entire optical flow trajectory is greater than a first ratio value B, if the ratio is less than or equal to the first ratio value B, making n equal to n +1, and then re-executing step S12; if the ratio is larger than the first ratio value B, the optical-flow trajectory is recorded as the first-type long optical-flow trajectory, and then step S12 is re-executed.

In the above technical solution, the determining the steering indication information of the mixing drum of the commercial concrete vehicle at the current moment according to the type of the optical flow trajectory of the feature point includes:

step S21, whether a first type of long optical flow track exists or not is searched in the optical flow tracks of the feature points, if yes, step S24 is executed, and if not, the next step is executed;

step S22, whether a second type of long optical flow track exists in the optical flow tracks of the characteristic points is searched, if yes, the steering indication information of the commercial concrete vehicle mixing drum at the current moment is set to be static, the step of determining the steering indication information of the commercial concrete vehicle mixing drum at the current moment is finished, and if not, the next step is executed;

step S23, counting the number of the light stream tracks flowing to the positive direction in the vertical direction and the number of the light stream tracks flowing to the negative direction in the vertical direction in all the short light stream tracks, and if the number of the light stream tracks flowing along the positive direction in the vertical direction is more and the difference between the number of the light stream tracks in the positive direction and the number of the light stream tracks in the negative direction is larger than a preset second threshold value, setting the steering indication information of the commercial concrete vehicle mixing drum at the current moment as the positive direction; if the number of the light stream tracks flowing in the reverse direction in the vertical direction is large, and the difference between the number of the light stream tracks in the positive direction and the number of the light stream tracks in the negative direction is larger than a preset second threshold value, setting the steering indication information of the commercial concrete vehicle mixing drum at the current moment to be reverse; if the quantity difference of the positive and negative opposite optical flow tracks is smaller than a second threshold value, setting the steering indication information of the commercial concrete vehicle mixing drum at the current moment as unknown; then finishing the step of determining the steering indication information of the commercial concrete vehicle mixing drum at the current moment;

step S24, counting the number of the optical flow tracks with the flow direction being the positive direction of the vertical direction and the number of the optical flow tracks with the flow direction being the negative direction of the vertical direction in all the first type of long optical flow tracks, and if the number of the optical flow tracks flowing along the positive direction of the vertical direction is more and the difference between the number of the optical flow tracks in the positive direction and the number of the optical flow tracks in the negative direction is larger than a preset third threshold value, setting the steering indication information of the mixing drum of the commercial concrete vehicle at the current moment to be in; if the number of the light stream tracks flowing in the reverse direction in the vertical direction is large, and the difference between the number of the light stream tracks in the positive direction and the number of the light stream tracks in the negative direction is larger than a preset third threshold value, setting the steering indication information of the commercial concrete vehicle mixing drum at the current moment to be reverse; and if the difference between the positive and negative optical flow track quantities is smaller than a third threshold value, setting the steering indication information of the commercial concrete vehicle mixing drum at the current moment as unknown.

In the above technical solution, the method further includes:

and checking the steering indication information of the commercial concrete vehicle mixing drum at the current moment according to the steering state of the commercial concrete vehicle mixing drum at the previous moment and the time constraint period to obtain the checked steering indication information.

In the above technical solution, the checking the steering indication information of the commercial concrete vehicle mixing drum at the current moment according to the steering state and the time constraint cycle of the commercial concrete vehicle mixing drum at the previous moment includes:

when the steering state of the commercial concrete vehicle mixing drum at the previous moment is consistent with the steering indication information of the commercial concrete vehicle mixing drum at the current moment, the checked steering indication information is the same as the steering indication information of the commercial concrete vehicle mixing drum at the current moment;

when the steering state of the commercial concrete vehicle mixing drum at the previous moment is inconsistent with the steering indication information of the commercial concrete vehicle mixing drum at the current moment, the checked steering indication information is the same as the steering state of the commercial concrete vehicle mixing drum at the previous moment in the time constraint period, and the checked steering indication information is the same as the steering indication information of the commercial concrete vehicle mixing drum at the current moment after the time constraint period.

An embodiment of a second aspect of the present invention provides a device for monitoring steering of a mixing drum of a commercial concrete truck, including:

the characteristic point light stream track acquisition module is used for acquiring a light stream track of a characteristic point in a video image frame of a monitoring video of the commercial concrete vehicle mixing drum;

the steering indication information determining module is used for determining steering indication information of the commercial concrete vehicle mixing drum at the current moment according to the information of the optical flow track of the characteristic points; wherein the content of the first and second substances,

In an embodiment of a third aspect of the present invention, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps of the method for monitoring steering of a mixing drum of a commercial concrete truck according to the embodiment of the first aspect of the present invention are implemented.

A fourth aspect of the present invention provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method for monitoring steering of a mixing drum of a commercial concrete vehicle as described in the first aspect of the present invention.

According to the method, the system, the electronic device and the storage medium for monitoring the steering of the commercial concrete vehicle mixing drum, the optical flow track is obtained from the video image frame of the monitoring video, and then the steering of the commercial concrete vehicle mixing drum is determined according to the flow direction of the optical flow track, so that the method and the system have the advantages of being more visual in detection result and better in visualization effect.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.

FIG. 1 is a flow chart of a method for monitoring steering of a mixing drum of a commercial concrete truck according to an embodiment of the invention;

FIG. 2 is a flow chart of determining the type of optical flow trajectory for feature points;

FIG. 3 is a flow chart of a process for determining a short optical flow trajectory;

FIG. 4 is a flow chart of a process for determining a first type of long optical flow trajectory;

FIG. 5 is a schematic view of a steering monitoring device for a mixing drum of a commercial concrete truck according to an embodiment of the invention;

fig. 6 is a schematic physical structure diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Fig. 1 is a flowchart of a method for monitoring steering of a mixing drum of a commercial concrete vehicle according to an embodiment of the present invention, and as shown in fig. 1, the method for monitoring steering of a mixing drum of a commercial concrete vehicle according to an embodiment of the present invention includes:

step 101, acquiring an optical flow track of a characteristic point from a video image frame of a monitoring video of a commercial concrete truck mixing drum.

In order to implement the method for monitoring the steering direction of the commercial concrete vehicle mixing drum provided by the embodiment of the invention, monitoring equipment for monitoring, such as a camera and equipment for storing and transmitting monitoring video data, needs to be installed near the commercial concrete vehicle mixing drum in advance. When the camera is installed, the camera is required to be installed at a position where the mixing drum can be shot, and the camera is required to be capable of shooting a sufficient characteristic surface. The specific manner of how the monitoring device is installed is well known to those skilled in the art and is therefore not repeated here.

And obtaining a monitoring video of the commercial concrete vehicle mixing drum from a pre-installed monitoring device, analyzing each video image frame in the monitoring video in the step, and obtaining an optical flow track of the characteristic point from the video image frame.

In the embodiment of the invention, the characteristic point refers to a point which is easy to identify on the mixing drum, such as a joint of two edges of the mixing drum and a certain convex point of the mixing drum. Those skilled in the art will readily appreciate that the feature points are points on the churn, and thus the position of the feature points in the video image frame will vary as the churn rotates. And obtaining the optical flow track corresponding to the characteristic point according to the position information of the characteristic point in different video image frames.

And 102, determining steering indication information of the commercial concrete vehicle mixing drum at the current moment according to the information of the optical flow tracks of the characteristic points.

In the previous step, the optical flow trajectories of the feature points are obtained. In the step, the steering indication information of the mixing drum of the commercial concrete vehicle at the current moment is determined according to the information of the optical flow track of the characteristic points.

In the embodiment of the invention, two standards for judging the steering of the mixing drum of the commercial concrete vehicle are provided, and the first judgment standard is as follows: the long light stream track flows to the first direction, the displacement of the head and tail positions of the long light stream track reaches a preset first threshold, and the proportion of adjacent position sequences pointing to the first direction in the long light stream track is larger than a preset first proportion value. And if the first judgment standard is met, the first direction is the steering direction of the mixing drum of the commercial concrete vehicle.

The second criterion is: although there is no long optical flow trajectory in the definite direction, the proportion of the short optical flow trajectories pointing in the first direction to all the short optical flow trajectories is higher than a preset second proportion value. And if the second judgment criterion is met, the first direction is the steering direction of the mixing drum of the commercial concrete vehicle.

Wherein the long optical flow trajectory is an optical flow trajectory having a length greater than a first length threshold; a short optical flow trace is an optical flow trace having a length less than or equal to a first length threshold and greater than a second length threshold. The second length threshold is less than the first length threshold.

The two criteria are not required to be satisfied simultaneously, as long as one of the criteria is satisfied.

Based on the above-described determination criterion, the type of the optical-flow trajectory of the feature point can be identified from the information of the optical-flow trajectory of the feature point, such as further dividing the long optical-flow trajectory into the first-type long optical-flow trajectory and the second-type long optical-flow trajectory. The first type of long optical flow track is an optical flow track with the length larger than a first length threshold value, the displacement of the head and tail positions larger than a preset displacement threshold value, and the proportion of the head and tail positions pointing to the optical flow track to the adjacent position sequence of the displacement direction is larger than a preset first proportion; the second type of long optical flow track is an optical flow track with the length larger than a first length threshold value and the displacement of the head and tail positions smaller than or equal to a preset displacement threshold value.

And determining the steering indication information of the commercial concrete vehicle mixing drum at the current moment as forward rotation, reverse rotation, static rotation and unknown according to whether the optical flow tracks of the characteristic points have the first-type long optical flow track, the second-type long optical flow track or the short optical flow track.

According to the method for monitoring the steering of the commercial concrete vehicle mixing drum, which is provided by the embodiment of the invention, the light stream track is obtained from the video image frame of the monitoring video, and then the steering of the commercial concrete vehicle mixing drum is determined according to the flow direction of the light stream track, so that the method has the advantages of more visual detection result and better visualization effect.

Based on any one of the above embodiments, in an embodiment of the present invention, the step 101 further includes:

Specifically, the step of determining the positions of the feature points in the video image frames of the monitoring video and storing the position information may further include:

step 1011, selecting one video image frame from the monitoring video of the commercial concrete vehicle mixing drum as a current video image frame, determining the position of the characteristic point in the current video image frame, storing the position information, and initializing the counting value of the counter.

Initially, one video image frame is selected from monitoring videos of a commercial concrete truck mixing drum to serve as a current video image frame. The selected video image frame may be a video image frame corresponding to the monitoring start time selected by the user.

After the current video image frame is determined, feature points are selected from the current video image frame. Due to the shooting position, the monitoring video can only reflect a part of images of the mixing drum of the commercial concrete vehicle, and therefore the selected feature points are visible in the current video image frame. In addition, the number of the selected feature points may be more than one, and may be specifically set according to the actual situation.

In the embodiment of the present invention, the position of the feature point in the current video image frame can be determined by an Opencv library function, and in other embodiments of the present invention, other technical means known to those skilled in the art can also be adopted.

The counter is used for counting the processed video image frames, and in the embodiment of the present invention, initializing the count value of the counter includes setting the count value to 0, and in other embodiments of the present invention, the count value may be set to other values.

And 1012, taking the next video image frame of the current video image frame in the monitoring video as the current video image frame, judging whether the counting value of the counter can be completely divided by a preset value K, if so, executing the next step, otherwise, executing 1014.

Due to the influence of external factors such as rotation of a mixing drum of the commercial concrete vehicle, light change and the like, the characteristic points do not exist in all video image frames of the monitoring video. For example, a characteristic point a on the side of the mixing drum of the commercial concrete vehicle is initially opposite to the camera of the monitoring device, and thus can be found in the video image frame of the monitoring video. After the commercial concrete vehicle mixing drum rotates for a certain time, the characteristic point rotates to the surface back to the camera, and the characteristic point cannot be found in a video image frame of a monitoring video.

For this situation, in order to avoid that the feature points cannot be found in the current video image frame and further the effective optical flow trajectory cannot be detected, in the embodiment of the present invention, the feature points need to be searched again at regular time intervals. In this step, the size of the time interval is set by a preset value K. Once the count value of the counter can be divided by the preset value K, the characteristic point needs to be searched again, and on the contrary, if the count value of the counter cannot be divided by the preset value K, the characteristic point does not need to be searched again.

K is a positive integer, and the specific value of K is preset according to the actual situation.

And 1013, searching a new characteristic point in the current video image frame, and clearing the count value of the counter.

In the step, corresponding to the situation that the count value of the counter can be divided by the preset value K, a new feature point needs to be searched in the current video image frame, then the count value of the counter is cleared, and after the count value can be divided by the preset value K, the operation of searching the new feature point is carried out again.

It should be noted that finding a new feature point does not mean that the previous feature point is discarded. Both the previous feature points and the new feature points will be processed as feature points in the following steps.

Step 1014, judging whether the characteristic points exist in the current video image frame, if so, determining the positions of the characteristic points in the current video image frame, then executing the next step, if not, removing the nonexistent characteristic points, and then executing the next step.

Due to the influence of external factors such as rotation of a mixing drum of the commercial concrete vehicle, light change and the like, characteristic points existing in a previous video image frame do not necessarily exist in a current video image frame. Therefore, in this step, it is necessary to determine whether or not the feature point exists in the current video image frame. If the feature point does not exist, the feature point disappears in the monitored video temporarily, and therefore the feature point is cleared. The characteristic point clearing means that the position of the characteristic point is not searched in the next video image frame, and the position information of the characteristic point stored before is cleared.

It should be noted that the number of the feature points that need to be determined whether there is in the current video image frame may be more than one, some feature points still exist in the current video image frame, and some feature points no longer exist in the current video image frame, and need to be processed respectively according to the specific situations of the feature points.

Step 1015, determining whether the position of the feature point in the current video image frame is abnormal, if no, saving the position information of the feature point in the current video image frame, meanwhile adding 1 to the count value of the counter, then executing step 1012 again, if so, removing the feature point with abnormality, adding 1 to the count value of the counter, then executing step 1012 again.

After the position of a certain characteristic point in the current video image frame is determined, the position information is compared with the position of the characteristic point in the previous video image frame of the current video image frame, and whether the position of the characteristic point in the current video image frame is abnormal or not is determined according to the comparison result. If the position deviation value is greater than or equal to a preset threshold value, determining that the position deviation value is abnormal, and removing the characteristic point; if the position deviation value is smaller than the preset threshold value, the abnormity is not existed, the position information of the characteristic point in the current video image frame is saved, and the count value of the counter is increased by 1.

It should be noted that, because the position between the monitoring device and the concrete vehicle is relatively fixed, different video image frames in the monitoring video can be regarded as being in the same coordinate system. The position deviation value of the characteristic point can be conveniently obtained under the same coordinate.

Through the above operation, the positions of the feature points in the plurality of consecutive video image frames can be determined, and the position sequence of the feature points, which is the optical flow trajectory corresponding to the feature points, can be obtained from the stored position information of the feature points. When each video image frame is processed, the optical flow track of the feature point corresponding to the current video image frame can be obtained by reading the position information of the feature point stored currently.

Based on any one of the above embodiments, in an embodiment of the present invention, the step 102 further includes:

step 1021, determining the type of the optical flow track of the feature point according to the information of the optical flow track of the feature point;

and 1022, determining steering indication information of the commercial concrete vehicle mixing drum at the current moment according to the type of the optical flow track of the characteristic point.

In the embodiment of the present invention, the types of the optical flow trajectories of the feature points are determined one by one.

Fig. 2 is a flowchart of determining the type of optical flow trajectory of the feature point, and as shown in fig. 2, step 1021 further includes:

step 1021-1, counting the total number of the optical flow tracks of the feature points, and recording the total number of the optical flow tracks as N; the value of the initialization parameter n is 1, and the parameter n is used to indicate the index of the processed optical flow trajectory.

And step 1021-2, judging whether N is larger than N, if so, ending the step 1021, and if not, executing the next step.

Step 1021-3, determining whether the length of the nth optical flow trajectory is greater than a second length threshold s, if the length of the nth optical flow trajectory is less than or equal to the second length threshold s, recording an index n of the optical flow trajectory, and simultaneously making n equal to n +1, and then re-executing step 1021-2; if the length of the nth optical flow trace is greater than the second length threshold s, the next step is performed.

Considering the complexity of the working scene of the commercial concrete vehicle, the light stream track with too short length is not suitable for being used as the basis for judging the steering of the mixing drum. Therefore, in this step, a second length threshold s is preset and used to screen whether the optical flow trajectory can be used as a statistical basis.

Step 1021-4, determining whether the length of the nth optical flow trajectory is greater than a first length threshold L, if the length of the nth optical flow trajectory is less than or equal to the first length threshold L, recording the optical flow trajectory as a short optical flow trajectory, making n equal to n +1, and then re-executing step 1021-2; if the length of the nth optical flow trace is greater than the first length threshold L, the next step is performed.

In this step, the threshold value L is used as a criterion for distinguishing whether the optical flow trajectory is a long optical flow trajectory or a short optical flow trajectory. If the length of the optical-flow trajectory is greater than the first length threshold L, the optical-flow trajectory is a long optical-flow trajectory, and if the length of the optical-flow trajectory is less than or equal to the first length threshold L, the optical-flow trajectory is a short optical-flow trajectory (in conjunction with the operations of steps 1021-3, the length of the short optical-flow trajectory should also be greater than the second length threshold s).

Step 1021-5, judging whether the displacement of the nth optical flow track in the vertical direction is larger than a displacement threshold value W, if the displacement of the nth optical flow track in the vertical direction is smaller than or equal to the displacement threshold value W, recording the optical flow track as a second type of long optical flow track, and executing step 1021-2 again after n is equal to n + 1; if the displacement of the nth optical flow trace in the vertical direction is greater than the displacement threshold W, the next step is performed.

In the embodiment of the present invention, the vertical direction is a vertical direction on the image coordinate system, not a vertical direction in a physical sense. When the camera is installed, the rotation direction of the mixing drum on the image coordinate system is ensured to correspond to the actual rotation direction of the mixing drum, the rotation direction of the mixing drum on the image coordinate system is parallel to one coordinate axis of the image coordinate system as much as possible, and the direction parallel to the coordinate axis is defined as the vertical direction on the image coordinate system.

Step 1021-6, judging whether the proportion of an adjacent position sequence pointing to the head and tail positions of the optical flow trajectory relative to the displacement direction in the nth optical flow trajectory in the whole optical flow trajectory is greater than a first proportion value B, if the proportion is less than or equal to the first proportion value B, making n equal to n +1, and then executing step 1021-2 again; if the ratio is larger than the first ratio value B, the optical-flow trajectory is recorded as a first-type long optical-flow trajectory, and then step 1021-2 is executed again.

The above is a description of a specific implementation procedure of step 1021. From the above description, it can be known that optical-flow trajectories can be classified into a plurality of categories, including: a first type of long optical flow trajectory, a second type of long optical flow trajectory, a short optical flow trajectory, an optical flow trajectory that is not statistical, and the like.

Step 1022 further includes:

step 1022-1, searching whether a first type long optical flow track exists in the optical flow tracks of the feature points, if so, executing step 1022-4, and if not, executing the next step;

step 1022-2, searching whether a second type of long optical flow track exists in the optical flow tracks of the characteristic points, if so, setting the steering indication information of the commercial concrete vehicle mixing drum at the current moment to be static, ending the step 1022, and if not, executing the next step;

when the mixing drum is in a static state, such as a vibration state that the commercial concrete vehicle and the mixing drum do not work or the commercial concrete vehicle starts but the mixing drum does not move, the continuous position sequence can be detected all the time according to the acquisition process of the light flow track of the characteristic points. Therefore, if the long optical flow track with the displacement not reaching the standard exists, the state of the commercial concrete vehicle mixing drum is static with a high probability, and therefore the commercial concrete vehicle mixing drum is judged to be in the static state currently.

And 1022-3, counting the number of the optical flow tracks flowing to the positive direction in the vertical direction and the number of the optical flow tracks flowing to the negative direction in the vertical direction in all the short optical flow tracks, and if the number of the optical flow tracks flowing along the positive direction in the vertical direction is large and the difference between the number of the optical flow tracks in the positive direction and the number of the optical flow tracks in the negative direction is larger than a preset second threshold value, setting the steering indication information of the commercial concrete vehicle mixing drum at the current moment as the forward direction. And if the number of the optical flow tracks flowing in the reverse direction in the vertical direction is large and the difference between the number of the optical flow tracks in the positive direction and the number of the optical flow tracks in the negative direction is larger than a preset second threshold value, setting the steering indication information of the commercial concrete vehicle mixing drum at the current moment to be reversed. And if the quantity difference of the positive and negative opposite optical flow tracks is smaller than a second threshold value, setting the steering indication information of the commercial concrete vehicle mixing drum at the current moment as unknown. Step 1022 is then ended.

In this step, the positive vertical direction is a direction in the image coordinate system corresponding to the actual agitating drum rotating forward, and the negative vertical direction is a direction in the image coordinate system corresponding to the actual agitating drum rotating backward. Fig. 3 is a flowchart of a determination process for a short optical flow trajectory, and the implementation of the step 1022-3 can refer to fig. 3.

And 1022-4, counting the number of the optical flow tracks with the positive direction of the vertical direction and the number of the optical flow tracks with the negative direction of the vertical direction in all the first type of long optical flow tracks, and if the number of the optical flow tracks flowing along the positive direction of the vertical direction is large and the difference between the number of the optical flow tracks in the positive direction and the number of the optical flow tracks in the negative direction is larger than a preset third threshold value, setting the steering indication information of the mixing drum of the commercial concrete vehicle at the current moment as the forward direction. And if the number of the optical flow tracks flowing in the reverse direction in the vertical direction is large and the difference between the number of the optical flow tracks in the positive direction and the number of the optical flow tracks in the negative direction is larger than a preset third threshold value, setting the steering indication information of the commercial concrete vehicle mixing drum at the current moment to be reversed. And if the difference between the positive and negative optical flow track quantities is smaller than a third threshold value, setting the steering indication information of the commercial concrete vehicle mixing drum at the current moment as unknown.

Fig. 4 is a flowchart of a determination process for the first-type long optical flow trajectory, and the implementation of the step 1022-4 can refer to fig. 4.

Based on any of the above embodiments, in an embodiment of the present invention, after step 102, the method further includes:

and checking the steering indication information of the commercial concrete vehicle mixing drum at the current moment according to the steering state of the commercial concrete vehicle mixing drum at the previous moment and the time constraint period.

In the previous step 102, the steering indication information of the commercial concrete vehicle mixing drum at the current moment is obtained. The complexity of the working condition is considered, and especially, the light change occurs under the conditions of commercial concrete vehicle movement and the like, and the detection fluctuation is caused by the unobvious surface characteristics when the mixing drum rotates. In the embodiment of the invention, the steering indication information of the mixing drum of the commercial concrete vehicle at the current moment needs to be further checked.

During checking, three types of information are involved. The first type is steering state information of the commercial concrete vehicle mixing drum at the previous moment obtained according to the previous video image frame, namely the commercial concrete vehicle mixing drum is in a forward rotation state, a reverse rotation state, a static state or an unknown state at the previous moment. The second type is a time constraint period, the time constraint period is related to the working logic of the commercial concrete vehicle mixing drum during working, if forward rotation is used for feeding and mixing, the process is usually continued for a long time, and mixing is required to be carried out constantly in the running process to prevent solidification; this process lasts for a relatively short time for unloading when reversing. Therefore, the time constraint period can be set according to the working logic of the mixing drum of the commercial concrete vehicle. The third type is the steering indication information of the commercial concrete vehicle mixing drum at the current moment. This information is the result of step 102.

According to the three types of information, a checking result of the steering indicating information can be obtained, and the checking result is higher in accuracy than the steering indicating information.

Specifically, the checking process includes:

For example, the checking result of the steering indication information of the mixing drum of the commercial concrete vehicle at the current moment is recorded as O_tRecording the steering indication information of the mixing drum of the commercial concrete vehicle at the current moment as C_tRecording the time constraint period as T and recording the steering state information of the mixing drum of the commercial concrete vehicle at the previous moment as O_t-1。

For convenience of description, forward rotation, reverse rotation, standstill, and unknown states are defined as 1, -1, 0, 2, respectively.

According to the steering state and the time constraint period of the commercial concrete vehicle mixing drum at the previous moment, the steering indication information of the commercial concrete vehicle mixing drum at the current moment is checked, and the method specifically comprises the following steps:

a. steering state information O of commercial concrete vehicle mixing drum at previous moment_t-1When the ratio is 1:

scene a1 steering indication information C of commercial concrete vehicle mixing drum at current moment_tChecking the result O if the result is 1_tIs 1. Namely: the steering indication information is not changed, and the time constraint period is not required to be considered.

Scene a2 steering indication information C of commercial concrete vehicle mixing drum at current moment_tIs-1, at this time, it is necessary to judge whether the time constraint period T is satisfied_{Switching between forward and reverse rotation}If yes, checking the result O_tIs-1, otherwise check the junctionFruit O_tIs 1.

Time constraint period T_{Switching between forward and reverse rotation}The time constraint period related to the action of switching the mixing drum of the commercial concrete vehicle from the forward direction to the reverse direction or from the reverse direction to the forward direction is described. In scenario a2, the time constrains the period T_{Switching between forward and reverse rotation}Is referred to as from C_tIs-1, O_t-1Starting at the first time of 1, T is performed_{Switching between forward and reverse rotation}Counting down the time length, checking the result O before the counting down is not finished_tStill 1, check result O after countdown is finished_tIs-1, i.e. has a time length of T_{Switching between forward and reverse rotation}In the time period of (2), steering indication information C_tTo always be-1, check the result O_tIt will transition from 1 to-1 at the previous time.

Scene a3 steering indication information C of commercial concrete vehicle mixing drum at current moment_t0, at this time, it is necessary to determine whether the time constraint period T is satisfied_{Motion → rest}If yes, checking the result O_tIs 0, otherwise the check result O_tIs 1.

Time constraint period T_{Motion → rest}Is referred to as from C_tIs 0, O_t-1T is started at the first time point of-1 or 1_{Motion → rest}Counting down the length of time, i.e. for a time length of T_{Motion → rest}In the time period of (2), steering indication information C_tAlways 0, check result O_tWill transition from a1 or-1 to a 0 at the previous time.

Scene a4 steering indication information C of commercial concrete vehicle mixing drum at current moment_tAt this time, it is determined whether the time constraint period T is satisfied_{Motion → unknown}If yes, checking the result O_tIs 2, otherwise the result O is checked_tIs 1.

Time constraint period T_{Motion → unknown}Is referred to as from C_tIs 2, O_t-1T is started at the first time point of-1 or 1_{Motion → unknown}Countdown of the length of time. I.e. for a time length of T_{Motion → unknown}In the time period of (2), steering indication information C_tAlways 2, check result O_tWill be got by1 or-1 at the previous time is changed to 2. In scenario a4, at a time length of T_{Motion → unknown}In the time period of (2), steering indication information C_tAlways 2, check result O_tWill change from 1 to 2 at the previous moment

b. Steering state information O of commercial concrete vehicle mixing drum at previous moment_t-1When the ratio is-1:

scene b1 steering indication information C of commercial concrete vehicle mixing drum at current moment_tIf it is-1, check the result O_tIs-1. Namely: the steering indication information is not changed, and the time constraint period is not required to be considered.

Scene b2 steering indication information C of commercial concrete vehicle mixing drum at current moment_tIs 1, at this time, it is necessary to determine whether the time constraint period T is satisfied_{Switching between forward and reverse rotation}If yes, checking the result O_tIs 1, otherwise the result O is checked_tIs-1.

Scene b3 steering indication information C of commercial concrete vehicle mixing drum at current moment_t0, at this time, it is necessary to determine whether the time constraint period T is satisfied_{Motion → rest}If yes, checking the result O_tIs 0, otherwise the check result O_tIs-1.

Scene b4 steering indication information C of commercial concrete vehicle mixing drum at current moment_tAt this time, it is determined whether the time constraint period T is satisfied_{Motion → unknown}If yes, checking the result O_tIs 2, otherwise the result O is checked_tIs-1.

c. Steering state information O of commercial concrete vehicle mixing drum at previous moment_t-1When the ratio is 0:

scene C1 steering indication information C of commercial concrete vehicle mixing drum at current moment_tIf 0, check the result O_tIs 0. Namely: the steering indication information is not changed, and the time constraint period is not required to be considered.

Scene C2 steering indication information C of commercial concrete vehicle mixing drum at current moment_tIs 1, at this time, it is necessary to determine whether the time constraint period T is satisfied_{Static → motion}If yes, checking the result O_tIs 1, otherwiseNuclear result O_tIs 0.

Time constraint period T_{Static → motion}Is referred to as from C_tIs 1 or-1, O_t-1T is started at the first time point of 0_{Static → motion}Countdown of the length of time. I.e. for a time length of T_{Static → motion}In the time period of (2), steering indication information C_tAlways 1 or-1, check result O_tWill transition from 0 to 1 or-1 at the previous time. In scenario c2, at a time length of T_{Static → motion}In the time period of (2), steering indication information C_tAlways 1, check result O_tWill transition from 0 to 1 at the previous time.

Scene C3 steering indication information C of commercial concrete vehicle mixing drum at current moment_tIs-1, at this time, it is necessary to judge whether the time constraint period T is satisfied_{Static → motion}If yes, checking the result O_tIs-1, otherwise the result O is checked_tIs 0.

Scene C4 steering indication information C of commercial concrete vehicle mixing drum at current moment_tAt this time, it is determined whether the time constraint period T is satisfied_{At rest → unknown}If yes, checking the result O_tIs 2, otherwise the result O is checked_tIs 0.

Time constraint period T_{At rest → unknown}Is referred to as from C_tIs 2, O_t-1T is started at the first time point of 0_{At rest → unknown}Countdown of the length of time. I.e. for a time length of T_{At rest → unknown}In the time period of (2), steering indication information C_tAlways 2, check result O_tIt will transition from 0 to 2 at the previous time.

d. Steering state information O of commercial concrete vehicle mixing drum at previous moment_t-1When the ratio is 2:

steering indication information C of scene d1 if commercial concrete vehicle mixing drum is at current moment_tIf it is 2, check the result O_tIs 2. Namely: the steering indication information is not changed, and the time constraint period is not required to be considered.

Steering indication information C of scene d2 if commercial concrete vehicle mixing drum is at current moment_tIs 1, thisWhether the time constraint period T is met or not is judged_{Unknown → motion}If yes, checking the result O_tIs 1, otherwise the result O is checked_tIs 2.

Time constraint period T_{Unknown → motion}Is referred to as from C_tIs 1 or-1, O_t-1T is started at the first time of 2_{Unknown → motion}Countdown of the length of time. I.e. for a time length of T_{Unknown → motion}In the time period of (2), steering indication information C_tAlways 1 or always-1, checking the result O_tWill transition from 0 to 1 or-1 at the previous time. In scenario d2, at a time length of T_{Unknown → motion}In the time period of (2), steering indication information C_tAlways 1, check result O_tWill transition from 0 to 1 at the previous time.

Steering indication information C of scene d3 if commercial concrete vehicle mixing drum is at current moment_tIs-1, at this time, it is necessary to judge whether the time constraint period T is satisfied_{Unknown → motion}If yes, checking the result O_tIs-1, otherwise the result O is checked_tIs 2.

Steering indication information C of scene d4 if commercial concrete vehicle mixing drum is at current moment_t0, at this time, it is necessary to determine whether the time constraint period T is satisfied_{Unknown → stationary}If yes, checking the result O_tIs 0, otherwise the check result O_tIs 2.

Time constraint period T_{Unknown → stationary}Is referred to as from C_tIs 0, O_t-1T is started at the first time of 2_{Unknown → stationary}Countdown of the length of time. I.e. for a time length of T_{Unknown → stationary}In the time period of (2), steering indication information C_tAlways 0, check result O_tIt will transition from 2 to 0 at the previous time.

According to the steering monitoring method for the commercial concrete vehicle mixing drum provided by the embodiment of the invention, the steering indication information of the commercial concrete vehicle mixing drum at the current moment is checked according to the steering state and the time constraint period of the commercial concrete vehicle mixing drum at the previous moment, so that the illumination change and low illumination condition can be effectively overcome, and the final result is stable and reliable.

Based on any one of the above embodiments, fig. 5 is a schematic view of a steering monitoring device for a commercial concrete vehicle mixing drum provided in an embodiment of the present invention, and as shown in fig. 5, the steering monitoring device for a commercial concrete vehicle mixing drum provided in an embodiment of the present invention includes:

the characteristic point light stream trajectory acquisition module 501 is used for acquiring a light stream trajectory of a characteristic point in a video image frame of a monitoring video of a commercial concrete vehicle mixing drum;

the steering indication information determining module 502 is configured to determine steering indication information of the commercial concrete vehicle mixing drum at the current moment according to the information of the optical flow trajectory of the feature point; wherein the content of the first and second substances,

The device for monitoring the steering of the commercial concrete vehicle mixing drum, provided by the embodiment of the invention, acquires the light stream track from the video image frame of the monitoring video, and then determines the steering of the commercial concrete vehicle mixing drum according to the flow direction of the light stream track, so that the device has the advantages of more visual detection result and better visualization effect.

Fig. 6 is a schematic physical structure diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 6, the electronic device may include: a processor (processor)610, a communication Interface (Communications Interface)620, a memory (memory)630 and a communication bus 640, wherein the processor 610, the communication Interface 620 and the memory 630 communicate with each other via the communication bus 640. The processor 610 may call logic instructions in the memory 630 to perform the following method: acquiring an optical flow track of a characteristic point in a video image frame of a monitoring video of a commercial concrete vehicle mixing drum; determining steering indication information of the commercial concrete vehicle mixing drum at the current moment according to the information of the optical flow tracks of the characteristic points; the characteristic points are points which are easy to identify of the commercial concrete vehicle mixing drum.

In addition, the logic instructions in the memory 630 may be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

In another aspect, an embodiment of the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored, where the computer program is implemented by a processor to perform the method provided by the foregoing embodiments, for example, including: acquiring an optical flow track of a characteristic point in a video image frame of a monitoring video of a commercial concrete vehicle mixing drum; determining steering indication information of the commercial concrete vehicle mixing drum at the current moment according to the information of the optical flow tracks of the characteristic points; the characteristic points are points which are easy to identify of the commercial concrete vehicle mixing drum.

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A steering monitoring method for a mixing drum of a commercial concrete truck is characterized by comprising the following steps:

2. The steering monitoring method for the commercial concrete vehicle mixing drum as claimed in claim 1, wherein the step of acquiring the optical flow track of the feature points in the video image frame of the monitoring video of the commercial concrete vehicle mixing drum comprises the following steps:

3. The steering monitoring method for the mixing drum of the commercial concrete vehicle as claimed in claim 2, wherein the determining the positions of the characteristic points in the video image frames of the monitoring video and storing the position information comprises:

4. The steering monitoring method for the commercial concrete vehicle mixing drum according to claim 1, wherein the step of determining the steering indication information of the commercial concrete vehicle mixing drum at the current moment according to the information of the optical flow tracks of the characteristic points comprises the following steps:

5. The steering monitoring method for the mixing drum of the commercial concrete vehicle as claimed in claim 4, wherein the determining the type of the optical flow tracks of the characteristic points according to the information of the optical flow tracks of the characteristic points comprises:

6. The steering monitoring method for the commercial concrete vehicle mixing drum as claimed in claim 5, wherein the step of determining the steering indication information of the commercial concrete vehicle mixing drum at the current moment according to the type of the optical flow track of the characteristic points comprises the following steps:

7. The steering monitoring method for the mixing drum of the commercial concrete vehicle as claimed in any one of claims 1 to 6, wherein the method further comprises the following steps:

8. The steering monitoring method for the mixing drum of the commercial concrete vehicle as claimed in claim 7, wherein the checking of the steering indication information of the mixing drum of the commercial concrete vehicle at the current moment according to the steering state of the mixing drum of the commercial concrete vehicle at the previous moment and the time constraint cycle comprises:

9. The utility model provides a merchant's concrete car churn turns to monitoring devices which characterized in that includes:

10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the steps of the method for monitoring steering of a mixing drum of a commercial concrete vehicle as recited in any one of claims 1 to 8.

11. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the steps of the method for monitoring steering of a mixing drum of a commercial concrete vehicle as recited in any one of claims 1 to 8.