WO2022193848A1 - 搅拌站智能卸料监控方法及系统 - Google Patents

搅拌站智能卸料监控方法及系统 Download PDF

Info

Publication number
WO2022193848A1
WO2022193848A1 PCT/CN2022/074291 CN2022074291W WO2022193848A1 WO 2022193848 A1 WO2022193848 A1 WO 2022193848A1 CN 2022074291 W CN2022074291 W CN 2022074291W WO 2022193848 A1 WO2022193848 A1 WO 2022193848A1
Authority
WO
WIPO (PCT)
Prior art keywords
hopper
mixer truck
mixing station
video frame
monitoring
Prior art date
Application number
PCT/CN2022/074291
Other languages
English (en)
French (fr)
Inventor
章博
吴俊�
Original Assignee
三一汽车制造有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 三一汽车制造有限公司 filed Critical 三一汽车制造有限公司
Publication of WO2022193848A1 publication Critical patent/WO2022193848A1/zh

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B28—WORKING CEMENT, CLAY, OR STONE
    • B28C—PREPARING CLAY; PRODUCING MIXTURES CONTAINING CLAY OR CEMENTITIOUS MATERIAL, e.g. PLASTER
    • B28C7/00—Controlling the operation of apparatus for producing mixtures of clay or cement with other substances; Supplying or proportioning the ingredients for mixing clay or cement with other substances; Discharging the mixture
    • B28C7/02—Controlling the operation of the mixing
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B28—WORKING CEMENT, CLAY, OR STONE
    • B28C—PREPARING CLAY; PRODUCING MIXTURES CONTAINING CLAY OR CEMENTITIOUS MATERIAL, e.g. PLASTER
    • B28C7/00—Controlling the operation of apparatus for producing mixtures of clay or cement with other substances; Supplying or proportioning the ingredients for mixing clay or cement with other substances; Discharging the mixture
    • B28C7/16—Discharge means, e.g. with intermediate storage of fresh concrete
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources

Definitions

  • the present application relates to the technical field of operation machinery, and in particular, to a method and system for monitoring intelligent discharge of a mixing station.
  • the unloading process of the mixing station basically relies on manual monitoring, such as manual monitoring to judge whether the hopper of the mixer truck is aligned with the discharge port of the mixing station, and whether to start the discharge, etc.
  • manual monitoring such as manual monitoring to judge whether the hopper of the mixer truck is aligned with the discharge port of the mixing station, and whether to start the discharge, etc.
  • the degree of automation of the discharge monitoring of the mixing station is relatively low. .
  • the present application provides an intelligent unloading monitoring method and system for a mixing plant to solve the technical problem that the unloading process of the mixing plant in the prior art basically relies on manual monitoring, and the automation degree of the unloading monitoring of the mixing plant is low.
  • the application provides an intelligent discharge monitoring method for a mixing station, including:
  • the first video frame is detected and identified, and based on the identification result, the discharge of the concrete mixing plant is monitored.
  • the monitoring of the unloading of the concrete mixing station based on the identification result specifically includes:
  • the recognition result is that the first video frame includes a mixer truck hopper, then based on the first video frame, determine the position information of the key points of the mixer truck hopper;
  • a first control signal is sent to the control system of the mixing station corresponding to the discharge port of the mixing station, and the first control signal is used to instruct the start of unloading. material.
  • a control signal specifically including:
  • the sending of the first control signal to the control system of the mixing station corresponding to the discharge port of the mixing station further includes:
  • a second control signal is generated, and the second control signal is sent to the mixing station control system, where the second control signal is used to instruct to adjust the discharge speed or stop the discharge.
  • the determination of the material level status of the hopper of the mixer truck based on the second video frame specifically includes:
  • the material level state of the hopper of the mixer truck is determined.
  • the determination of the material level status of the hopper of the mixer truck based on the second video frame specifically includes:
  • the material level state of the hopper of the mixer truck is determined.
  • the sending of the first control signal to the control system of the mixing station corresponding to the discharge port of the mixing station further includes:
  • a third control signal is generated, and the third control signal is used to instruct to adjust the discharge speed.
  • the determining of the position information of the key points of the hopper of the mixer truck based on the first video frame specifically includes:
  • the weight parameters in the identification algorithm model of the mixer truck receiving hopper are obtained by training based on the image samples carrying the label of the mixer truck receiving hopper.
  • the determination of the connection of the mixer truck is based on the position information of the key points of the hopper of the mixer truck and the preset key position information of the discharge port of the mixing station. Whether the hopper is aligned with the discharge port of the mixing station, specifically including:
  • the first video frame is obtained by shooting with a monitoring camera
  • determining whether the hopper of the mixer truck is aligned with the discharge port of the mixing station based on the position information of the key points of the hopper of the mixer truck and the preset key position information of the discharge port of the mixing station, Also included before:
  • distortion correction is performed on the position information of the key points of the mixer truck hopper, and the position information of the key points of the mixer truck hopper after the distortion correction is obtained. ;
  • the determining whether the hopper of the mixer truck is aligned with the discharge port of the mixing station based on the position information of the key points of the hopper of the mixer truck and the preset key position information of the discharge port of the mixing station includes:
  • the first video frame is obtained by shooting with a monitoring camera, and the first video frame is a video frame obtained by shooting the monitoring camera after the shooting angle is changed;
  • determining the location information of the key points of the mixer truck hopper based on the first video frame and then further including:
  • the reference video frame is a video frame obtained by shooting before the shooting angle of the surveillance camera changes;
  • mapping and correcting the position information of the key points of the hopper of the mixer truck to obtain the position information of the key points of the hopper of the mixer truck after the mapping correction;
  • the determining whether the hopper of the mixer truck is aligned with the discharge port of the mixing station based on the position information of the key points of the hopper of the mixer truck and the preset key position information of the discharge port of the mixing station includes:
  • the monitoring camera is specifically an RGB camera or a TOF deep-sensing camera.
  • the application also provides an intelligent discharge monitoring system for a mixing station, including:
  • the monitoring video acquisition module is used to acquire the monitoring video frame of the unloading area of the mixing station, and obtain the first video frame;
  • the unloading monitoring module is used to detect and identify the first video frame based on a preset algorithm model, and monitor the unloading of the concrete mixing station based on the identification result.
  • the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, the processor implementing the program to realize the intelligent mixing station as described in any of the above Steps of the discharge monitoring method.
  • the present application also provides a non-transitory computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the steps of any of the above-mentioned methods for monitoring intelligent unloading of a mixing station.
  • the intelligent unloading monitoring of the mixing station provided by this application, by acquiring the monitoring video frames of the unloading area of the mixing station, and based on the preset algorithm model, the monitoring video is identified, and based on the identification result, the mixing station is monitored for the discharge process. .
  • the method does not require manual participation in the discharge monitoring process, can realize the automation and intelligence of the discharge monitoring, improves the automation degree of the discharge monitoring of the mixing station, ensures the effect of the discharge monitoring, and also improves the discharge monitoring. efficiency.
  • Fig. 1 is the schematic flow chart of the intelligent discharge monitoring method of the mixing station provided by the embodiment of the present application;
  • FIG. 2 is a schematic diagram of an image of the mixer truck hopper area in the embodiment of the present application.
  • FIG. 3 is a schematic diagram of the installation positions of two surveillance cameras in the embodiment of the present application.
  • Fig. 4 is the concrete schematic flow chart of the intelligent discharge monitoring method of the mixing station provided by the embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of an intelligent discharge monitoring system for a mixing station provided by an embodiment of the present application
  • Fig. 6 is the concrete structural schematic diagram of the intelligent discharge monitoring system of the mixing station provided by the embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of an electronic device provided by the present application.
  • a mixing plant unloading anti-overflow device is usually fixedly installed at the discharge port of the mixing plant.
  • the device monitors the current material level height through the distance measuring sensor fixed on the outside of the hopper, transmits the current material level height to the mixer truck receiving detection device, and controls the discharge door through the flow adjustment device.
  • the ranging sensor cannot be adapted to various types of mixer trucks. And because the mixer driver can only align within the specified range when reversing, and the size of the mixer hopper is also different, this will make the mixer of the same height identify the height of the material level through the ranging sensor.
  • the distance measuring sensor measures the point-to-point distance. The distribution of the material level in the hopper of the mixer truck is uneven, and the measuring effect of the distance measuring sensor is not good.
  • the distance measuring sensor is installed on the outside of the hopper, which is easily covered by concrete and powdered cement, which affects the measurement effect.
  • this solution cannot judge whether the hopper of the mixer truck is aligned with the discharge port of the mixing station, which still requires manual judgment, and cannot measure the height of the material level or directly monitor the discharge.
  • the embodiments of the present application provide an intelligent discharge monitoring method for a mixing station.
  • Figure 1 is a schematic flow chart of the method. As shown in Figure 1, the method includes:
  • the execution body may be an embedded edge computing power module, and the module may be configured in the intelligent discharge monitoring system of the mixing station.
  • the intelligent discharge monitoring system of the mixing station can be set in the central control room of the mixing station.
  • the mixing station sends the mixed concrete into the mixer through the discharge port of the mixing station and the hopper of the mixer. This process is the unloading process.
  • the intelligent unloading monitoring method of the mixing station provided in the embodiment of the present application is that It is used to monitor the unloading process, such as monitoring whether the mixer hopper is aligned with the mixing station discharge port, monitoring the discharge information of the mixing station discharge port, and identifying the concrete shape in the mixer hopper area. Wait.
  • Step S1 is performed first.
  • the first video frame can be captured by a surveillance camera and uploaded to the embedded edge computing power module.
  • the monitoring video frame of the discharge area of the mixing station obtained by the embedded edge computing module includes the discharge port of the mixing station, and the acquisition of the monitoring video frame can be carried out in real time to display the real-time picture of the discharge port of the concrete mixing station.
  • the preset algorithm model may be a model constructed based on an artificial intelligence vision algorithm or an algorithm related thereto.
  • a model constructed based on a deep learning algorithm may be used to perform feature extraction on the first video frame.
  • the first video frame is detected and identified with the obtained features, and the identified result can be the material level information of the current concrete in the mixer hopper. According to the material level information, it can be judged whether the current concrete has overflowed and the discharge flow of the concrete. , concrete level change information and current concrete form information.
  • the intelligent unloading monitoring of the mixing station acquires the first video frame of the unloading area of the mixing station, identifies the first video frame based on a preset algorithm model, and conducts the monitoring of the mixing station based on the identification result. Unloading process monitoring.
  • the method does not require manual participation in the discharge monitoring process, can realize the automation of the discharge monitoring, ensures the effect of the discharge monitoring, and also improves the efficiency of the discharge monitoring.
  • the method for monitoring the intelligent discharge of the mixing station provided by the embodiment of the present application, the monitoring of the discharge of the concrete mixing station based on the identification result, specifically includes:
  • the recognition result is that the first video frame includes a mixer truck hopper, then based on the first video frame, determine the position information of the key points of the mixer truck hopper;
  • a first control signal is sent to the control system of the mixing station corresponding to the discharge port of the mixing station, and the first control signal is used to instruct the start of unloading. material.
  • the mixer truck since the mixer truck gradually enters the monitoring range, there will be a situation in which the mixer truck does not have a hopper in the monitoring video frame. Therefore, it is necessary to globally identify the first video frame to determine whether the first video frame appears in the first video frame.
  • the mixer truck is connected to the hopper.
  • the first video frame is processed to obtain position information of key points of the mixer truck hopper.
  • the key point is the point with certain characteristics on the outer boundary of the mixer hopper. That is to say, the obtained key points are located on the cross section of the mixer hopper, and the distance from the center of the mixer hopper is equal.
  • the location information here may be coordinates.
  • the position information of the key points of the mixer truck's hopper can be represented by coordinates, that is, the key point coordinates of the mixer truck's hopper.
  • the preset key position information of the discharge port of the mixing station may be the position information of the central axis of the discharge port of the mixing station determined in advance, and the position information of the central axis may be set according to the actual situation, which is not specified in the embodiment of the present application. limited.
  • the mixer's hopper and the mixing station's discharge port are aligned. If the vertical shortest distance between the position information of the center of the key point of the hopper of the mixer and the preset key position information of the discharge port of the mixing station is within the preset range, it means that the hopper of the mixer and the discharge port of the mixing station are within the preset range. already aligned.
  • the preset range may be set according to actual needs, which is not specifically limited in this embodiment of the present application.
  • the first control signal contains an instruction to start unloading, that is, when the mixing station control system receives the first control signal, it can start unloading.
  • the position information of the key points of the hopper of the mixer truck and the preset key position information of the discharge port of the mixing station may be the coordinates in the world coordinate system or the coordinates in any coordinate system. It is sufficient to ensure that the above two are compared under the same coordinate system, and the present application does not specifically limit the coordinate system.
  • the world coordinate system refers to the coordinate system of the three-dimensional world defined by the user, which is introduced to describe the position of the target object in the real world.
  • the method for monitoring the intelligent unloading of the mixing station identifies the first video frame in which the mixing hopper of the mixer truck appears in the monitoring video frame through the monitoring video frame of the unloading area of the mixing station, and obtains the key of the hopper of the mixer truck. Then compare the position information of the key points of the mixer truck's hopper with the key position information of the mixing station's discharge port, so as to judge whether the mixer truck's receiving hopper and the mixing station's discharge port are aligned, and determine when After the two are aligned, the first control signal is sent to the control system of the mixing station corresponding to the discharge port of the mixing station, indicating that the material unloading can be started at this time.
  • the method monitors the unloading process by monitoring video frames, and can automatically detect whether the mixing hopper and the unloading port of the mixing station are aligned during unloading.
  • the misalignment of the hopper and the discharge opening of the batching plant causes waste of concrete during the discharge process.
  • the intelligent discharge monitoring method of the mixing station provided by the embodiment of the present application, if it is determined that the hopper of the mixer truck is aligned with the discharge port of the mixing station, the material is discharged to the mixing station
  • the control system of the mixing station corresponding to the port sends the first control signal, which specifically includes:
  • the mixer truck hopper and the mixing station discharge port are aligned, it means that if the unloading starts at this time, the concrete can be accurately unloaded from the mixer station discharge port into the mixer truck hopper. But at the same time, the state of the mixer truck should also be judged, and the stability of the unloading process can be ensured when the mixer truck is in a stationary state. Therefore, after confirming that the mixer truck hopper is aligned with the discharge port of the mixing station, it should also be judged whether the mixer truck corresponding to the mixer truck hopper has stopped at this time.
  • the mixer truck corresponding to the hopper of the mixer truck When judging whether the mixer truck corresponding to the hopper of the mixer truck is stationary, it can be determined whether the position information of the key points of the hopper of the mixer truck in the first video frame changes within the preset distance range within the preset time range. If the position information of the key points of the mixer truck's hopper in the first video frame within the preset time range changes within the preset distance range, it means that the mixer truck has stopped and can start unloading.
  • the preset time range and the preset distance range may be set according to actual needs, which are not specifically limited in this embodiment of the present application.
  • the first control signal can be sent to the control system of the mixing station corresponding to the discharge port of the mixing station, indicating that the hopper of the mixer truck and the discharge port of the mixing station have been prepared, and the discharge can be started.
  • the method for monitoring the intelligent unloading of the mixing station provided by the embodiment of the present application, when the mixer truck receiving hopper and the mixing station discharge port are aligned and the mixer truck corresponding to the mixer truck receiving hopper is in a stationary state, the corresponding discharge port of the mixing station is sent to the mixing station.
  • the control system of the mixing station sends the first control signal, indicating that the unloading can be started at this time, thus ensuring that the concrete can accurately enter the hopper of the mixer truck from the discharge port of the mixing station, avoiding the waste of concrete.
  • the sending of the first control signal to the control system of the mixing station corresponding to the discharge port of the mixing station further includes:
  • a second control signal is generated, and the second control signal is sent to the mixing station control system, where the second control signal is used to instruct to adjust the discharge speed or stop the discharge.
  • the second video frame can be obtained by acquiring the monitoring video frame of the discharge area of the mixing station, and then based on the The second video frame monitors the unloading process.
  • the second video frame can be extracted from the surveillance video.
  • the number of second video frames extracted from the surveillance video may be two or more, and the extracted second video frames may be continuous or discontinuous.
  • the process of extracting the second video frame from the surveillance video is also the sampling process, and the sampling rate can be set according to actual needs, which is not specifically limited in this application.
  • FIG. 2 it is a schematic diagram of the image of the hopper area of the mixer truck in the embodiment of the application.
  • the material level state of the mixer truck receiving hopper can be determined by identifying the second video frame.
  • a recognition method based on statistical pattern recognition, structural pattern recognition or deep learning can be used to identify the material level state of the mixer truck hopper, and determine the material level state of the mixer truck hopper.
  • the material level change information may include material level change direction information and material level change speed information, and the material level change direction information may include material level rise information and material level drop information.
  • the discharge flow refers to the amount of concrete entering the mixer truck's hopper per unit time.
  • the feedback signal returned by the mixing station control system in response to the first control signal will be received. Since the first control signal is used to instruct the start of unloading, When the feedback signal corresponding to the first control signal is received, it means that the mixer truck hopper and the discharge port of the mixing station have been aligned, and the mixer truck is also in a stationary state, and can start unloading. Therefore, in the case of receiving the feedback signal returned by the mixing station control system in response to the first control signal, the material level state of the hopper of the mixer truck can be determined based on the second video frame.
  • the second control signal may be generated according to the determined material level state, and the second control signal may be sent to the mixing station control system. Since the second control signal is determined according to the state of the material level of the hopper of the mixer truck, the second control signal can be used to instruct to adjust the discharge speed or stop the discharge. For example, when it is determined that the height of the material level of the mixer truck is close to a preset critical value, the second control signal can be used to instruct to stop unloading; when it is determined that the material level rising speed of the mixer truck is less than the preset speed, the second control signal It can be used to instruct to speed up the unloading speed.
  • the method for monitoring the intelligent unloading of the mixing station determines the state of the material level of the hopper of the mixer truck through the second monitoring video, generates a second control signal based on the state of the material level, and sends the second control signal to the control of the mixing station. system, adjust the speed of unloading or stop unloading. In this way, the unloading speed can be adjusted according to the actual material level state, and the unloading can be stopped when the material is about to overflow, thereby preventing the waste of concrete caused by the overflow and improving the efficiency of unloading.
  • the method for monitoring the intelligent discharge of the mixing station provided by the embodiment of the present application, the determining the material level state of the hopper of the mixer truck based on the second video frame specifically includes:
  • the material level state of the hopper of the mixer truck is determined.
  • the second video frame after acquiring the second video frame, the second video frame can be globally identified, the image of the mixer truck receiving hopper area in the second video frame can be determined, and the pixel information of the mixer truck receiving hopper area image can be analyzed to determine this When the mixer truck receives the material level status of the hopper.
  • the pixel information of the image may include the value of each pixel in the image.
  • the intelligent unloading monitoring method of the mixing station determines the material level state of the mixing hopper by acquiring the pixel information of the image of the mixing hopper area in the second video frame, and realizes the connection of the mixing truck to the mixer. Real-time monitoring of material level status during hopper discharge process avoids waste caused by overflow.
  • the method for monitoring the intelligent discharge of the mixing station provided by the embodiment of the present application, the determining the material level state of the hopper of the mixer truck based on the second video frame specifically includes:
  • the material level state of the hopper of the mixer truck is determined.
  • pixel-level segmentation can be performed on the image of the mixer truck hopper area in the second video frame, and the pixel-level segmentation can be performed by inputting the mixer truck hopper area image into the semantic segmentation nerve.
  • pixel-level segmentation is performed on the image of the hopper area of the mixer truck through the semantic segmentation neural network model.
  • the outer edge area of the hopper can be specifically constructed based on Mask RCNN.
  • the outer boundary of the concrete and the outer boundary of the hopper of the mixer truck can be obtained.
  • the outer boundary of the concrete area is the outer boundary of the concrete
  • the outer boundary of the outer boundary of the mixer truck hopper is the outer boundary of the mixer truck hopper.
  • the outer boundary of the concrete and the outer boundary of the mixer hopper when monitoring the material level height information in the material level state, you can first determine the concrete section according to the outer boundary of the concrete and the outer boundary of the mixer hopper.
  • the radius and the radius of the mixer's hopper section according to the size of the two radii, it is indirectly judged whether the material level height information exceeds the height of the mixer's hopper and overflow occurs. Since the concrete is installed in the hopper of the mixer truck, the center of the concrete section coincides with the center of the hopper section of the mixer truck, and the two sections are concentric circles. Among them, the radius of the mixer hopper section is a fixed value. When the radius of the concrete section is close to or equal to the radius of the mixer hopper section, it means that flashing will occur.
  • the outer boundary of the concrete and the outer boundary of the hopper of the mixer truck when monitoring the material level change information in the material level state, it can be indirectly determined through the change direction information of the radius of the concrete section in at least two consecutive second video frames. Information about the direction of material level change, if the radius increases, the material level increases, and if the radius decreases, the material level decreases. It is also possible to indirectly determine the change speed information of the material level according to the change speed information of the radius. Here, it can be considered that the change speed information of the material level is the change speed information of the radius.
  • the discharge flow can be estimated according to the speed information of the material level change.
  • the intelligent unloading monitoring method for a mixing station processes the second consecutive video frames through semantic segmentation, and can monitor the material level status of the hopper of the mixer truck during the unloading process, avoiding the occurrence of overflow caused by the material. loss. Compared with directly using the pixel information of the image of the mixer truck's hopper area in the second video frame to determine the material level status of the mixer truck's hopper, the monitoring accuracy is higher.
  • the sending of the first control signal to the control system of the mixing station corresponding to the discharge port of the mixing station further includes:
  • Obtain the monitoring video frame of the unloading area of the mixing station obtain a second video frame, and perform concrete shape recognition on the mixer truck hopper area in the second video frame; generate a third control signal according to the concrete shape recognition result , the third control signal is used to instruct to adjust the discharge speed.
  • the second video frame can be obtained by acquiring the monitoring video frame of the discharge area of the mixing station.
  • the hopper area of the mixer truck in the second video frame is then identified to determine the shape of the concrete.
  • the concrete features of the unloading area extracted by the Mask RCNN backbone network model can be used to add a classification module to perform morphological recognition.
  • the weights of the newly added classification module network are trained based on the concrete morphological labels.
  • a third control signal can be generated and sent to the mixing station control system corresponding to the discharge port of the mixing station, where the third control signal includes an instruction to adjust the discharge speed. For example, when the identified concrete form is too dry, the third control signal may be used to instruct to slow down the discharge speed.
  • the feedback signal returned by the mixing station control system in response to the first control signal will be received. Since the first control signal is used to instruct the start of unloading, When the feedback signal of the first control signal is received, it means that the hopper of the mixer truck and the discharge port of the mixing station have been aligned, and the mixer truck is also in a stationary state, and can start unloading.
  • the intelligent discharge monitoring method of the mixing station provided by the embodiment of the present application, by identifying the concrete form, avoids the concrete being too dry or too thin, and ensures the quality of the concrete.
  • the sending of the first control signal to the control system of the mixing station corresponding to the discharge port of the mixing station further includes:
  • a target control signal is generated, and the target control signal is used to instruct to adjust the discharge speed or stop the discharge.
  • the monitoring video frame of the discharge area of the mixing station can also be obtained first to obtain the second video frame, and then the second video frame can be obtained through By identifying the second video frame, the material level status of the mixer truck’s hopper and the concrete shape recognition result in the mixer truck’s hopper area can be determined. Finally, the target control signal is generated according to the identified material level status and the concrete shape recognition result. Adjust the unloading speed or stop unloading with instructions.
  • the material level state of the mixer truck's hopper and the concrete shape of the mixer truck's hopper area can be determined synchronously, which can not only save the waste caused by overflow, but also ensure the concrete quality.
  • the determination of the position information of the key points of the hopper of the mixer truck based on the first video frame specifically includes:
  • the weight parameters in the identification algorithm model of the mixer truck receiving hopper are obtained by training based on the image samples carrying the label of the mixer truck receiving hopper.
  • the first video frame after acquiring the first video frame, the first video frame can be identified through a preset recognition algorithm model of the mixer truck hopper, and the position information of the key points of the mixer truck hopper can be obtained.
  • the preset recognition algorithm model of the mixer truck hopper may be a Region Proposal Network (RPN) model with the addition of a key point detection head module and a target detection frame head module.
  • RPN Region Proposal Network
  • the location information of the key points determined in the embodiment of the present application may be the coordinates in the pixel coordinate system.
  • the pixel coordinate system is introduced to describe the coordinates of the image points on the digital image after the object is imaged, and can actually be read from the surveillance camera.
  • the coordinate system in which the received information is located. Since the unloading port of the mixing station and the hopper of the mixer truck can be compared in the world coordinate system for alignment, when the unloading port of the mixing station and the hopper of the mixer truck are compared in the world coordinate system for alignment, the determined key points can be compared.
  • the position information is converted to the world coordinate system to make the comparison operation go smoothly.
  • the position information of the key points in the pixel coordinate system can be transformed to the world based on the inverse perspective transformation of the internal parameter matrix and the external parameter matrix of the surveillance camera, or the inverse matrix of the homography matrix. in the coordinate system.
  • Inverse perspective transformation is the inverse of perspective transformation, which refers to projecting an image onto a new viewing plane.
  • the homography matrix refers to the transformation matrix corresponding to the homography, and the homography describes the position mapping relationship between the world coordinate system and the pixel coordinate system.
  • the homography matrix can be calculated by the internal parameter matrix and the external parameter matrix of the surveillance camera.
  • the extrinsic parameter matrix of the surveillance camera can include a rotation matrix and a translation matrix.
  • the internal parameter matrix and external parameter matrix of the surveillance camera can be obtained by calibrating the surveillance camera.
  • the method for monitoring the intelligent unloading of the mixing station provided by the embodiment of the present application, by calculating the position information of the coordinates of the key points of the hopper of the mixer truck, it is convenient to compare whether the hopper of the mixer truck and the discharge port of the mixing station are aligned in the same coordinate system, and improve the the accuracy of judgment.
  • the position information based on the key points of the hopper of the mixer truck and the preset key of the discharge port of the mixing station The location information is used to determine whether the hopper of the mixer truck is aligned with the discharge port of the mixing station, specifically including:
  • the known position information of at least three key points can be used to determine the center of the hopper of the mixer truck.
  • Point coordinates the coordinates of the center point can be coordinates in the world coordinate system.
  • the position of the discharge port of the mixing station is fixed in the mixing station, and the discharge port of the mixing station is a barrel-shaped structure with a circular cross-section, it can be set in the intelligent discharge monitoring system software of the mixing station by setting The position information of the central axis of the discharge port of the batching plant. Finally, compare the vertical shortest distance between the coordinates of the center point of the hopper of the mixer truck and the position information of the central axis of the discharge port of the mixing station. If the vertical shortest distance is within the preset range, it can be determined that the mixer truck is connected to The hopper is already aligned with the discharge opening of the batching plant.
  • the preset range may be set according to actual needs, which is not specifically limited in this embodiment of the present application.
  • the connection of the mixing station is judged. Whether the hopper and the discharge opening of the mixing station are already aligned, so that the hopper of the mixer truck and the discharge opening of the mixing station can be more accurately aligned, reducing concrete waste during the unloading process.
  • the first video frame is obtained by shooting a monitoring camera
  • determining whether the hopper of the mixer truck is aligned with the discharge port of the mixing station based on the position information of the key points of the hopper of the mixer truck and the preset key position information of the discharge port of the mixing station, Also included before:
  • distortion correction is performed on the position information of the key points of the mixer truck hopper, and the position information of the key points of the mixer truck hopper after the distortion correction is obtained. ;
  • the determining whether the hopper of the mixer truck is aligned with the discharge port of the mixing station based on the position information of the key points of the hopper of the mixer truck and the preset key position information of the discharge port of the mixing station includes:
  • the first video frame can be captured by a monitoring camera
  • the monitoring camera can be the original camera hardware of the mixing station, which has the function of obtaining a monitoring image of the unloading area of the mixing station.
  • the installation position of the monitoring camera can be on the wall opposite the discharge port of the mixing station, and the installation height should reach the height that can monitor the complete mixer truck receiving hopper. It can also be at the intersection of the reversing route of the mixer truck and the extension line of the discharge opening of the mixing station and the wall of the mixing station, and the intersection of the straight line passing through the center of the discharge opening of the mixing station and perpendicular to the reversing route of the mixer truck and the wall of the mixing station, respectively.
  • FIG. 3 it is a schematic diagram of the installation positions of the two surveillance cameras in the embodiment of the present application.
  • the surveillance cameras 31 and 32 are both installed on the wall 33 of the mixing station, and both face toward the discharge port 34 of the mixing station.
  • the discharge port 34 of the mixing station is on the reversing route of the mixer truck 35 .
  • the arrow direction in FIG. 3 is the reverse direction of the mixer truck 35 .
  • Distortion correction is realized by the preset internal parameter matrix of the surveillance camera and the preset distortion coefficient.
  • the preset internal parameter matrix of the surveillance camera refers to a matrix composed of parameters only related to the surveillance camera and not related to the external environment.
  • the preset internal parameter matrix of the surveillance camera can be a 3*3 matrix.
  • the preset internal parameter matrix can be obtained by calibrating the surveillance camera by a single-plane checkerboard calibration method.
  • the steps of the calibration method may be: printing a checkerboard, sticking it on a plane as a calibration object, and taking pictures in different directions for the calibration object by adjusting the direction of the calibration object or the monitoring camera. Extract checkerboard corners from photos. Estimate the values of five internal parameters and six external parameters in the case of ideal no distortion, and obtain the preset internal parameter matrix and external parameter matrix of the surveillance camera.
  • the distortion refers to an offset to the projection of the straight line. Distortion is an inherent characteristic of the surveillance camera itself, which is the same as the internal reference of the surveillance camera. It is enough to calibrate the surveillance camera once.
  • the radial distortion of the surveillance camera can be corrected. Radial distortion includes barrel distortion and pincushion distortion. The radial distortion comes from the shape of the lens. For radial distortion, the distortion in the center of the imager is 0, and the distortion becomes more and more severe as it moves to the edge.
  • the preset distortion coefficient of radial distortion can be calculated through the above-mentioned camera calibration process, so as to correct the position information. First, the internal parameter matrix and external parameter matrix of the surveillance camera are obtained, and then the least squares method is used to estimate the distortion coefficient under the actual radial distortion. Finally, the Taylor series expansion is used to correct the position information on the key points.
  • the position information of the key points of the hopper of the mixer truck after the distortion correction can be obtained.
  • the position information of the key points of the hopper of the mixer truck can be obtained.
  • the preset range may be set according to actual needs, which is not specifically limited in this embodiment of the present application.
  • the method for monitoring the intelligent unloading of the mixing station monitors the unloading process by installing a monitoring camera, so that the modification of the mixing station is simple and the hardware and construction costs are low.
  • the installation position of the monitoring camera is far away from the discharge port of the mixing station, so it is not easy to be contaminated with concrete, which affects the monitoring effect.
  • the first video frame is obtained by shooting a monitoring camera, and the first video frame is the change of the shooting angle of the monitoring camera at the shooting angle. After shooting the obtained video frame;
  • determining the location information of the key points of the mixer truck hopper based on the first video frame and then further including:
  • the reference video frame is a video frame obtained by shooting before the shooting angle of the surveillance camera changes;
  • mapping and correcting the position information of the key points of the hopper of the mixer truck to obtain the position information of the key points of the hopper of the mixer truck after the mapping correction;
  • the determining whether the hopper of the mixer truck is aligned with the discharge port of the mixing station based on the position information of the key points of the hopper of the mixer truck and the preset key position information of the discharge port of the mixing station includes:
  • the shooting angles of the video frames extracted from them will also be different, and such different angles will increase the error generated when the video frames are identified. Therefore, in order to make the identification results more accurate Accurately, the first video frame is captured by the surveillance camera after the shooting angle is changed.
  • the region of the mixer truck hopper in the first video frame can be identified, and the position information of the key points of the mixer truck hopper can be determined.
  • the position information of the key points of the mixer truck hopper can be determined.
  • the location information of the key points of the mixer truck's hopper can be mapped and corrected through the reference video frame.
  • the reference video frame is the video frame obtained by the surveillance camera before the shooting angle is changed.
  • the reference video frame can be obtained by the reference calibration board.
  • the reference calibration board can be a reference calibration board with black squares printed on a white background of any plane.
  • the monitoring camera can obtain the reference video frame by shooting the reference calibration board before the shooting angle changes.
  • the reference video frame can be detected at sub-pixel level based on opencv, and a quadratic polynomial can be used to approximate the corner response function in the surrounding 3*3 field, Use the linear solution to find the sub-pixel corner coordinates.
  • the cornerSubPix() function can be used to iteratively calculate the coordinates of the four corners of the reference calibration board in the reference video frame image, that is, the coordinates of the reference calibration board in the pixel coordinate system. Correct the coordinates of the four corner points in the pixel coordinate system according to the above distortion correction method to obtain the coordinates of the four corner points of the undistorted reference calibration plate, and then realize the distortion correction of the first video frame and the reference video frame.
  • the homography matrix of the first video frame and the reference video frame can be calculated.
  • a plane homography is defined as a projection mapping from one plane to another. According to the obtained coordinates of the four corners of the undistorted reference calibration plate and the mapping relationship between the first video frame and the reference video frame, the homography matrix of the first video frame and the reference video frame can be obtained.
  • the key point position information of the mixer truck hopper After acquiring the homography matrix of the first video frame and the reference video frame, the key point position information of the mixer truck hopper can be mapped and corrected. According to the above calculated homography matrix of the surveillance camera, after continuous perspective transformation of the two homography matrices, the position information of the key points on the hopper of the mixer truck after the shooting angle is changed can be mapped and corrected.
  • the position information of the key points of the hopper of the mixer truck after mapping correction and the key position information of the preset discharge port of the mixing station can be used to judge whether the hopper of the mixer truck and the discharge port of the mixing station are aligned. If the distance between the position information of the key points of the hopper of the mixer truck after mapping correction and the key position information of the preset discharge port of the mixing station is within the preset range, it can be judged that the hopper of the mixer truck and the mixing The station discharge openings are already aligned.
  • the preset range may be set according to actual conditions, which is not specifically limited in this embodiment of the present application.
  • the secondary mapping correction can be performed on the coordinates of the key points on the hopper of the mixer truck after the shooting angle is changed.
  • the key point position information of the mixer truck hopper in the first video frame after the shooting angle is transformed through the reference video frame is mapped and corrected, so that after the shooting angle changes, the The position information of the key points of the hopper of the mixer truck without distortion can be obtained, which ensures that the hopper of the mixer truck and the discharge port of the mixing station can be aligned in the same coordinate system.
  • the monitoring camera is specifically a red green blue (Red Green Blue, RGB) camera or a Time of Flight (TOF) deep sense camera.
  • RGB in an RGB camera refers to red, green, and blue, respectively.
  • An RGB camera is given the above three basic color components by three different cables. This type of camera usually uses three separate charge-coupled device image sensors to acquire the above three color signals. RGB cameras are often used to do very accurate color image capture. The RGB camera can be used to obtain clear surveillance video at the discharge port of the batching plant.
  • the TOF depth camera is an active camera, which can perform depth measurement by illuminating an area with an infrared light source and observing the time it takes to arrive at the scene and return to the scene.
  • TOF depth cameras capture the entire field of view of each light pulse without any moving parts. This allows fast data collection.
  • the TOF depth camera has a long measurement distance and is not affected by illumination changes and object textures. It has a high frame rate and low software complexity.
  • the depth of each pixel in the monitoring image can be obtained by using the TOF deep-sensing camera, which increases the effect of identifying the discharge information of the discharge port of the subsequent batching plant, and makes the identification of the discharge process more accurate.
  • the method for monitoring the intelligent unloading of the mixing station obtains the first video frame by acquiring the monitoring video frame of the unloading area of the mixing station, and when the mixer truck hopper appears in the first video frame, the first video frame is processed. Processing, calculate the coordinates of the key points of the mixer truck's hopper, and then calculate the center point coordinates of the mixer's hopper through the key point coordinates of the mixer's hopper, and calculate the center point of the mixer's hopper and the discharge of the mixing station in the same coordinate system. The vertical shortest distance between the position information of the central axis of the mouth is used to determine whether the two are aligned.
  • the pixel-level segmentation is used to segment the mixer hopper area in the monitoring video frame, and the material level status of the mixer hopper during the unloading process is monitored; The segmented concrete state is identified, and the concrete state in the current surveillance video frame is obtained.
  • the method does not need to manually judge whether the mixer truck's hopper and the discharge port of the mixing station are aligned, is not easily affected by human beings, and can be applied to mixer trucks of various heights and hopper sizes. It avoids concrete waste caused by misalignment during the unloading process, and can monitor the unloading process to avoid losses caused by overflow.
  • FIG. 4 is a schematic flow chart of the specific flow of the monitoring method for intelligent discharge of a mixing station provided by an embodiment of the present application. As shown in Figure 4, the method includes:
  • FIG. 5 is a schematic structural diagram of an intelligent unloading monitoring system for a mixing station provided by an embodiment of the present application. As shown in FIG. 5 , the system includes a monitoring video acquisition module 501 and a discharge monitoring module 502 . in:
  • the monitoring video acquisition module 501 is used to acquire the monitoring video frame of the unloading area of the mixing station to obtain the first video frame;
  • the unloading monitoring module 502 is configured to detect and identify the first video frame based on a preset algorithm model, and monitor the unloading of the concrete mixing plant based on the identification result.
  • the discharge monitoring module specifically includes:
  • a position information acquisition sub-module configured to determine the position information of the key points of the mixer truck hopper based on the first video frame if the identification result is that the first video frame includes a mixer truck hopper;
  • the alignment sub-module is used to determine whether the mixer truck hopper and the mixing station discharge port are aligned based on the position information of the key points of the mixer truck receiving hopper and the preset key position information of the mixing station discharge port ;
  • the first control signal generation sub-module is used to send a first control signal to the mixing station control system corresponding to the mixing station discharge port if it is determined that the mixer truck receiving hopper is aligned with the mixing station discharge port, so that the The first control signal is used to instruct the start of unloading.
  • the first control signal generation sub-module specifically includes:
  • the first control signal sending unit is used to unload the mixer truck to the mixing station if it is determined that the mixer truck receiving hopper is aligned with the discharge port of the mixing station, and the mixer truck corresponding to the mixer truck receiving hopper is in a stationary state.
  • the mixing station control system corresponding to the material port sends the first control signal.
  • the discharge monitoring module further includes:
  • a material level state determination sub-module is used to acquire the monitoring video frame of the unloading area of the mixing station, obtain a second video frame, and determine the material level state of the mixer truck hopper based on the second video frame;
  • the second control signal generating sub-module is configured to generate a second control signal according to the material level state, and send the second control signal to the mixing station control system, where the second control signal is used to instruct the adjustment of unloading speed or stop unloading.
  • the material level state determination sub-module specifically includes:
  • a material level state determining unit configured to determine the material level state of the mixer truck receiving hopper based on the pixel information of the image of the mixer truck receiving hopper area in the second video frame.
  • the material level state determination sub-module specifically further includes:
  • the segmentation unit is configured to perform pixel-level segmentation on the image of the mixer truck hopper area in the second video frame, and based on the result of the pixel-level segmentation, determine the concrete outer edge boundary in the mixer truck hopper area and the mixer truck connection area.
  • An outer boundary determination unit configured to determine the material level state of the mixer truck hopper based on the concrete outer boundary boundary and the mixer truck hopper outer boundary.
  • the discharge monitoring module further includes:
  • a concrete shape recognition submodule used for obtaining the monitoring video frame of the unloading area of the mixing station, obtaining a second video frame, and performing concrete shape recognition on the mixer truck hopper area in the second video frame;
  • the third control signal generating sub-module is configured to generate a third control signal according to the concrete shape identification result, where the third control signal is used to instruct to adjust the discharge speed.
  • the location information acquisition sub-module specifically includes:
  • the key point position information acquisition unit is used to input the first video frame into the preset identification algorithm model of the mixer truck hopper, and obtain the first video frame output by the mixer truck hopper identification algorithm model.
  • the weight parameters in the identification algorithm model of the mixer truck receiving hopper are obtained by training based on the image samples carrying the label of the mixer truck receiving hopper.
  • the alignment sub-module specifically includes:
  • a center point position information acquisition unit configured to determine the position information of the center point of the mixer truck hopper based on the position information of the key points of the mixer truck hopper;
  • an alignment unit configured to determine, based on the position information of the center point of the hopper of the mixer truck and the preset position information of the central axis of the discharge port of the mixing station, to determine the hopper of the mixer truck and the discharge of the mixing station Are the mouths aligned.
  • the first video frame is obtained by shooting a monitoring camera; correspondingly, the system further includes:
  • the distortion correction sub-module is used to perform distortion correction on the position information of the key points of the mixer truck receiving hopper based on the preset internal parameter matrix and preset distortion coefficient of the monitoring camera, and obtain the distortion correction of the mixer truck connection.
  • the distortion correction alignment sub-module is used to determine the position information of the mixer truck receiving hopper and the mixer based on the position information of the key points of the mixer truck receiving hopper after the distortion correction and the preset key position information of the discharge port of the mixing station. Whether the discharge port of the station is aligned.
  • the first video frame is obtained by shooting a monitoring camera, and the first video frame is the change of the shooting angle of the monitoring camera at the shooting angle. Then shoot the obtained video frames; the system further includes:
  • a homography matrix acquisition submodule is used to obtain a homography matrix based on the first video frame and the reference video frame, and the reference video frame is the video frame obtained before the shooting angle of the monitoring camera changes;
  • mapping corrector module configured to perform mapping correction on the position information of the key points of the mixer truck hopper based on the homography matrix, and obtain the position information of the key points of the mixer truck hopper after the mapping correction;
  • the mapping correction and alignment sub-module is used to determine the position information of the mixer truck receiving hopper and the mixer based on the position information of the key points of the mixer truck receiving hopper and the preset key position information of the mixing station discharge port after the mapping correction. Whether the discharge port of the station is aligned.
  • the monitoring camera is specifically an RGB camera or a TOF depth-sensing camera.
  • the functions of the modules in the intelligent unloading monitoring system for the mixing station correspond one-to-one with the operation procedures of the steps in the above method embodiments, and the achieved effects are also the same.
  • the above implementation For example, this is not repeated in this embodiment of the present application.
  • FIG. 6 is a schematic diagram of a specific structure of an intelligent discharge monitoring system for a mixing station provided in an embodiment of the present application.
  • the system includes: a monitoring video acquisition module, a reversing alignment monitoring service module and an overflow monitoring service module.
  • the monitoring video acquisition module is used to acquire the monitoring video at the discharge port of the mixing station.
  • the surveillance video acquisition module is also used for video streaming analysis and preprocessing such as denoising and enhancement of the parsed video frames.
  • the reversing alignment monitoring service module is used to monitor whether the hopper of the mixer truck and the discharge port of the mixing station are aligned.
  • the overflow monitoring service module is used to monitor the discharge process.
  • the reversing alignment monitoring service module specifically includes:
  • the position information acquisition module is used to identify the preprocessed first video frame, and obtain the position information of the key points of the mixer truck receiving the hopper;
  • the camera calibration module is used to calibrate the surveillance camera, and perform distortion correction on the acquired position information of the key points of the mixer truck receiving hopper;
  • the world coordinate system mapping module is used to map the position information of the key points of the hopper of the mixer truck after distortion correction, that is, the coordinates of the key points of the hopper of the mixer truck in the pixel coordinate system to the world coordinate system;
  • the reversing monitoring module that is, the alignment sub-module, is used to judge whether the hopper of the mixer truck and the discharge port of the mixing station are aligned; in the case of alignment, the overflow monitoring service is performed; in the case of misalignment, the first video frame is re-acquired to identify;
  • the mapping correction module is used for monitoring the position information of the key points on the mixer truck in the first video frame obtained by the monitoring camera after the shooting angle changes when the shooting angle changes after the current video frame is obtained. Perform mapping correction;
  • the image post-processing module is used to perform image post-processing such as enhancement on the video frames that have been aligned in reverse.
  • the overflow monitoring service module specifically includes:
  • the image preprocessing module is used to perform image preprocessing such as cropping and denoising on the enhanced video frames that have been aligned in reverse;
  • the segmentation model module that is, the segmentation sub-module, is used for pixel-level segmentation of the mixer truck receiving hopper area;
  • a pixel edge monitoring module used to determine the outer boundary of the concrete in the hopper area of the mixer truck and the outer edge boundary of the hopper of the mixer truck;
  • the concrete state recognition module that is, the concrete shape recognition sub-module, is used to perform the concrete shape recognition on the hopper area of the mixer truck in the current video frame;
  • the flow rate monitoring module is used to monitor the flow information of concrete
  • the overflow recognition module is used to determine whether the concrete in the current video frame is overflowing.
  • FIG. 7 illustrates a schematic diagram of the physical structure of an electronic device.
  • the electronic device may include: a processor (processor) 710, a communication interface (Communications Interface) 720, a memory (memory) 730 and a communication bus 740,
  • the processor 710 , the communication interface 720 , and the memory 730 communicate with each other through the communication bus 740 .
  • the processor 710 can call the logic instructions in the memory 730 to execute the method for monitoring the intelligent unloading of the mixing station provided in the above embodiments, the method comprising: acquiring the monitoring video frame of the unloading area of the mixing station, and obtaining the first video frame; Based on a preset algorithm model, the first video frame is detected and identified, and based on the identification result, the discharge of the concrete mixing plant is monitored.
  • the above-mentioned logic instructions in the memory 730 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.
  • the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution.
  • the computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .
  • the present application also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer
  • the computer can implement the intelligent unloading monitoring method of the mixing station provided in the above embodiments.
  • the method includes: acquiring the monitoring video frame of the unloading area of the mixing station, and obtaining the first video frame; based on a preset algorithm model, The first video frame is detected and identified, and based on the identification result, the discharge of the concrete mixing plant is monitored.
  • the present application also provides a non-transitory computer-readable storage medium on which a computer program is stored, and when the computer program is executed by the processor, the intelligent discharge monitoring of the mixing station provided in the above embodiments can be implemented.
  • the method includes: acquiring a monitoring video frame of a discharge area of a mixing station to obtain a first video frame; detecting and identifying the first video frame based on a preset algorithm model, and monitoring the concrete mixing station based on the identification result. of unloading.
  • the device embodiments described above are only illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed over multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.
  • each embodiment can be implemented by means of software plus a necessary general hardware platform, and certainly can also be implemented by hardware.
  • the above-mentioned technical solutions can be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic A disc, an optical disc, etc., includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the methods described in various embodiments or some parts of the embodiments.

Abstract

本申请提供一种搅拌站智能卸料监控方法及系统。该方法包括：获取搅拌站卸料区域的监控视频帧，得到第一视频帧；基于预设的算法模型，对所述第一视频帧进行检测识别，并基于识别结果，监控混凝土搅拌站的卸料。所述方法不需要人工参与卸料监控过程，可以实现卸料监控的智能化，保证卸料过程的监控效果，同时也提高了控制系统的卸料效率。

Description

搅拌站智能卸料监控方法及系统
相关申请的交叉引用
本申请要求于2021年03月19日提交的申请号为202110296818.2，名称为“搅拌站智能卸料监控方法及系统”以及于2021年07月02日提交的申请号为202110751261.7，名称为“搅拌站智能卸料监控方法及系统”的中国专利申请的优先权，其通过引用方式全部并入本文。
技术领域
本申请涉及作业机械技术领域，尤其涉及搅拌站智能卸料监控方法及系统。
背景技术
目前，混凝土应用越来越多，混凝土从搅拌站生产后需要通过搅拌车运送到施工现场进行泵送、浇灌等作业。混凝土从搅拌站到搅拌车的过程即为搅拌站卸料过程，而搅拌站卸料过程中经常会出现溢料现象，这将造成混凝土浪费，进而增加成本。因此，防止搅拌站卸料过程中出现溢料现象至关重要。
相关技术中，搅拌站卸料过程基本依赖于人工监控，如人工监控判断搅拌车接料斗与搅拌站卸料口是否对齐、人工判断是否开始卸料等，搅拌站卸料监控的自动化程度较低。
发明内容
本申请提供一种搅拌站智能卸料监控方法及系统，用以解决现有技术搅拌站卸料过程基本依赖于人工监控，搅拌站卸料监控的自动化程度较低的技术问题。
本申请提供一种搅拌站智能卸料监控方法，包括：
获取搅拌站卸料区域的监控视频帧，得到第一视频帧；
基于预设的算法模型，对所述第一视频帧进行检测识别，并基于识别结果，监控混凝土搅拌站的卸料。
根据本申请提供的搅拌站智能卸料监控方法，所述基于识别结果，监控混凝土搅拌站的卸料，具体包括：
若所述识别结果为所述第一视频帧中包含搅拌车接料斗，则基于所述第一视频帧，确定所述搅拌车接料斗的关键点的位置信息；
基于所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐；
若确定所述搅拌车接料斗与所述搅拌站卸料口对齐，则向所述搅拌站卸料口对应的搅拌站控制系统发送第一控制信号，所述第一控制信号用于指示开始卸料。
根据本申请提供的搅拌站智能卸料监控方法，所述若确定所述搅拌车接料斗与所述搅拌站卸料口对齐，则向所述搅拌站卸料口对应的搅拌站控制系统发送第一控制信号，具体包括：
若确定所述搅拌车接料斗与所述搅拌站卸料口对齐，且所述搅拌车接料斗对应的搅拌车处于停稳状态，则向所述搅拌站卸料口对应的搅拌站控制系统发送所述第一控制信号。
根据本申请提供的搅拌站智能卸料监控方法，所述向所述搅拌站卸料口对应的搅拌站控制系统发送第一控制信号，之后还包括：
获取所述搅拌站卸料区域的监控视频帧，得到第二视频帧，基于所述第二视频帧，确定所述搅拌车接料斗的料位状态；
根据所述料位状态，生成第二控制信号，并向所述搅拌站控制系统发送所述第二控制信号，所述第二控制信号用于指示调整卸料速度或停止卸料。
根据本申请提供的搅拌站智能卸料监控方法，所述基于所述第二视频帧，确定所述搅拌车接料斗的料位状态，具体包括：
基于所述第二视频帧中的搅拌车接料斗区域图像的像素信息，确定所述搅拌车接料斗的料位状态。
根据本申请提供的搅拌站智能卸料监控方法，所述基于所述第二视频帧，确定所述搅拌车接料斗的料位状态，具体包括：
对所述第二视频帧中的搅拌车接料斗区域图像进行像素级分割，并基于像素级分割的结果，确定所述搅拌车接料斗区域内混凝土外沿边界以及 搅拌车接料斗外沿边界；
基于所述混凝土外沿边界与所述搅拌车接料斗外沿边界，确定所述搅拌车接料斗的料位状态。
根据本申请提供的搅拌站智能卸料监控方法，所述向所述搅拌站卸料口对应的搅拌站控制系统发送第一控制信号，之后还包括：
获取所述搅拌站卸料区域的监控视频帧，得到第二视频帧，并对所述第二视频帧中的搅拌车接料斗区域进行混凝土形态识别；
根据混凝土形态识别结果，生成第三控制信号，所述第三控制信号用于指示调整卸料速度。
根据本申请提供的搅拌站智能卸料监控方法，所述基于所述第一视频帧，确定所述搅拌车接料斗的关键点的位置信息，具体包括：
将所述第一视频帧输入至预设的搅拌车接料斗识别算法模型，得到所述搅拌车接料斗识别算法模型输出的所述第一视频帧中所述搅拌车接料斗的关键点的位置信息；
其中，所述搅拌车接料斗识别算法模型中的权重参数基于携带有搅拌车接料斗标签的图像样本训练得到。
根据本申请提供的搅拌站智能卸料监控方法，所述基于所述搅拌车接料斗的关键点的位置信息以及预设的所述搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐，具体包括：
基于所述搅拌车接料斗的关键点的位置信息，确定所述搅拌车接料斗的中心点的位置信息；
基于所述搅拌车接料斗的中心点的位置信息以及预设的所述搅拌站卸料口的中轴线的位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐。
根据本申请提供的搅拌站智能卸料监控方法，所述第一视频帧通过监控摄像头拍摄获得；
相应的，所述基于所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐，之前还包括：
基于所述监控摄像头的预设内参矩阵以及预设畸变系数，对所述搅拌 车接料斗的关键点的位置信息进行畸变矫正，得到畸变矫正后的所述搅拌车接料斗的关键点的位置信息；
所述基于所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐，具体包括：
基于畸变矫正后的所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐。根据本申请提供的搅拌站智能卸料监控方法，所述第一视频帧通过监控摄像头拍摄获得，且所述第一视频帧为所述监控摄像头在拍摄角度变化之后拍摄获得的视频帧；
相应的，所述基于所述第一视频帧，确定所述搅拌车接料斗的关键点的位置信息，之后还包括：
基于所述第一视频帧以及基准视频帧，得到单应性矩阵，所述基准视频帧为所述监控摄像头的拍摄角度发生变化之前拍摄获得的视频帧；
基于所述单应性矩阵，对所述搅拌车接料斗的关键点的位置信息进行映射校正，得到映射校正后的所述搅拌车接料斗的关键点的位置信息；
所述基于所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐，具体包括：
基于映射校正后的所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐。
根据本申请提供的搅拌站智能卸料监控方法，所述监控摄像头具体为RGB摄像头或TOF深感摄像头。
本申请还提供一种搅拌站智能卸料监控系统，包括：
监控视频获取模块，用于获取搅拌站卸料区域的监控视频帧，得到第一视频帧；
卸料监控模块，用于基于预设的算法模型，对所述第一视频帧进行检测识别，并基于识别结果，监控混凝土搅拌站的卸料。本申请还提供一种电子设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的 计算机程序，所述处理器执行所述程序时实现如上述任一种所述搅拌站智能卸料监控方法的步骤。
本申请还提供一种非暂态计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现如上述任一种所述搅拌站智能卸料监控方法的步骤。
本申请提供的搅拌站智能卸料监控，通过获取搅拌站卸料区域的监控视频帧，并基于预设的算法模型，对监控视频进行识别，并基于识别结果，对搅拌站进行卸料流程监控。所述方法不需要人工参与卸料监控过程，可以实现卸料监控的自动化、智能化，提高了搅拌站卸料监控的自动化程度，保证了卸料监控的效果，同时也提高了卸料监控的效率。
附图说明
为了更清楚地说明本申请或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的搅拌站智能卸料监控方法的流程示意图；
图2是本申请实施例中搅拌车接料斗区域图像的示意图；
图3是本申请实施例中两个监控摄像头的安装位置示意图；
图4是本申请实施例提供的搅拌站智能卸料监控方法的具体流程示意图；
图5是本申请实施例提供的搅拌站智能卸料监控系统结构示意图；
图6是本申请实施例提供的搅拌站智能卸料监控系统的具体结构示意图；
图7是本申请提供的电子设备的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合本申请中的附图，对本申请中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其 他实施例，都属于本申请保护的范围。
搅拌站卸料过程中经常会发生溢料，这会造成混凝土的浪费，增加成本。现有技术中为了防止溢料，通常会在搅拌站出料口处固定安装搅拌站卸料防溢料装置。该装置通过固定在拢料斗外侧的测距传感器监测当前料位高度，并将当前料位高度传送至搅拌车接料检测装置，并通过流量调整装置控制卸料门。
但由于搅拌车的大小和高度各有不同，测距传感器无法适应于各种车型的搅拌车。且由于搅拌车司机在倒车时，只能在指定的区间范围内对齐，搅拌车接料斗的大小也不同，这就会使得相同高度的搅拌车通过测距传感器识别料位高度的效果不同。测距传感器测量的是点到点的距离，搅拌车接料斗内料位高低分布不均匀，测距传感器测量的效果不佳。测距传感器安装在拢料斗的外侧，容易被混凝土和粉状水泥遮蔽，影响测量的效果。且该方案也无法判断搅拌车接料斗和搅拌站卸料口之间是否对齐，仍需要人工判断，也无法测量料位高度或直接监控卸料。为此，本申请实施例中提供了一种搅拌站智能卸料监控方法。
图1是该方法的流程示意图。如图1所示，该方法包括：
S1，获取搅拌站卸料区域的监控视频帧，得到第一视频帧；
S2，基于预设的算法模型，对所述第一视频帧进行检测识别，并基于识别结果，监控混凝土搅拌站的卸料。
具体地，本申请实施例中提供的搅拌站智能卸料监控方法，其执行主体可以是嵌入式边缘算力模块，该模块可以配置在搅拌站智能卸料监控系统中。其中，搅拌站智能卸料监控系统可以设置在搅拌站的中控室中。搅拌站经由搅拌站卸料口以及搅拌车接料斗将搅拌好的混凝土送入至搅拌车中，这一过程即为卸料过程，本申请实施例中提供的搅拌站智能卸料监控方法，即是用于对卸料过程的监控，例如对搅拌车接料斗与搅拌站卸料口是否对齐进行监控、对搅拌站卸料口的卸料信息进行监控以及对搅拌车接料斗区域进行混凝土形态识别等。
首先执行步骤S1。其中，第一视频帧可以通过监控摄像头拍摄得到并上传至嵌入式边缘算力模块。嵌入式边缘算力模块获取的搅拌站卸料区域的监控视频帧中包含有搅拌站卸料口，监控视频帧的获取可以实时进行， 以展示混凝土搅拌站卸料口的实时画面。
然后执行步骤S2。其中，预设的算法模型可以是基于人工智能视觉算法或与之相关的算法构建的模型，本申请实施例中可以采用基于深度学习算法构建的模型，对第一视频帧进行特征提取，基于提取到的特征对第一视频帧进行检测识别，识别到的结果可以是当前混凝土在搅拌车接料斗内的料位信息，根据料位信息可以判断当前混凝土是否已经发生溢料、混凝土的卸料流量、混凝土的料位变化信息以及当前混凝土的形态信息。通过上述这些识别到的结果即可以对混凝土搅拌站进行卸料监控。
本申请实施例提供的搅拌站智能卸料监控，通过获取搅拌站卸料区域的第一视频帧，并基于预设的算法模型，对第一视频帧进行识别，基于识别结果，对搅拌站进行卸料流程监控。所述方法不需要人工参与卸料监控过程，可以实现卸料监控的自动化，保证卸料监控的效果，同时也提高了卸料监控的效率。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控方法，所述基于识别结果，监控混凝土搅拌站的卸料，具体包括：
若所述识别结果为所述第一视频帧中包含搅拌车接料斗，则基于所述第一视频帧，确定所述搅拌车接料斗的关键点的位置信息；
基于所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐；
若确定所述搅拌车接料斗与所述搅拌站卸料口对齐，则向所述搅拌站卸料口对应的搅拌站控制系统发送第一控制信号，所述第一控制信号用于指示开始卸料。
具体地，由于搅拌车是逐步进入监控范围，因此在监控视频帧中会出现不存在搅拌车接料斗的情况，因此需要先对第一视频帧进行全局识别，以判断第一视频帧中是否出现了搅拌车接料斗。在确定第一视频帧中存在搅拌车接料斗时，对第一视频帧进行处理，获取搅拌车接料斗的关键点的位置信息。关键点是搅拌车接料斗外沿边界上、具有一定特征的点。也就是说，获取到的关键点位于搅拌车接料斗截面上，距离搅拌车接料斗的中心的距离相等。此处的位置信息可以是坐标。搅拌车接料斗的关键点的位置信息可以通过坐标表示，即搅拌车接料斗的关键点坐标。
本申请实施例中，可以通过对加入关键点检测头部模块和目标检测框头部模块的区域生成网络(Region Proposal Network，RPN)模型进行大量训练，使其可以直接返回第一视频帧中搅拌车接料斗的关键点坐标。预设的搅拌站卸料口的关键位置信息可以是事先确定的搅拌站卸料口的中轴线的位置信息，该中轴线的位置信息可以根据实际情况进行设置，本申请实施例对此不作具体限定。
通过比较搅拌车接料斗的关键点的位置信息和预设的搅拌站卸料口的关键位置信息之间的距离，可以判断搅拌车接料斗和搅拌站卸料口是否对齐。若搅拌车接料斗的关键点的中心的位置信息和预设的搅拌站卸料口的关键位置信息之间的垂直最短距离在预设范围内，则说明搅拌车接料斗和搅拌站卸料口已经对齐。其中，预设范围可以根据实际的需要进行设置，本申请实施例对此不作具体限定。
当搅拌车接料斗和搅拌站卸料口已经对齐的情况下，说明此时已经可以开始进行卸料，向搅拌站卸料口对应的搅拌站控制系统发送第一控制信号。第一控制信号中包含了开始卸料的指令，即当搅拌站控制系统收到第一控制信号后，可以开始进行卸料。
本申请实施例中，搅拌车接料斗的关键点的位置信息和预设的搅拌站卸料口的关键位置信息可以是世界坐标系下的坐标，也可以是任意坐标系下的坐标，只需要保证上述二者在同一坐标系下进行比较即可，本申请对坐标系不作具体限定。其中，世界坐标系是指用户定义的三维世界的坐标系，为了描述目标物在真实世界里的位置而引入。
本申请实施例提供的搅拌站智能卸料监控方法，通过搅拌站卸料区域的监控视频帧，对监控视频帧中出现了搅拌车接料斗的第一视频帧进行识别，获取搅拌车接料斗关键点的位置信息，再将搅拌车接料斗关键点的位置信息与搅拌站卸料口的关键位置信息进行比较，由此判断搅拌车接料斗和搅拌站卸料口之间是否对齐，并在确定二者对齐之后，向搅拌站卸料口对应的搅拌站控制系统发送第一控制信号，指示此时可以开始卸料。该方法通过监控视频帧对卸料过程进行监控，能够自动检测卸料时搅拌车接料斗和搅拌站卸料口是否对齐，不需要人工操作，不易受到人为的影响，且能够避免因搅拌车接料斗和搅拌站卸料口未对齐而造成卸料过程中混凝 土的浪费。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控方法，所述若确定所述搅拌车接料斗与所述搅拌站卸料口对齐，则向所述搅拌站卸料口对应的搅拌站控制系统发送第一控制信号，具体包括：
若确定所述搅拌车接料斗与所述搅拌站卸料口对齐，且所述搅拌车接料斗对应的搅拌车处于停稳状态，则向所述搅拌站卸料口对应的搅拌站控制系统发送所述第一控制信号。
在本申请实施例中，当搅拌车接料斗和搅拌站卸料口对齐后，说明若此时开始卸料，可以准确地将混凝土从搅拌站卸料口卸入搅拌车接料斗中。但同时还应该判断搅拌车的状态，在搅拌车处于停稳状态时才能保证卸料过程的稳定。因此，在确定搅拌车接料斗与搅拌站卸料口对齐后，还应该判断搅拌车接料斗对应的搅拌车此时是否已经停稳。
在判断搅拌车接料斗对应的搅拌车是否停稳时，可以判断预设时间范围内第一视频帧中的搅拌车接料斗的关键点的位置信息变化是否在预设的距离范围内，如果在预设时间范围内第一视频帧中的搅拌车接料斗的关键点的位置信息变化在预设的距离范围内，则说明此时搅拌车已经停稳，可以开始进行卸料。其中，预设时间范围和预设的距离范围可以根据实际需要进行设置，本申请实施例对此不作具体限定。此时可以向搅拌站卸料口对应的搅拌站控制系统发送第一控制信号，表明此时搅拌车接料斗和搅拌站卸料口均已准备完毕，可以开始实施卸料。
本申请实施例提供的搅拌站智能卸料监控方法，通过在搅拌车接料斗和搅拌站卸料口对齐且搅拌车接料斗对应的搅拌车处于停稳状态时，向搅拌站卸料口对应的搅拌站控制系统发送第一控制信号，表明此时可以开始进行卸料，从而保证了混凝土能准确的从搅拌站卸料口进入搅拌车接料斗，避免了混凝土的浪费。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控方法，所述向所述搅拌站卸料口对应的搅拌站控制系统发送第一控制信号，之后还包括：
获取所述搅拌站卸料区域的监控视频帧，得到第二视频帧，基于所述第二视频帧，确定所述搅拌车接料斗的料位状态；
根据所述料位状态，生成第二控制信号，并向所述搅拌站控制系统发送所述第二控制信号，所述第二控制信号用于指示调整卸料速度或停止卸料。
具体地，本申请实施例中，在向搅拌站卸料口对应的搅拌站控制系统发送第一控制信号之后，可以通过获取搅拌站卸料区域的监控视频帧，得到第二视频帧，然后基于第二视频帧对卸料过程进行监控。其中，第二视频帧可以从监控视频中抽取得到。从监控视频中抽取得到的第二视频帧的数量可以是两个或两个以上，所抽取的第二视频帧可以是连续的，也可以是不连续的。从监控视频中抽取第二视频帧的过程也即是采样的过程，采样率可以根据实际需要进行设置，本申请对此不做具体限定。如图2所示，为本申请实施例中搅拌车接料斗区域图像的示意图。
在得到第二视频帧后，可以通过对第二视频帧进行识别，确定搅拌车接料斗的料位状态。本申请实施例中，可以采用基于统计模式识别、结构模式识别或基于深度学习的识别方法对搅拌车接料斗的料位状态进行识别，确定搅拌车接料斗的料位状态。
搅拌车接料斗的料位状态可以包括搅拌车接料斗内的料位高度信息、料位变化信息以及卸料流量。确定料位高度信息可以是确定搅拌车接料斗是否发生溢料；确定料位变化信息可以是确定当前搅拌车接料斗内的料位变化速度。其中，料位变化信息可以包括料位变化方向信息和料位变化速度信息，料位变化方向信息可以包括料位上涨信息和料位下降信息。卸料流量是指单位时间内进入搅拌车接料斗的混凝土量。
本申请实施例中，在向搅拌站控制系统发送第一控制信号后，会接收到搅拌站控制系统响应于第一控制信号返回的反馈信号，由于第一控制信号是用于指示开始卸料，在收到第一控制信号对应的反馈信号时，说明搅拌车接料斗和搅拌站卸料口已经对齐且搅拌车也处于停稳状态，可以开始卸料。因此，可以在接收到搅拌站控制系统响应于第一控制信号返回的反馈信号的情况下，基于第二视频帧，确定搅拌车接料斗的料位状态。
本申请实施例中，可以根据确定的料位状态，生成第二控制信号，将第二控制信号发送至搅拌站控制系统。由于第二控制信号是根据搅拌车接料斗的料位状态确定的，因此第二控制信号可以用于指示调整卸料速度或 停止卸料。例如，当确定搅拌车的料位高度接近预设的临界值时，第二控制信号可以用于指示停止卸料；当确定搅拌车的料位上涨速度小于预设的速度时，第二控制信号可以用于指示调快卸料速度。
本申请实施例提供的搅拌站智能卸料监控方法，通过第二监控视频确定搅拌车接料斗的料位状态，并基于料位状态生成第二控制信号，将第二控制信号发送至搅拌站控制系统，调整卸料的速度或停止卸料。这样可以根据实际的料位状态对卸料的速度进行调整并能够在快要溢料时停止卸料，防止了混凝土因溢料造成的浪费，且能够提高卸料的效率。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控方法，所述基于所述第二视频帧，确定所述搅拌车接料斗的料位状态，具体包括：
基于所述第二视频帧中的搅拌车接料斗区域图像的像素信息，确定所述搅拌车接料斗的料位状态。
具体地，在获取第二视频帧后，可以对第二视频帧进行全局识别，确定第二视频帧中的搅拌车接料斗区域图像，对搅拌车接料斗区域图像的像素信息进行分析，确定此时搅拌车接料斗的料位状态。其中，图像的像素信息可以包括图像中每个像素点的值。
本申请实施例提供的搅拌站智能卸料监控方法，通过获取第二视频帧中的搅拌车接料斗区域的图像的像素信息，从而确定搅拌车接料斗的料位状态，实现了对搅拌车接料斗卸料过程中料位状态的实时监测，避免了因溢料造成的浪费。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控方法，所述基于所述第二视频帧，确定所述搅拌车接料斗的料位状态，具体包括：
对所述第二视频帧中的搅拌车接料斗区域图像进行像素级分割，并基于像素级分割的结果，确定所述搅拌车接料斗区域内混凝土外沿边界以及搅拌车接料斗外沿边界；
基于所述混凝土外沿边界与所述搅拌车接料斗外沿边界，确定所述搅拌车接料斗的料位状态。
本申请实施例中，在获取第二视频帧后，可以对第二视频帧中的搅拌 车接料斗区域图像进行像素级分割，像素级分割可以通过将搅拌车接料斗区域图像输入至语义分割神经网络模型中，通过语义分割神经网络模型对搅拌车接料斗区域图像进行像素级分割，由语义分割神经网络模型输出搅拌车接料斗区域图像中的混凝土区域、搅拌车接料斗内壁区域及搅拌车接料斗外沿区域。语义分割神经网络模型具体可以是基于Mask RCNN构建。
本申请实施例中，通过像素级分割的结果，可以得到混凝土外沿边界和搅拌车接料斗外沿边界。混凝土区域的外边界即为混凝土外沿边界，搅拌车接料斗外沿区域的外边界即为搅拌车接料斗外沿边界。然后根据混凝土外沿边界与搅拌车接料斗外沿边界，确定所述搅拌车接料斗的料位状态。其中，料位状态与上述料位状态意义相同。
根据混凝土外沿边界与搅拌车接料斗外沿边界，对料位状态中的料位高度信息进行监控时，可以先根据混凝土外沿边界与搅拌车接料斗外沿边界，分别确定出混凝土截面的半径以及搅拌车接料斗截面的半径，根据两个半径的大小间接判断料位高度信息是否超过搅拌车接料斗高度进而发生溢料。由于混凝土装在搅拌车接料斗中，所以混凝土截面的圆心与搅拌车接料斗截面的圆心重合，二者的截面是同心圆。其中，搅拌车接料斗截面的半径为固定值。当混凝土截面的半径接近或等于搅拌车接料斗截面的半径时，表示将要发生溢料。
根据混凝土外沿边界与搅拌车接料斗外沿边界，对料位状态中的料位变化信息进行监控时，可以通过至少两个连续第二视频帧中混凝土截面的半径的变化方向信息，间接确定料位变化方向信息，如果半径增加则料位上涨，如果半径减少则料位下降。还可以根据半径的变化速度信息，间接确定料位变化速度信息，此处可以认为料位变化速度信息即为半径的变化速度信息。
根据混凝土外沿边界与搅拌车接料斗外沿边界，对料位状态中的卸料流量进行监控时，可以根据料位变化速度信息，估计卸料流量。
本申请实施例提供的搅拌站智能卸料监控方法，通过语义分割对连续第二视频帧进行处理，可以对卸料过程中搅拌车接料斗的料位状态进行监控，避免了因溢料产生的损失。相比于直接利用第二视频帧中的搅拌车接料斗区域图像的像素信息确定搅拌车接料斗的料位状态，监控精度更高。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控方法，所述向所述搅拌站卸料口对应的搅拌站控制系统发送第一控制信号，之后还包括：
获取所述搅拌站卸料区域的监控视频帧，得到第二视频帧，并对所述第二视频帧中的搅拌车接料斗区域进行混凝土形态识别；根据混凝土形态识别结果，生成第三控制信号，所述第三控制信号用于指示调整卸料速度。
具体地，本申请实施例中，在向搅拌站卸料口对应的搅拌站控制系统发送第一控制信号之后，可以通过获取搅拌站卸料区域的监控视频帧，可以得到第二视频帧。然后对第二视频帧中的搅拌车接料斗区域进行识别，以确定混凝土的形态。
本申请实施例中，可以使用Mask RCNN骨干网络模型所提取的卸料区域的混凝土特征增加分类模块进行形态识别。修改后新增的分类模块网络权重基于携带有混凝土形态标签训练得到。将当前第二视频帧中的搅拌车接料斗区域图像输入至Mask RCNN修改后新增的分类神经网络模块，输出混凝土形态，即当前混凝土是干还是稀。基于混凝土形态识别的结果，可以生成第三控制信号，将第三控制信号发送至搅拌站卸料口对应的搅拌站控制系统，第三控制信号包含有调整卸料速度的指示。例如，当识别出的混凝土形态为过干时，第三控制信号可以用于指示调慢卸料速度。
本申请实施例中，当向搅拌站控制系统发送第一控制信号后，会接收到搅拌站控制系统响应于第一控制信号返回的反馈信号，由于第一控制信号是用于指示开始卸料，在收到第一控制信号的反馈信号时，说明搅拌车接料斗和搅拌站卸料口已经对齐且搅拌车也处于停稳状态，可以开始卸料。
本申请实施例提供的搅拌站智能卸料监控方法，通过对混凝土形态进行识别，避免了混凝土过干或过稀，保证了混凝土质量。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控方法，所述向所述搅拌站卸料口对应的搅拌站控制系统发送第一控制信号，之后还包括：
获取所述搅拌站卸料区域的监控视频帧，得到第二视频帧；
基于所述第二视频帧，确定所述搅拌车接料斗的料位状态，并对所述第二视频帧中的搅拌车接料斗区域进行混凝土形态识别；
根据所述料位状态和混凝土形态识别结果，生成目标控制信号，所述目标控制信号用于指示调整卸料速度或停止卸料。
具体地，本申请实施例中，在向搅拌站卸料口对应的搅拌站控制系统发送第一控制信号后，还可以先获取搅拌站卸料区域的监控视频帧得到第二视频帧，然后通过对第二视频帧进行识别，可以确定出搅拌车接料斗的料位状态以及搅拌车接料斗区域的混凝土形态识别结果，最后根据识别得到的料位状态和混凝土形态识别结果，生成目标控制信号，以指示调整卸料速度或停止卸料。
本申请实施例中，通过对第二视频帧的识别，可以同步确定出搅拌车接料斗的料位状态以及搅拌车接料斗区域的混凝土形态，既可以因溢料造成的浪费，也可以保证混凝土质量。在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控方法，所述基于所述第一视频帧，确定所述搅拌车接料斗的关键点的位置信息，具体包括：
将所述第一视频帧输入至预设的搅拌车接料斗识别算法模型，得到所述搅拌车接料斗识别算法模型输出的所述第一视频帧中所述搅拌车接料斗的关键点的位置信息；
其中，所述搅拌车接料斗识别算法模型中的权重参数基于携带有搅拌车接料斗标签的图像样本训练得到。
具体地，在获取第一视频帧后，可以通过预设的搅拌车接料斗识别算法模型对第一视频帧进行识别，获取搅拌车接料斗的关键点的位置信息。本申请实施例中，预设的搅拌车接料斗识别算法模型可以是加入关键点检测头部模块和目标检测框头部模块的区域生成网络(Region Proposal Network，RPN)模型。
本申请实施例中确定的关键点的位置信息可以是像素坐标系下的坐标，像素坐标系是为了描述物体成像后的像点在数字图像上的坐标引入，是能够真正从监控摄像头内读取到的信息所在的坐标系。由于搅拌站卸料口和搅拌车接料斗可以在世界坐标系下比较是否对齐，因此当搅拌站卸料口和搅拌车接料斗在世界坐标系下比较是否对齐时，可以将确定的关键点的位置信息转化到世界坐标系下，以使比较动作顺利进行。本申请实施例的一种实施方式中，可以基于对监控摄像头的内参矩阵以及外参矩阵，或 者单应性矩阵的逆矩阵进行逆透视变换，将像素坐标系下关键点的位置信息变换到世界坐标系下。逆透视变换是透视变换的逆变换，透视变换是指将图像投影到一个新的视平面。单应性矩阵是指单应性对应的变换矩阵，单应性是描述物体在世界坐标系和像素坐标系之间的位置映射关系。单应性矩阵可以通过监控摄像头的内参矩阵和外参矩阵计算。监控摄像头的外参矩阵可以包括旋转矩阵和平移矩阵。监控摄像头的内参矩阵和外参矩阵可以通过对监控摄像头进行标定得到。
本申请实施例提供的搅拌站智能卸料监控方法，通过计算搅拌车接料斗的关键点坐标的位置信息，便于后续在同一坐标系下比较搅拌车接料斗和搅拌站卸料口是否对齐，提高了判断的准确性。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控方法，所述基于所述搅拌车接料斗的关键点的位置信息以及预设的所述搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐，具体包括：
基于所述搅拌车接料斗的关键点的位置信息，确定所述搅拌车接料斗的中心点的位置信息；
基于所述搅拌车接料斗的中心点的位置信息以及预设的所述搅拌站卸料口的中轴线的位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐。
具体地，本申请实施例中，搅拌车接料斗上可以有至少三个关键点，根据三点求圆心的方法，可以利用已知的至少三个关键点的位置信息确定搅拌车接料斗的中心点坐标，该中心点坐标可以是世界坐标系下的坐标。
本申请实施例中，由于搅拌站卸料口在搅拌站的位置是固定的，搅拌站卸料口是截面为圆形的桶状结构，因此可以通过在搅拌站智能卸料监控系统软件中设置搅拌站卸料口的中轴线的位置信息。最后，比较搅拌车接料斗的中心点坐标以及所述搅拌站卸料口的中轴线的位置信息之间的垂直最短距离，若所述垂直最短距离在预设范围内，则可以确定搅拌车接料斗与搅拌站卸料口已经对齐。其中，预设范围可以根据实际的需要进行设置，本申请实施例对此不作具体限定。
本申请实施例提供的搅拌站智能卸料监控方法，通过计算搅拌车接料 斗中心点的坐标和搅拌站卸料口的中轴线的位置信息之间的垂直最短距离，以此来判断搅拌车接料斗和搅拌站卸料口之间是否已经对齐，从而使搅拌车接料斗和搅拌站卸料口之间能够更准确的对齐，减少在卸料过程中的混凝土浪费。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控方法，所述第一视频帧通过监控摄像头拍摄获得；
相应的，所述基于所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐，之前还包括：
基于所述监控摄像头的预设内参矩阵以及预设畸变系数，对所述搅拌车接料斗的关键点的位置信息进行畸变矫正，得到畸变矫正后的所述搅拌车接料斗的关键点的位置信息；
所述基于所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐，具体包括：
基于畸变矫正后的所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐。
具体地，本申请实施例中，第一视频帧可以通过监控摄像头拍摄得到，监控摄像头可以是搅拌站原有的摄像头硬件，具有能够获取搅拌站卸料区域的监控画面的功能。该监控摄像头的安装位置可以在搅拌站卸料口对面的墙上，安装高度要达到能够监控到完整搅拌车接料斗的高度。也可以在搅拌车的倒车路线与搅拌站卸料口的延长线与搅拌站墙壁的交点处以及过搅拌站卸料口圆心垂直于搅拌车的倒车路线的直线与搅拌站墙壁的交点处，分别安装一个监控摄像头，通过这两个监控摄像头协同对卸料过程进行监控。如图3所示，为本申请实施例中两个监控摄像头的安装位置示意图。图3中监控摄像头31和32均安装在搅拌站墙壁33上，且均朝向搅拌站卸料口34，搅拌站卸料口34处于搅拌车35的倒车路线上。图3中箭头方向为搅拌车35的倒车方向。
本申请实施例中，为了能够使获取到的搅拌车接料斗的关键点的位置 信息更准确，在计算所述搅拌车接料斗的关键点的位置信息之前，还需要畸变矫正。畸变矫正通过监控摄像头的预设内参矩阵以及预设畸变系数实现，监控摄像头的预设内参矩阵是指只与监控摄像头有关，与外界环境无关的参数构成的矩阵。监控摄像头的预设内参矩阵可以是一个3*3的矩阵。预设内参矩阵可以通过单平面棋盘格的标定方法对监控摄像头进行标定后求出。该标定方法的步骤可以是：打印一张棋盘格，把它贴在一个平面上，作为标定物，通过调整标定物或监控摄像头的方向，为标定物拍摄一些不同方向的照片。从照片中提取棋盘格角点。估算理想无畸变的情况下，五个内参和六个外参的数值，得到监控摄像头的预设内参矩阵和外参矩阵。
本申请实施例中，畸变是指对直线投影的一种偏移。畸变是监控摄像头本身的固有特性，和监控摄像头内参相同，对监控摄像头标定一次即可。在本申请实施例的一种实施方式中，可以对监控摄像头的径向畸变进行校正。径向畸变包括桶形畸变和枕形畸变。径向畸变来自于透镜形状，对于径向畸变，成像仪中心的畸变为0，随着向边缘移动，畸变越来越严重。其中，径向畸变的预设畸变系数可以通过上述相机标定过程计算，从而对位置信息进行校正。先求出监控摄像头内参矩阵和外参矩阵，再应用最小二乘法估算实际存在径向畸变下的畸变系数，最后利用泰勒级数展开式对关键点上的位置信息进行矫正。
在对搅拌车接料斗的关键点的位置信息进行了畸变矫正后，即可得到畸变矫正后的搅拌车接料斗的关键点的位置信息，此时再将搅拌车接料斗的关键点的位置信息与预设的搅拌站卸料口的关键位置信息进行比较，若二者间的距离在预设的范围内，则说明搅拌车接料斗与搅拌站卸料口已经对齐。其中，预设的范围可以根据实际需要进行设置，本申请实施例对此不作具体限定。
本申请实施例提供的搅拌站智能卸料监控方法，通过安装监控摄像头监控卸料过程，对搅拌站的改装简便，硬件及施工成本低。且监控摄像头安装的位置与搅拌站卸料口处距离较远，不易沾染上混凝土，影响监控的效果。再对搅拌车接料斗的关键点的位置信息进行畸变矫正，得出无畸变情况下的关键点位置信息，使得搅拌站卸料口和搅拌车接料斗的对齐更准确。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控方法，所述第一视频帧通过监控摄像头拍摄获得，且所述第一视频帧为所述监控摄像头在拍摄角度变化之后拍摄获得的视频帧；
相应的，所述基于所述第一视频帧，确定所述搅拌车接料斗的关键点的位置信息，之后还包括：
基于所述第一视频帧以及基准视频帧，得到单应性矩阵，所述基准视频帧为所述监控摄像头的拍摄角度发生变化之前拍摄获得的视频帧；
基于所述单应性矩阵，对所述搅拌车接料斗的关键点的位置信息进行映射校正，得到映射校正后的所述搅拌车接料斗的关键点的位置信息；
所述基于所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐，具体包括：
基于映射校正后的所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐。
具体地，由于监控摄像头在不同的角度拍摄视频时，从中提取的视频帧的拍摄角度也会不同，这种角度不同会增加对视频帧进行识别时产生的误差，因此，为了使识别的结果更准确，第一视频帧是监控摄像头在拍摄角度变化后拍摄得到的。
在获取第一视频帧后，即可对第一视频帧中的搅拌车接料斗区域进行识别，确定搅拌车接料斗的关键点的位置信息。但因为拍摄角度影响带来的畸变，所以在获取搅拌车接料斗的关键点的位置信息后，还需要对获取的位置信息进行映射校正。
本申请实施例中，可以通过基准视频帧对搅拌车接料斗的关键点的位置信息进行映射校正。基准视频帧是监控摄像头在拍摄角度发生变化前进行拍摄获得的视频帧。基准视频帧可以通过基准标定板获得。基准标定板可以是一块任意平面的白底上打印黑色正方形的基准标定板，监控摄像头在拍摄角度变化前对基准标定板进行拍摄即可得到基准视频帧。
在得到第一视频帧和基准视频帧后，本申请实施例中，可以基于opencv对基准视频帧进行亚像素级角点检测，用二次多项式逼近周围3*3 领域内的角点反应函数，用线性解法求得亚像素级角点坐标。可以使用cornerSubPix()函数迭代计算基准视频帧图像中基准标定板四个角点的坐标，即基准标定板在像素坐标系下的坐标。根据上述畸变矫正的方法对四个角点在像素坐标系下的坐标进行矫正，得到无畸变的基准标定板四个角点的坐标，进而实现对第一视频帧和基准视频帧的畸变校正。
在完成畸变矫正后，即可计算第一视频帧和基准视频帧的单应性矩阵。平面的单应性被定义为一个平面到另外一个平面的投影映射。根据得到的无畸变的基准标定板四个角点的坐标以及第一视频帧与基准视频帧之间的映射关系，可以求得第一视频帧以及基准视频帧的单应性矩阵。
在获取第一视频帧以及基准视频帧的单应性矩阵后，即可对搅拌车接料斗的关键点位置信息进行映射校正。根据上述已经计算出的监控摄像头的单应性矩阵，对两个单应性矩阵进行连续透视变换后即可对拍摄角度改变后搅拌车接料斗上关键点的位置信息进行映射校正。
在映射校正完成后，即可用映射校正后的搅拌车接料斗的关键点的位置信息和预设的搅拌站卸料口的关键位置信息判断搅拌车接料斗和搅拌站卸料口是否对齐。若映射校正后的搅拌车接料斗的关键点的位置信息和预设的搅拌站卸料口的关键位置信息之间的距离在预设的范围内，则可以判断此时搅拌车接料斗和搅拌站卸料口已经对齐。其中，预设的范围可以根据实际情况进行设置，本申请实施例对此不作具体限定。
然后，根据监控摄像头的单应性矩阵，对两个单应性矩阵进行连续透视变换后即可对拍摄角度改变后搅拌车接料斗上关键点坐标进行二次映射修正。
本申请实施例提供的搅拌站智能卸料监控方法，通过基准视频帧对拍摄角度变换后的第一视频帧中的搅拌车接料斗的关键点位置信息进行映射校正，使得拍摄角度变化后，也能得到无畸变的搅拌车接料斗关键点的位置信息，保证了在同一坐标系中，搅拌车接料斗和搅拌站卸料口能够对齐。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控方法，所述监控摄像头具体为红绿蓝(Red Green Blue，RGB)摄像头或飞行时间(Time of Flight，TOF)深感摄像头。
具体地，RGB摄像头中的RGB分别指红色、绿色和蓝色。RGB摄像头由三根不同的线缆给出了上述三个基本彩色成分。这种类型的摄像头通常是用三个独立的电荷耦合器件图像传感器来获取上述三种彩色信号。RGB摄像头经常被用来做非常精确的彩色图像采集。利用RGB摄像头可以获取清晰的搅拌站卸料口处的监控视频。
本申请实施例中，TOF深感摄像头是一种主动相机，它可以通过用红外光源照射一个区域并观察到达现场和返回现场所花费的时间来执行深度测量。TOF深感摄像头可以捕获每个光脉冲的整个视场，而没有任何移动部件。这样可以快速采集数据。TOF深感摄像头测量距离较远，且不受光照变化和物体纹理影响，其帧率较高，软件复杂度较低。采用TOF深感摄像头可以获得监控图像中每个像素点的深度，增加后续搅拌站卸料口的卸料信息识别的效果，使卸料过程的识别更准确。
本申请实施例提供的搅拌站智能卸料监控方法，通过获取搅拌站卸料区域的监控视频帧，得到第一视频帧，在第一视频帧出现搅拌车接料斗时，对第一视频帧进行处理，计算搅拌车接料斗关键点的坐标，再通过搅拌车接料斗的关键点坐标计算搅拌车接料斗的中心点坐标，并在同一坐标系下计算搅拌车接料斗中心点和搅拌站卸料口的中轴线的位置信息之间的垂直最短距离，判断二者是否对齐。在搅拌车接料斗和搅拌站卸料口对齐的情况下，通过像素级分割对监控视频帧中的搅拌车接料斗区域进行分割，监控卸料过程中搅拌车接料斗的料位状态；通过对分割出的混凝土状态进行识别，获取当前监控视频帧中的混凝土状态。该方法不需要人工判断搅拌车接料斗和搅拌站卸料口是否对齐，不易受人为影响，且能够适用于各种高度和接料斗大小的搅拌车。避免了卸料过程中因对不齐造成的混凝土浪费，且能够监控卸料过程，避免了因溢料造成的损失。
图4是本申请实施例提供的搅拌站智能卸料监控方法的具体流程示意图。如图4所示，该方法包括：
S41，获取搅拌站卸料区域的监控视频帧，得到第一视频帧。
S42，在识别到第一视频帧中出现了搅拌车接料斗的情况下，计算搅拌车接料斗关键点位置信息。
S43，对搅拌车接料斗的关键点位置信息进行畸变修正，得到无畸变 的搅拌车接料斗的关键点位置信息。
S44，基于无畸变的搅拌车接料斗的关键点位置信息，计算搅拌车接料斗中心点的位置信息。
S45，判断搅拌站卸料口与搅拌车接料斗是否对齐，也即是判断搅拌车接料斗中心点与搅拌站卸料口的关键位置信息在同一坐标系下的位置信息的距离是否在预设范围内。若判断获知对齐，则继续执行S46，否则返回继续执行S41。
S46，判断搅拌车是否停稳，若停稳，则继续执行S47，否则返回继续执行S46。
S47，向所述搅拌站卸料口对应的搅拌站控制系统发送第一控制信号。
S48，基于监控视频帧中的搅拌车接料斗区域，对搅拌车接料斗的料位状态进行监控，并根据监控的结果，生成第二控制信号。
S49，基于监控视频帧中的搅拌车接料斗区域，对搅拌车接料斗区域进行混凝土形态识别，并根据识别的结果，生成第三控制信号。
图5是本申请实施例提供的搅拌站智能卸料监控系统结构示意图，如图5所示，该系统包括：监控视频获取模块501和卸料监控模块502。其中：
监控视频获取模块501，用于获取搅拌站卸料区域的监控视频帧，得到第一视频帧；
卸料监控模块502，用于基于预设的算法模型，对所述第一视频帧进行检测识别，并基于识别结果，监控混凝土搅拌站的卸料。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控系统，所述卸料监控模块具体包括：
位置信息获取子模块，用于若所述识别结果为所述第一视频帧中包含搅拌车接料斗，则基于所述第一视频帧，确定所述搅拌车接料斗的关键点的位置信息；
对齐子模块，用于基于所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐；
第一控制信号生成子模块，用于若确定所述搅拌车接料斗与所述搅拌 站卸料口对齐，则向所述搅拌站卸料口对应的搅拌站控制系统发送第一控制信号，所述第一控制信号用于指示开始卸料。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控系统，所述第一控制信号生成子模块具体包括：
第一控制信号发送单元，用于若确定所述搅拌车接料斗与所述搅拌站卸料口对齐，且所述搅拌车接料斗对应的搅拌车处于停稳状态，则向所述搅拌站卸料口对应的搅拌站控制系统发送所述第一控制信号。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控系统，所述卸料监控模块还包括：
料位状态确定子模块，用于获取所述搅拌站卸料区域的监控视频帧，得到第二视频帧，基于所述第二视频帧，确定所述搅拌车接料斗的料位状态；
第二控制信号生成子模块，用于根据所述料位状态，生成第二控制信号，并向所述搅拌站控制系统发送所述第二控制信号，所述第二控制信号用于指示调整卸料速度或停止卸料。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控系统，所述料位状态确定子模块具体包括：
料位状态确定单元，用于基于所述第二视频帧中的搅拌车接料斗区域图像的像素信息，确定所述搅拌车接料斗的料位状态。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控系统，所述料位状态确定子模块具体还包括：
分割单元，用于对所述第二视频帧中的搅拌车接料斗区域图像进行像素级分割，并基于像素级分割的结果，确定所述搅拌车接料斗区域内混凝土外沿边界以及搅拌车接料斗外沿边界；
外沿边界确定单元，用于基于所述混凝土外沿边界与所述搅拌车接料斗外沿边界，确定所述搅拌车接料斗的料位状态。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控系统，所述卸料监控模块还包括：
混凝土形态识别子模块，用于获取所述搅拌站卸料区域的监控视频帧，得到第二视频帧，并对所述第二视频帧中的搅拌车接料斗区域进行混凝土 形态识别；
第三控制信号生成子模块，用于根据混凝土形态识别结果，生成第三控制信号，所述第三控制信号用于指示调整卸料速度。
在上述实施例的基础上，本申请实施例中提供的搅拌站智能卸料监控系统，所述位置信息获取子模块具体包括：
关键点位置信息获取单元，用于将所述第一视频帧输入至预设的搅拌车接料斗识别算法模型，得到所述搅拌车接料斗识别算法模型输出的所述第一视频帧中所述搅拌车接料斗的关键点的位置信息；
其中，所述搅拌车接料斗识别算法模型中的权重参数基于携带有搅拌车接料斗标签的图像样本训练得到。
在上述实施例的基础上，本申请实施例中提供的搅拌站智能卸料监控系统，所述对齐子模块具体包括：
中心点位置信息获取单元，用于基于所述搅拌车接料斗的关键点的位置信息，确定所述搅拌车接料斗的中心点的位置信息；
对齐单元，用于基于所述搅拌车接料斗的中心点的位置信息以及预设的所述搅拌站卸料口的中轴线的位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控系统，所述第一视频帧通过监控摄像头拍摄获得；相应地，所述系统还包括：
畸变矫正子模块，用于基于所述监控摄像头的预设内参矩阵以及预设畸变系数，对所述搅拌车接料斗的关键点的位置信息进行畸变矫正，得到畸变矫正后的所述搅拌车接料斗的关键点的位置信息；
畸变矫正对齐子模块，用于基于畸变矫正后的所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控系统，所述第一视频帧通过监控摄像头拍摄获得，且所述第一视频帧为所述监控摄像头在拍摄角度变化之后拍摄获得的视频帧；所述系统还包括：
单应性矩阵获取子模块，用于基于所述第一视频帧以及基准视频帧，得到单应性矩阵，所述基准视频帧为所述监控摄像头的拍摄角度发生变化 之前拍摄获得的视频帧；
映射校正子模块，用于基于所述单应性矩阵，对所述搅拌车接料斗的关键点的位置信息进行映射校正，得到映射校正后的所述搅拌车接料斗的关键点的位置信息；
映射校正对齐子模块，用于基于映射校正后的所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐。
在上述实施例的基础上，本申请实施例提供的搅拌站智能卸料监控系统，所述监控摄像头具体为RGB摄像头或TOF深感摄像头。
具体地，本申请实施例提供的搅拌站智能卸料监控系统中各模块的作用与上述方法类实施例中各步骤的操作流程是一一对应的，实现的效果也是一致的，具体参见上述实施例，本申请实施例中对此不再赘述。
图6是本申请实施例提供的搅拌站智能卸料监控系统的具体结构示意图。如图6所示，该系统包括：监控视频获取模块、倒车对齐监测服务模块以及溢料监测服务模块。其中，监控视频获取模块，用于获取搅拌站卸料口处的监控视频。除此之外，监控视频获取模块还用于视频拉流解析以及将解析后的视频帧进行去噪、增强等预处理。倒车对齐监测服务模块用于监测搅拌车接料斗和搅拌站卸料口是否对齐。溢料监测服务模块用于对卸料过程进行监控。
倒车对齐监测服务模块具体包括：
位置信息获取模块，用于对预处理后的第一视频帧进行识别，获取搅拌车接料斗的关键点的位置信息；
摄像头标定模块，用于对监控摄像头进行标定，将获取的搅拌车接料斗的关键点的位置信息进行畸变矫正；
世界坐标系映射模块，用于将畸变矫正后的搅拌车接料斗的关键点的位置信息，即搅拌车接料斗的关键点在像素坐标系下的坐标映射到世界坐标系；
倒车监控模块，即对齐子模块，用于判断搅拌车接料斗和搅拌站卸料口是否对齐；在对齐的情况下，进行溢料监测服务；在未对齐的情况下，重新获取第一视频帧进行识别；
映射校正模块，用于监控摄像头在拍摄得到当前视频帧之后拍摄角度发生变化时，对拍摄角度变化后所述监控摄像头拍摄得到的第一视频帧中所述搅拌车接料斗上关键点的位置信息进行映射校正；
图像后处理模块，用于对已倒车对齐的视频帧进行增强等图像后处理。
溢料监测服务模块具体包括：
图像前处理模块，用于对增强后的已倒车对齐的视频帧进行剪裁、去噪等图像前处理；
分割模型模块，即分割子模块，用于对搅拌车接料斗区域进行像素级分割；
像素边缘监测模块，用于确定所述搅拌车接料斗区域内混凝土外沿边界以及搅拌车接料斗外沿边界；
混凝土状态识别模块，即混凝土形态识别子模块，用于对所述当前视频帧中的搅拌车接料斗区域进行混凝土形态识别；
流速监测模块，用于对混凝土的流量信息进行监测；
溢料识别模块，用于判断当前视频帧中混凝土是否发生了溢料。
图7示例了一种电子设备的实体结构示意图，如图8所示，该电子设备可以包括：处理器(processor)710、通信接口(Communications Interface)720、存储器(memory)730和通信总线740，其中，处理器710，通信接口720，存储器730通过通信总线740完成相互间的通信。处理器710可以调用存储器730中的逻辑指令，以执行上述各实施例中提供的搅拌站智能卸料监控方法，该方法包括：获取搅拌站卸料区域的监控视频帧，得到第一视频帧；基于预设的算法模型，对所述第一视频帧进行检测识别，并基于识别结果，监控混凝土搅拌站的卸料。
此外，上述的存储器730中的逻辑指令可以通过软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动 硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
另一方面，本申请还提供一种计算机程序产品，所述计算机程序产品包括存储在非暂态计算机可读存储介质上的计算机程序，所述计算机程序包括程序指令，当所述程序指令被计算机执行时，计算机实现能够执行上述各实施例中提供的搅拌站智能卸料监控方法，该方法包括：获取搅拌站卸料区域的监控视频帧，得到第一视频帧；基于预设的算法模型，对所述第一视频帧进行检测识别，并基于识别结果，监控混凝土搅拌站的卸料。
又一方面，本申请还提供一种非暂态计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现能够执行上述各实施例中提供的搅拌站智能卸料监控方法，该方法包括：获取搅拌站卸料区域的监控视频帧，得到第一视频帧；基于预设的算法模型，对所述第一视频帧进行检测识别，并基于识别结果，监控混凝土搅拌站的卸料。
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下，即可以理解并实施。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，上述技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在计算机可读存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。
最后应说明的是：以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。

Claims (14)

  1. 一种搅拌站智能卸料监控方法，包括：
    获取搅拌站卸料区域的监控视频帧，得到第一视频帧；
    基于预设的算法模型，对所述第一视频帧进行检测识别，并基于识别结果，监控混凝土搅拌站的卸料。
  2. 根据权利要求1所述的搅拌站智能卸料监控方法，其中所述基于识别结果，监控混凝土搅拌站的卸料，具体包括：
    若所述识别结果为所述第一视频帧中包含搅拌车接料斗，则基于所述第一视频帧，确定所述搅拌车接料斗的关键点的位置信息；
    基于所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐；
    若确定所述搅拌车接料斗与所述搅拌站卸料口对齐，则向所述搅拌站卸料口对应的搅拌站控制系统发送第一控制信号，所述第一控制信号用于指示开始卸料。
  3. 根据权利要求2所述的搅拌站智能卸料监控方法，其中所述若确定所述搅拌车接料斗与所述搅拌站卸料口对齐，则向所述搅拌站卸料口对应的搅拌站控制系统发送第一控制信号，具体包括：
    若确定所述搅拌车接料斗与所述搅拌站卸料口对齐，且所述搅拌车接料斗对应的搅拌车处于停稳状态，则向所述搅拌站卸料口对应的搅拌站控制系统发送所述第一控制信号。
  4. 根据权利要求2所述的搅拌站智能卸料监控方法，其中所述向所述搅拌站卸料口对应的搅拌站控制系统发送第一控制信号，之后还包括：
    获取所述搅拌站卸料区域的监控视频帧，得到第二视频帧，基于所述第二视频帧，确定所述搅拌车接料斗的料位状态；
    根据所述料位状态，生成第二控制信号，并向所述搅拌站控制系统发送所述第二控制信号，所述第二控制信号用于指示调整卸料速度或停止卸料。
  5. 根据权利要求4所述的搅拌站智能卸料监控方法，其中所述基于所述第二视频帧，确定所述搅拌车接料斗的料位状态，具体包括：
    基于所述第二视频帧中的搅拌车接料斗区域图像的像素信息，确定所 述搅拌车接料斗的料位状态。
  6. 根据权利要求4所述的搅拌站智能卸料监控方法，其中所述基于所述第二视频帧，确定所述搅拌车接料斗的料位状态，具体包括：
    对所述第二视频帧中的搅拌车接料斗区域图像进行像素级分割，并基于像素级分割的结果，确定所述搅拌车接料斗区域内混凝土外沿边界以及搅拌车接料斗外沿边界；
    基于所述混凝土外沿边界与所述搅拌车接料斗外沿边界，确定所述搅拌车接料斗的料位状态。
  7. 根据权利要求2所述的搅拌站智能卸料监控方法，其中所述向所述搅拌站卸料口对应的搅拌站控制系统发送第一控制信号，之后还包括：
    获取所述搅拌站卸料区域的监控视频帧，得到第二视频帧，并对所述第二视频帧中的搅拌车接料斗区域进行混凝土形态识别；
    根据混凝土形态识别结果，生成第三控制信号，所述第三控制信号用于指示调整卸料速度。
  8. 根据权利要求2所述的搅拌站智能卸料监控方法，其中所述基于所述第一视频帧，确定所述搅拌车接料斗的关键点的位置信息，具体包括：
    将所述第一视频帧输入至预设的搅拌车接料斗识别算法模型，得到所述搅拌车接料斗识别算法模型输出的所述第一视频帧中所述搅拌车接料斗的关键点的位置信息；
    其中，所述搅拌车接料斗识别算法模型中的权重参数基于携带有搅拌车接料斗标签的图像样本训练得到。
  9. 根据权利要求2所述的搅拌站智能卸料监控方法，其中所述基于所述搅拌车接料斗的关键点的位置信息以及预设的所述搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐，具体包括：
    基于所述搅拌车接料斗的关键点的位置信息，确定所述搅拌车接料斗的中心点的位置信息；
    基于所述搅拌车接料斗的中心点的位置信息以及预设的所述搅拌站卸料口的中轴线的位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐。
  10. 根据权利要求2所述的搅拌站智能卸料监控方法，其中所述第一视频帧通过监控摄像头拍摄获得；
    相应的，所述基于所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐，之前还包括：
    基于所述监控摄像头的预设内参矩阵以及预设畸变系数，对所述搅拌车接料斗的关键点的位置信息进行畸变矫正，得到畸变矫正后的所述搅拌车接料斗的关键点的位置信息；
    所述基于所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐，具体包括：
    基于畸变矫正后的所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐。
  11. 根据权利要求2所述的搅拌站智能卸料监控方法，其中所述第一视频帧通过监控摄像头拍摄获得，且所述第一视频帧为所述监控摄像头在拍摄角度变化之后拍摄获得的视频帧；
    相应的，所述基于所述第一视频帧，确定所述搅拌车接料斗的关键点的位置信息，之后还包括：
    基于所述第一视频帧以及基准视频帧，得到单应性矩阵，所述基准视频帧为所述监控摄像头的拍摄角度发生变化之前拍摄获得的视频帧；
    基于所述单应性矩阵，对所述搅拌车接料斗的关键点的位置信息进行映射校正，得到映射校正后的所述搅拌车接料斗的关键点的位置信息；
    所述基于所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐，具体包括：
    基于映射校正后的所述搅拌车接料斗的关键点的位置信息以及预设的搅拌站卸料口的关键位置信息，确定所述搅拌车接料斗与所述搅拌站卸料口是否对齐。
  12. 一种搅拌站智能卸料监控系统，包括：
    监控视频获取模块，用于获取搅拌站卸料区域的监控视频帧，得到第一视频帧；
    卸料监控模块，用于基于预设的算法模型，对所述第一视频帧进行检测识别，并基于识别结果，监控混凝土搅拌站的卸料。
  13. 一种电子设备，包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述程序时实现如权利要求1至11任一项所述搅拌站智能卸料监控方法的步骤。
  14. 一种非暂态计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至11任一项所述搅拌站智能卸料监控方法的步骤。
PCT/CN2022/074291 2021-03-19 2022-01-27 搅拌站智能卸料监控方法及系统 WO2022193848A1 (zh)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202110296818 2021-03-19
CN202110296818.2 2021-03-19
CN202110751261.7A CN113580380B (zh) 2021-03-19 2021-07-02 搅拌站智能卸料监控方法及系统
CN202110751261.7 2021-07-02

Publications (1)

Publication Number Publication Date
WO2022193848A1 true WO2022193848A1 (zh) 2022-09-22

Family

ID=78245750

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/074291 WO2022193848A1 (zh) 2021-03-19 2022-01-27 搅拌站智能卸料监控方法及系统

Country Status (2)

Country Link
CN (1) CN113580380B (zh)
WO (1) WO2022193848A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116442393A (zh) * 2023-06-08 2023-07-18 山东博硕自动化技术有限公司 基于视频识别的搅拌站智能卸料方法、系统及控制设备
CN116694129A (zh) * 2023-08-07 2023-09-05 济宁九德半导体科技有限公司 用于紫外固化型油墨制备的自动控制系统及其方法
CN116934847A (zh) * 2023-09-15 2023-10-24 蓝思系统集成有限公司 卸料方法、装置、电子设备及存储介质

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113580380B (zh) * 2021-03-19 2022-10-04 三一汽车制造有限公司 搅拌站智能卸料监控方法及系统
CN114227934B (zh) * 2021-11-25 2023-05-26 三一专用汽车有限责任公司 协同作业系统、搅拌站及搅拌车
CN114131763A (zh) * 2021-11-30 2022-03-04 三一专用汽车有限责任公司 搅拌车与搅拌站协同作业系统、方法、搅拌站及搅拌车
CN114536563A (zh) * 2022-02-28 2022-05-27 常德市三一机械有限公司 卸料控制方法及搅拌站
CN115139413A (zh) * 2022-05-16 2022-10-04 中联重科股份有限公司 进料控制方法、系统及计算机可读存储介质
CN115592819B (zh) * 2022-12-15 2023-03-17 福建南方路面机械股份有限公司 基于深度学习的搅拌站卸料口尺寸调整方法及装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080028988A1 (en) * 2006-02-21 2008-02-07 Welker Charles D Method and apparatus for making air-entrained or cellular high-strengthconcrete
CN103072207A (zh) * 2013-01-31 2013-05-01 中联重科股份有限公司 搅拌车的进料位置调整控制方法、装置、系统及搅拌车
CN105507590A (zh) * 2015-11-30 2016-04-20 北汽福田汽车股份有限公司 一种卸料控制方法、装置以及工程机械
CN112487898A (zh) * 2020-11-18 2021-03-12 中科云谷科技有限公司 搅拌站搅拌车进、卸口对齐自动判定方法、设备及系统
CN113580380A (zh) * 2021-03-19 2021-11-02 三一汽车制造有限公司 搅拌站智能卸料监控方法及系统

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE3006690C2 (de) * 1980-02-22 1982-09-09 Elba Maschinenfabrik Kaiser GmbH & Co, 6670 St Ingbert Verfahren zur definierten und reproduzierbaren Abgabe von Teilmengen des Mischgutes eines Transportmischers
CN203665714U (zh) * 2013-12-13 2014-06-25 徐州徐工施维英机械有限公司 一种搅拌站卸料防溢料装置及其混凝土搅拌站
CN105058590A (zh) * 2015-08-03 2015-11-18 长沙奥科电子科技有限公司 一种搅拌站监控的方法、装置及系统
CN105217198B (zh) * 2015-10-13 2017-08-25 长沙中联重科环境产业有限公司 一种垃圾上料控制方法、装置及上料板机
CN106696090A (zh) * 2016-12-16 2017-05-24 三汽车制造有限公司 卸料控制系统和搅拌设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080028988A1 (en) * 2006-02-21 2008-02-07 Welker Charles D Method and apparatus for making air-entrained or cellular high-strengthconcrete
CN103072207A (zh) * 2013-01-31 2013-05-01 中联重科股份有限公司 搅拌车的进料位置调整控制方法、装置、系统及搅拌车
CN105507590A (zh) * 2015-11-30 2016-04-20 北汽福田汽车股份有限公司 一种卸料控制方法、装置以及工程机械
CN112487898A (zh) * 2020-11-18 2021-03-12 中科云谷科技有限公司 搅拌站搅拌车进、卸口对齐自动判定方法、设备及系统
CN113580380A (zh) * 2021-03-19 2021-11-02 三一汽车制造有限公司 搅拌站智能卸料监控方法及系统

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116442393A (zh) * 2023-06-08 2023-07-18 山东博硕自动化技术有限公司 基于视频识别的搅拌站智能卸料方法、系统及控制设备
CN116442393B (zh) * 2023-06-08 2024-02-13 山东博硕自动化技术有限公司 基于视频识别的搅拌站智能卸料方法、系统及控制设备
CN116694129A (zh) * 2023-08-07 2023-09-05 济宁九德半导体科技有限公司 用于紫外固化型油墨制备的自动控制系统及其方法
CN116694129B (zh) * 2023-08-07 2023-10-17 济宁九德半导体科技有限公司 用于紫外固化型油墨制备的自动控制系统及其方法
CN116934847A (zh) * 2023-09-15 2023-10-24 蓝思系统集成有限公司 卸料方法、装置、电子设备及存储介质
CN116934847B (zh) * 2023-09-15 2024-01-05 蓝思系统集成有限公司 卸料方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN113580380B (zh) 2022-10-04
CN113580380A (zh) 2021-11-02

Similar Documents

Publication Publication Date Title
WO2022193848A1 (zh) 搅拌站智能卸料监控方法及系统
CN1584729A (zh) 图像投影方法和设备
KR101941801B1 (ko) Led 디스플레이에 이용되는 영상 처리 방법 및 장치
WO2020237565A1 (zh) 一种目标追踪方法、装置、可移动平台及存储介质
US11880993B2 (en) Image processing device, driving assistance system, image processing method, and program
CN113554002B (zh) 对齐状态检测方法、系统、电子设备及存储介质
CN111243034A (zh) 一种全景辅助泊车标定方法、装置、设备及存储介质
CN115239820A (zh) 一种分体式飞行车辆鸟瞰图实时拼接及车位检测方法
JP2008176768A (ja) 画像処理装置
CN113538431B (zh) 显示屏瑕疵定位方法、装置、终端设备及系统
CN114140521A (zh) 一种识别投影位置的方法、装置、系统和存储介质
TWI383690B (zh) 影像處理的方法
CN106546196B (zh) 一种光轴实时校准方法及系统
WO2018087424A1 (en) Determining an intersection location of an optical axis of a lens with a camera sensor
CN117097872A (zh) 一种投影设备自动梯形校正系统及方法
CN112288677A (zh) 摄像机
CN104933430A (zh) 一种用于移动终端的交互式图像处理方法及系统
CN114638880A (zh) 平面测距方法、单目摄像头及计算机可读存储介质
CN112637575B (zh) 一种超短焦激光投影机的自动图像校正系统及方法
US11659270B2 (en) Image processing device for detecting horizontal direction of photographing angle
CN106413280B (zh) 贴片机喂料器元件位置自动矫正装置及自动矫正方法
CN111174706A (zh) 地板安装检测方法、电子装置及存储介质
CN114332234B (zh) 一种基于棋盘格的自动标定方法及系统
CN117649367B (zh) 一种图像方位的校正方法及系统
CN116017134B (zh) 基于球形摄像机的粮仓粮面害虫图像采集方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22770208

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023107878

Country of ref document: RU

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 523440581

Country of ref document: SA

122 Ep: pct application non-entry in european phase

Ref document number: 22770208

Country of ref document: EP

Kind code of ref document: A1