CN114549988A - Image processing method and device for intelligent transportation, electronic equipment and medium - Google Patents

Image processing method and device for intelligent transportation, electronic equipment and medium Download PDF

Info

Publication number
CN114549988A
CN114549988A CN202210178793.0A CN202210178793A CN114549988A CN 114549988 A CN114549988 A CN 114549988A CN 202210178793 A CN202210178793 A CN 202210178793A CN 114549988 A CN114549988 A CN 114549988A
Authority
CN
China
Prior art keywords
image frame
image
road surface
camera
coordinate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210178793.0A
Other languages
Chinese (zh)
Inventor
冯文茜
张永乐
刘少耿
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Apollo Zhilian Beijing Technology Co Ltd
Original Assignee
Apollo Zhilian Beijing Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Apollo Zhilian Beijing Technology Co Ltd filed Critical Apollo Zhilian Beijing Technology Co Ltd
Priority to CN202210178793.0A priority Critical patent/CN114549988A/en
Publication of CN114549988A publication Critical patent/CN114549988A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/20—Administration of product repair or maintenance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/26—Government or public services

Landscapes

  • Business, Economics & Management (AREA)
  • Engineering & Computer Science (AREA)
  • Human Resources & Organizations (AREA)
  • Strategic Management (AREA)
  • Economics (AREA)
  • Tourism & Hospitality (AREA)
  • Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Marketing (AREA)
  • General Physics & Mathematics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Quality & Reliability (AREA)
  • Operations Research (AREA)
  • Development Economics (AREA)
  • Educational Administration (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Primary Health Care (AREA)
  • Game Theory and Decision Science (AREA)
  • Image Analysis (AREA)

Abstract

The present disclosure provides an image processing method, an image processing apparatus, an electronic device, and a medium for intelligent transportation, which relate to the technical field of image processing, and in particular, to the technical field of intelligent transportation and computer vision. The implementation scheme is as follows: acquiring a first image frame and a second image frame which are continuously shot by a camera aiming at a road surface in the process of moving along the road; determining a real-world distance between the first image frame and the second image frame; performing pavement damage identification on the first image frame to obtain a pavement damage prediction image indicating a pavement damage prediction result in the first image frame; determining a first pixel position corresponding to a first edge position of a second image frame in the road surface disease prediction image according to the real world distance, the calibration parameter of the camera and the corresponding relation; and removing the pavement damage prediction result in the repeated pavement area from the pavement damage prediction image based on the first pixel position to obtain a duplicate removal image.

Description

Image processing method and device for intelligent transportation, electronic equipment and medium
Technical Field
The present disclosure relates to the field of image processing technologies, particularly to the field of intelligent transportation and computer vision technologies, and in particular, to an image processing method and apparatus for intelligent transportation, an electronic device, a computer-readable storage medium, and a computer program product.
Background
Along with the continuous improvement of road infrastructure construction, road traffic is more and more developed, roads are influenced by load capacity, traffic volume and natural factors, road damage of different degrees can occur, the condition of road damage is more and more common, and higher requirements can be provided for road surface disease identification and road maintenance. In the intelligent highway maintenance, the identification mode of the road surface diseases mainly comprises a manual inspection mode and a mode of shooting the road surface area at a fixed distance through a camera and identifying the road surface diseases in the pictures.
The approaches described in this section are not necessarily approaches that have been previously conceived or pursued. Unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Similarly, unless otherwise indicated, the problems mentioned in this section should not be considered as having been acknowledged in any prior art.
Disclosure of Invention
The present disclosure provides an image processing method, apparatus, electronic device, computer-readable storage medium, and computer program product for intelligent transportation.
According to an aspect of the present disclosure, there is provided an image processing method for intelligent transportation, including: acquiring a first image frame and a second image frame which are continuously shot by a camera aiming at a road surface in the process of moving along the road, wherein repeated road surface areas exist in the first image frame and the second image frame, and the bottom edge of the second image frame corresponds to the bottom boundary of the repeated road surface areas in the first image frame; determining a real-world distance between the first image frame and the second image frame, the real-world distance being indicative of a distance between real-world locations at which the camera was located when the first image frame and the second image frame were taken, respectively; performing pavement damage identification on the first image frame to obtain a pavement damage prediction image which indicates a pavement damage prediction result in the first image frame, wherein pixel positions in the pavement damage prediction image have a corresponding relation with pixel positions in the first image frame; determining a first pixel position corresponding to a first edge position of a second image frame in the pavement damage prediction image according to the real world distance, the calibration parameters of the camera and the corresponding relation, wherein the first edge position is any pixel position on the bottom edge of the second image frame; and removing the pavement damage prediction result in the repeated pavement area from the pavement damage prediction image based on the first pixel position to obtain a duplicate removal image.
According to another aspect of the present disclosure, there is provided an image processing apparatus for intelligent transportation, including: a first acquisition unit configured to acquire a first image frame and a second image frame continuously captured by a camera for a road surface during movement along the road, the first image frame and the second image frame having a repeated road surface area therein, a bottom edge of the second image frame corresponding to a bottom boundary of the repeated road surface area in the first image frame; a first determining unit configured to determine a real-world distance between the first image frame and the second image frame, the real-world distance indicating a distance between real-world positions at which the camera is located when the first image frame and the second image frame are captured, respectively; a first identification unit configured to perform pavement damage identification on the first image frame to obtain a pavement damage prediction image indicating a pavement damage prediction result in the first image frame, wherein pixel positions in the pavement damage prediction image have a corresponding relationship with pixel positions in the first image frame; the second determining unit is configured to determine a first pixel position corresponding to a first edge position of the second image frame in the road surface damage prediction image according to the real world distance, the calibration parameter of the camera and the corresponding relation, wherein the first edge position is any one pixel position on the bottom edge of the second image frame; and a removing unit configured to remove the road surface defect prediction result in the repeated road surface area from the road surface defect prediction image based on the first pixel position to obtain a duplicate removal image.
According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and at least one memory communicatively coupled to the at least one processor; wherein the at least one memory stores instructions executable by the at least one processor to enable the at least one processor to perform the image processing method.
According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the above-described image processing method.
According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program realizes the above-described image processing method when executed by a processor.
According to one or more embodiments of the disclosure, repeated parts of the pavement area in the prediction result image can be determined based on the real world distance between two adjacent image frames, and the repeated pavement disease prediction result is removed, so that the efficiency of removing the repeated result is improved, and the cost of manually removing the repeated result is saved.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the embodiments and, together with the description, serve to explain the exemplary implementations of the embodiments. The illustrated embodiments are for purposes of illustration only and do not limit the scope of the claims. Throughout the drawings, identical reference numbers designate similar, but not necessarily identical, elements.
FIG. 1 illustrates a schematic diagram of an exemplary system in which various methods described herein may be implemented, according to an embodiment of the present disclosure;
fig. 2 shows a flowchart of an image processing method for intelligent transportation according to an embodiment of the present disclosure;
fig. 3A and 3B are schematic diagrams respectively showing an example of a first image frame and a second image frame, and fig. 3C is a schematic diagram showing a pavement damage prediction image obtained after pavement damage identification is performed on the first image frame shown in fig. 3A;
fig. 4 shows a flowchart of an image processing method for intelligent transportation according to an embodiment of the present disclosure;
fig. 5 shows a flowchart of an image processing method for intelligent transportation according to an embodiment of the present disclosure;
fig. 6 shows a schematic diagram of the principle of a binocular camera;
fig. 7 illustrates a block diagram of an image processing apparatus for intelligent transportation according to an embodiment of the present disclosure;
fig. 8 is a block diagram showing a configuration of an image processing apparatus for intelligent transportation according to an embodiment of the present disclosure;
fig. 9 shows a block diagram of the structure of an image processing apparatus for intelligent transportation according to an embodiment of the present disclosure;
FIG. 10 illustrates a block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In the present disclosure, unless otherwise specified, the use of the terms "first", "second", and the like to describe various elements is not intended to limit the positional relationship, the temporal relationship, or the importance relationship of the elements, and such terms are used only to distinguish one element from another. In some examples, a first element and a second element may refer to the same instance of the element, and in some cases, based on the context, they may also refer to different instances.
The terminology used in the description of the various described examples in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the elements may be one or more. Furthermore, the term "and/or" as used in this disclosure is intended to encompass any and all possible combinations of the listed items.
At present, in the maintenance of an intelligent highway, a road is usually photographed continuously, pavement diseases are identified by the obtained pictures in a deep learning mode, and then the identification results of the pavement diseases identified in all the pictures are summarized. However, since the adjacent photographs obtained by continuous shooting have repeated road surface areas, the same recognition results obtained by the adjacent photographs have a large number of repeated recognition results, and the repeated recognition results need to be manually de-duplicated, so that the labor cost is greatly increased, and the efficiency of road surface defect recognition is also reduced.
In order to solve the problems, the inventor provides an image processing method for intelligent transportation, and the method can determine repeated parts of a pavement area in a prediction result image based on the moving distance of a camera between two adjacent image frames, so that the repeated pavement disease prediction result is automatically identified and removed, the efficiency of removing the repeated result is improved, and the cost of manually removing the repeated result is saved.
Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.
Fig. 1 illustrates a schematic diagram of an exemplary system 100 in which various methods and apparatus described herein may be implemented in accordance with embodiments of the present disclosure. Referring to fig. 1, the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. Client devices 101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.
In an embodiment of the present disclosure, the server 120 may run one or more services or software applications that enable the image processing method to be performed.
In some embodiments, the server 120 may also provide other services or software applications that may include non-virtual environments and virtual environments. In certain embodiments, these services may be provided as web-based services or cloud services, for example, provided to users of client devices 101, 102, 103, 104, 105, and/or 106 under a software as a service (SaaS) model.
In the configuration shown in fig. 1, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or a combination thereof, which may be executed by one or more processors. A user operating a client device 101, 102, 103, 104, 105, and/or 106 may, in turn, utilize one or more client applications to interact with the server 120 to take advantage of the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from system 100. Accordingly, fig. 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.
A user may use client devices 101, 102, 103, 104, 105, and/or 106 to acquire a first image frame and a second image frame. The client device may provide an interface that enables a user of the client device to interact with the client device. The client device may also output information to the user via the interface. Although fig. 1 depicts only six client devices, those skilled in the art will appreciate that any number of client devices may be supported by the present disclosure.
Client devices 101, 102, 103, 104, 105, and/or 106 may include various types of computer devices, such as portable handheld devices, general purpose computers (such as personal computers and laptops), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, gaming systems, thin clients, various messaging devices, sensors or other sensing devices, and so forth. These computer devices may run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux, or Linux-like operating systems (e.g., GOOGLE Chrome OS); or include various Mobile operating systems such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, Android. Portable handheld devices may include cellular telephones, smart phones, tablets, Personal Digital Assistants (PDAs), and the like. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. The gaming system may include a variety of handheld gaming devices, internet-enabled gaming devices, and the like. The client device is capable of executing a variety of different applications, such as various Internet-related applications, communication applications (e.g., email applications), Short Message Service (SMS) applications, and may use a variety of communication protocols.
Network 110 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including but not limited to TCP/IP, SNA, IPX, etc. By way of example only, one or more networks 110 may be a Local Area Network (LAN), an ethernet-based network, a token ring, a Wide Area Network (WAN), the internet, a virtual network, a Virtual Private Network (VPN), an intranet, an extranet, a Public Switched Telephone Network (PSTN), an infrared network, a wireless network (e.g., bluetooth, WIFI), and/or any combination of these and/or other networks.
The server 120 may include one or more general purpose computers, special purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-end servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and/or combination. The server 120 may include one or more virtual machines running a virtual operating system, or other computing architecture involving virtualization (e.g., one or more flexible pools of logical storage that may be virtualized to maintain virtual storage for the server). In various embodiments, the server 120 may run one or more services or software applications that provide the functionality described below.
The computing units in server 120 may run one or more operating systems including any of the operating systems described above, as well as any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and/or middle tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, and the like.
In some implementations, the server 120 may include one or more applications to analyze and consolidate data feeds and/or event updates received from users of the client devices 101, 102, 103, 104, 105, and/or 106. Server 120 may also include one or more applications to display data feeds and/or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and/or 106.
In some embodiments, the server 120 may be a server of a distributed system, or a server incorporating a blockchain. The server 120 may also be a cloud server, or a smart cloud computing server or a smart cloud host with artificial intelligence technology. The cloud Server is a host product in a cloud computing service system, and is used for solving the defects of high management difficulty and weak service expansibility in the traditional physical host and Virtual Private Server (VPS) service.
The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. The database 130 may reside in various locations. For example, the database used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The database 130 may be of different types. In certain embodiments, the database used by the server 120 may be, for example, a relational database. One or more of these databases may store, update, and retrieve data to and from the databases in response to the commands.
In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the application may be different types of databases, such as key-value stores, object stores, or regular stores supported by a file system.
The system 100 of fig. 1 may be configured and operated in various ways to enable application of the various methods and apparatus described in accordance with the present disclosure.
According to an embodiment of the present disclosure, as shown in fig. 2, there is provided an image processing method 200 for intelligent transportation, including: step S201, acquiring a first image frame and a second image frame which are continuously shot by a camera aiming at a road surface in the process of moving along the road, wherein repeated road surface areas exist in the first image frame and the second image frame, and the bottom edge of the second image frame corresponds to the bottom boundary of the repeated road surface areas in the first image frame; step S202, determining a real world distance between the first image frame and the second image frame, wherein the real world distance indicates a distance between real world positions where the camera respectively locates when the first image frame and the second image frame are shot; step S203, performing pavement damage identification on the first image frame to obtain a pavement damage prediction image which indicates a pavement damage prediction result in the first image frame, wherein pixel positions in the pavement damage prediction image have a corresponding relation with pixel positions in the first image frame; step S204, determining a first pixel position corresponding to a first edge position of a second image frame in the road surface disease prediction image according to the real world distance, the calibration parameters of the camera and the corresponding relation, wherein the first edge position is any pixel position on the bottom edge of the second image frame; and step S205, based on the first pixel position, removing the road surface damage prediction result in the repeated road surface area from the road surface damage prediction image to obtain a duplication-removed image.
Therefore, the repeated parts of the pavement area in the prediction result image are determined based on the moving distance of the camera between the two adjacent image frames, and the repeated pavement disease prediction result is removed, so that the efficiency of removing the repeated result is improved, and the cost of manually removing the repeated result is saved.
In some embodiments, the camera for continuously shooting the road surface may be mounted on a vehicle such as a vehicle or an unmanned aerial vehicle, and the shooting direction of the camera may be a direction facing the camera forward, a direction facing away from the camera forward, a direction perpendicular to the downward direction for taking a downward shot of the road surface, or the like. It can be understood that the carrying tool of the camera and the direction of the camera shooting road surface can be set according to actual needs, and are not limited herein.
The camera can continuously shoot the road surface during moving along the road, so that a group of images with the road surface area are obtained. In two adjacent images in the set of images (i.e., the first image frame and the second image frame), there are repeated road surface regions. Fig. 3A and 3B are schematic diagrams of an example of a first image frame and a second image frame, respectively, in which a part of a road region (i.e., a road surface region 312 above a straight line 311) in the first image frame 310 and a road surface region 323 shown in the second image frame 320 are repeated road surface regions, where the road surface defect 322 and the road surface defect 314 are the same road surface defect on the road surface. The bottom edge of the second image frame 320 corresponds to the bottom boundary of the repeated road surface region in the first image frame 310 (i.e., the position shown by the line 311).
Herein, spatially relative terms such as "bottom", "top", and the like are used herein for convenience of description to describe relative positional relationships between edges of an image/image area as illustrated in the drawings. It will be understood that these spatially relative terms are intended to encompass different orientations of the image in use or operation in addition to the orientation depicted in the figures. For example, if the image in the figure is flipped, what is described as "bottom edge" will be oriented as "top edge". Similarly, the images may be oriented in other ways (e.g., rotated 90 degrees or rotated in other orientations) and the spatially relative terms used herein are interpreted accordingly.
Therefore, it can be understood that fig. 3A and 3B are only one example of the first image frame and the second image frame, and when the shooting direction of the camera changes, the orientations of the first image frame and the second image frame change accordingly, and the explanation of the bottom edge and the bottom boundary is adjusted accordingly.
In some embodiments, the real world distance between the first image frame and the second image frame may be acquired by a Global Navigation Satellite System (GNSS). Specifically, the real-world positions at which the cameras respectively located when the first image frame and the second image frame were taken may be respectively acquired by a global satellite navigation system (e.g., a GPS receiver or a beidou satellite positioning system receiver), and the real-world distance may be calculated based on the two positions.
In some embodiments, the real-world distance between the first image frame and the second image frame may also be obtained by a rotary encoder or the like having similar functions to obtain the number of turns of the wheel of the vehicle on which the camera is located between the first image frame and the second image frame, thereby obtaining the real-world distance moved by the camera.
It is understood that the manner of acquiring the real-world distance between the first image frame and the second image frame may be selected according to actual situations, and is not limited herein.
In some embodiments, performing pavement damage identification on the first image frame may include: and identifying the pavement diseases from the first image frame by using a pavement disease identification model.
Thus, road surface damage recognition is performed on the first image frame by applying the road surface damage recognition model, so that a road surface damage prediction image indicating a prediction result of road surface damage in the first image frame is obtained, and the pixel position of the road surface damage prediction image has a corresponding relationship with the pixel position of the first image frame. The corresponding relationship may be that the pixel position of the road surface damage prediction image corresponds to the pixel position of the first image frame one by one, or that the road surface damage prediction image and the first image frame have an equal scaling relationship.
Fig. 3C shows a road surface damage prediction image 330 obtained after the first image frame 310 is subjected to the road surface damage recognition, and only the road surface damage prediction result is shown in the image, wherein the road surface damage prediction result 331 corresponds to the road surface damage 313 in the first image frame 310, and the road surface damage prediction result 332 corresponds to the road surface damage 314 in the first image frame 310.
In some embodiments, the camera may be a monocular camera, and as shown in fig. 4, the image processing method 400 for intelligent transportation includes steps S401 to S407, wherein determining that the first edge position of the second image frame is at the first pixel position corresponding to the road surface damage prediction image may include: step S404, determining a first coordinate of a second edge position of the pavement damage prediction image in a world coordinate system based on the calibration parameters and the corresponding relation, wherein the second edge position is any pixel position on the bottom edge of the pavement damage prediction image; step S405, determining a second coordinate in a world coordinate system, wherein the second coordinate is apart from the first coordinate by a real world distance along the moving direction of the camera; and step S406, determining a corresponding pixel position of the second coordinate in the road surface disease prediction image as a first pixel position based on the calibration parameter and the corresponding relation. The operations of step S401, step S402, step S403 and step S407 in the image processing method 400 for intelligent transportation are similar to the operations of step S201, step S202, step S203 and step S205 in the image processing method 200 for intelligent transportation, and are not repeated herein.
Before the monocular camera is used for continuously shooting the road surface, when the monocular camera is installed on transportation equipment such as a vehicle, camera calibration needs to be performed on the monocular camera (for example, a Zhang friend camera calibration method can be applied), so that calibration parameters of the monocular camera are obtained, and specifically, an internal parameter matrix and an external parameter matrix of the monocular camera are included. The parameters in the internal parameter matrix comprise the focal length of the monocular camera, the coordinates of an image principal point, the scaling between the pixel and the physical size in the real world and a distortion coefficient, wherein the image principal point is the intersection point of a vertical line between the photographing center and the imaging plane; the extrinsic parameter matrices include a rotation parameter matrix and a translation parameter matrix of the monocular camera.
For a pixel position (u, v) of a point in the image, it is associated with a corresponding coordinate (X) in the world coordinate systemW，YW，ZW) The conversion relationship between them can be expressed by the following formula:
Figure BDA0003521460160000101
Figure BDA0003521460160000102
wherein (X)C，YC，ZC) Representing the corresponding coordinates of the point in the camera coordinate system by aligning the corresponding coordinates (X) in the world coordinate systemW，YW，ZW) Performing rigid body transformation to obtain a rotation parameter matrix of 3 × 3, and a translation parameter matrix of 3 × 3; f denotes the focal length of the camera, (u)0，v0) Representing principal point coordinates, gamma representing a distortion coefficient,
Figure BDA0003521460160000103
representing the scaling between the pixels in the image and the physical dimensions in the real world.
In one example, as shown in fig. 3A, 3B, and 3C, the pixel locations of the road surface damage prediction image 330 correspond to the pixel locations of the first image frame 310. Firstly, acquiring a first coordinate of an edge position 333 (namely a second edge position in the method) in the road surface disease prediction image 330 in a world coordinate system through the formula; acquiring a second coordinate that is a distance from the first coordinate by a real world distance along a camera movement direction based on the acquired real world distance moved by the camera between the first image frame 310 and the second image frame 320; subsequently, the second coordinates are converted into coordinates of pixel positions in the road surface disturbance prediction image 330 by the above formula, thereby obtaining a first pixel position 334 in the road surface disturbance prediction image 330 corresponding to the first edge position 321 of the second image frame 320.
In some embodiments, when the correspondence between the road surface damage prediction image and the first image frame is an equal scaling relationship between the road surface damage prediction image and the first image frame, coordinate information of a second edge position in the road surface damage prediction image corresponding to a pixel position in the first image frame may be first obtained through the correspondence, and a corresponding first coordinate thereof may be obtained based on the above formula; and then, acquiring a second coordinate through the method, and converting the second coordinate into a coordinate of a pixel position in the pavement disease prediction image through the formula and the corresponding relation, thereby acquiring the first pixel position.
In some embodiments, the cameras may be binocular cameras having a first camera and a second camera, wherein the first camera may be a left or right camera of the binocular cameras and the second camera is the other of the binocular cameras outside of the first camera.
The first image frame and the second image frame may be captured by a first camera of a binocular camera, and as shown in fig. 5, the image processing method 500 for intelligent transportation includes steps S501 to S508, wherein determining the first edge position of the second image frame at a first pixel position corresponding to the road surface damage prediction image may include: step S504, determining a scene depth corresponding to a second edge position of the first image frame based on the first image frame, a reference image frame corresponding to the first image frame and shot by a second camera in the binocular camera, and calibration parameters, wherein the second edge position is any pixel position on the bottom edge of the first image frame; step S505, determining a third coordinate of a third edge position, corresponding to the second edge position, of the road surface disease prediction image in a world coordinate system based on the scene depth, the calibration parameters and the corresponding relation; step S506, determining a fourth coordinate in a world coordinate system, wherein the fourth coordinate is apart from the third coordinate by a real world distance along the moving direction of the camera; and step S507, determining a pixel position corresponding to the fourth coordinate in the road surface disease prediction image as a first pixel position based on the calibration parameter and the corresponding relation.
The operations of step S501, step S502, step S503 and step S508 in the image processing method 500 for intelligent transportation are similar to the operations of step S201, step S202, step S203 and step S205 in the image processing method 200 for intelligent transportation, and are not repeated herein.
FIG. 6 shows a schematic diagram of the principle of a binocular camera, in which OLAnd ORThe projection centers of the first camera and the second camera are respectively, the connecting line between the projection centers of the two cameras is a base line 601, and the length of the base line is b; the focal lengths of both cameras are f. For a certain point P in the first image frame 602LThe length Z of the scene depth 604 of the point in the world coordinate system may be obtained by first combining the reference image frame 603 taken by the second camera of the binocular cameras corresponding to the first image frame 602 and the calibration parameters by the following formula:
Figure BDA0003521460160000121
wherein the scene depth 604 represents a certain point P in the first image frame 602LThe distance between the corresponding point P in the world coordinate system and the binocular camera; the imaging point of the point P in the reference image frame is PR；XL、XRAre respectively a point PLPoint PRA distance from the left imaging plane in the first image frame and the reference image frame.
On the basis of the scene depth, the coordinates (X, Y, Z) of the point P in the world coordinate system corresponding to the point PL may be further obtained by the following formula:
Figure BDA0003521460160000122
Figure BDA0003521460160000123
wherein (x, y) represents a point PLCoordinates in the image coordinate system, (x)0，y0) Representing the origin coordinates of the image coordinate system.
Point PLThe conversion relationship of the coordinates (u, v) in the pixel coordinate system and the coordinates (x, y) in the image coordinate system is shown as follows:
Figure BDA0003521460160000124
wherein (u)0，v0) The representation is like the coordinates of the main point,
Figure BDA0003521460160000125
representing the scaling between the pixels in the image and the physical dimensions in the real world.
In one example, as shown in fig. 3A, 3B, and 3C, the pixel locations of the road surface damage prediction image 330 correspond to the pixel locations of the first image frame 310. Firstly, the scene depth of the edge position 315 (i.e. the second edge position in the above method) in the first image frame 310 in the world coordinate system is obtained through the above formula; further, a third coordinate of the edge position 333 (i.e. a third edge position in the above method) in the road surface damage prediction image 330 corresponding to the edge position 315 in the world coordinate system is obtained; acquiring a fourth coordinate that is separated from the third coordinate by the real world distance along the camera movement direction based on the acquired real world distance moved by the camera between the first image frame 310 and the second image frame 320; subsequently, the fourth coordinates are converted into coordinates of pixel positions in the road surface damage prediction image 330 by the above formula, thereby obtaining the first pixel position 334.
In some embodiments, when the correspondence between the road surface defect prediction image and the first image frame is an equal scaling relationship between the road surface defect prediction image and the first image frame, the pixel coordinates of a third edge position in the road surface defect prediction image corresponding to the second edge position in the first image frame may be first obtained through the correspondence, and the corresponding third coordinates thereof may be obtained based on the above formula; and then, acquiring a fourth coordinate through the method, and converting the fourth coordinate into a coordinate of a pixel position in the pavement damage prediction image through the formula and the corresponding relation, thereby acquiring the first pixel position.
The calibration parameters of the binocular camera can be obtained by calibration before the camera leaves a factory, and the binocular camera can be directly used when being obtained without calibration again, so that the operation is simpler. The binocular camera can also be obtained through self-assembly according to needs, and the binocular camera needs to be calibrated after the assembly is completed, so that calibration parameters of the binocular camera are obtained. The binocular camera can acquire scene depth information of a certain point in the image by utilizing the left image and the right image, so that the position of each point in the image in the real world is acquired more accurately, and the accuracy of determining the duplicate removal position is further improved.
In some embodiments, the bottom edge of the first image frame may be perpendicular to a direction of movement of the camera, and based on the first pixel position, removing the pavement damage prediction result in the repeated pavement area from the pavement damage prediction image may include: determining a straight line in the pavement damage prediction image, wherein the straight line is parallel to the bottom edge of the pavement damage prediction image and passes through the first pixel position; and removing the road surface damage prediction result in the image area above the straight line from the road surface damage prediction image.
In one example, as shown in fig. 3B and 3C, after the first pixel position 334 is determined, a straight line 335 parallel to the bottom edge of the road surface damage prediction image 330 may be determined based on the first pixel position 334, where the straight line 335 passes through the first pixel position 334, and the road surface area in the portion above the straight line 335 is the road surface area overlapping with the road surface area 323 in the second image frame 320; subsequently, a duplicate image can be obtained by removing the road surface defect prediction result 331 in the portion above the straight line 335 in the road surface defect prediction image 330.
In some embodiments, the image processing method for intelligent transportation may further include: identifying a plurality of lane regions in a first image frame; and acquiring a road surface disease prediction result in an area of the duplicate removal image corresponding to at least one of the plurality of lane areas based on a corresponding relationship between the pixel position of the first image frame and the pixel position of the duplicate removal image.
In some embodiments, identifying the plurality of lane regions in the first image frame may include: a plurality of lane regions in the first image frame are identified using a lane line identification model.
In one example, as shown in fig. 3A and 3C, a lane line recognition model may be first applied to recognize a plurality of lane lines 316 in the first image frame 310 to obtain a plurality of lane regions 317; the road surface damage prediction result in the area of the deduplication image corresponding to at least one of the plurality of lane areas may then be acquired based on the correspondence between the deduplication image (i.e., the road surface damage prediction image 330 from which the road surface damage prediction result 331 is removed) and the first image frame 310.
Therefore, after the duplicate of the repeated result in the predicted image is removed, the road surface disease prediction result in a certain specific lane can be further obtained through the corresponding relation, so that the automatic screening of the road surface disease prediction result of the specific road surface area is realized, the screening efficiency is improved, and the labor cost is saved.
In some embodiments, the camera may be arranged such that the scene depth at the bottom edge of the first and second image frames is less than the scene depth at the top edge of the first and second image frames.
Therefore, the image acquired by the camera has a clearer imaging effect on the part close to the camera, and therefore the pavement damage prediction result in the part is more accurate. The camera is processed by the image processing method, and the obtained road surface disease prediction result is more accurate.
According to some embodiments, as shown in fig. 7, there is also provided an image processing apparatus 700 for intelligent transportation, which may include: a first acquisition unit 710 configured to acquire a first image frame and a second image frame continuously photographed by the camera with respect to a road surface during movement along the road, the first image frame and the second image frame having a repeated road surface area therein, a bottom edge of the second image frame corresponding to a bottom boundary of the repeated road surface area in the first image frame; a first determining unit 720 configured to determine a real-world distance between the first image frame and the second image frame, the real-world distance indicating a distance between real-world positions at which the camera was located when the first image frame and the second image frame were taken, respectively; a first identifying unit 730 configured to perform pavement damage identification on the first image frame to obtain a pavement damage prediction image indicating a pavement damage prediction result in the first image frame, wherein pixel positions in the pavement damage prediction image have a corresponding relationship with pixel positions in the first image frame; a second determining unit 740, configured to determine, according to the real world distance, the calibration parameter of the camera, and the corresponding relationship, a first pixel position corresponding to a first edge position of the second image frame in the road surface damage prediction image, where the first edge position is any one pixel position on a bottom edge of the second image frame; and a removing unit 750 configured to remove the road surface defect prediction result in the repeated road surface region from the road surface defect prediction image based on the first pixel position to obtain a duplicate removal image.
The operations performed by the units 710 to 750 in the image processing apparatus 700 for intelligent transportation are similar to the operations performed by the steps S201 to S205 in the image processing method 200 for intelligent transportation, and are not described herein again.
According to some embodiments, as shown in fig. 8, there is also provided an image processing apparatus 800 for smart transportation for a monocular camera, which includes an acquisition unit 810, a first determination unit 820, a first recognition unit 830, a second determination unit 840, and a removal unit 850. The second determination unit 840 may include: a first determining subunit 841 configured to determine, based on the calibration parameters and the corresponding relationship, a first coordinate of a second edge position of the road surface disease prediction image in a world coordinate system, where the second edge position is any pixel position on a bottom edge of the road surface disease prediction image; a second determining subunit 842 configured to determine a second coordinate in the world coordinate system, the second coordinate being the real-world distance away from the first coordinate along the direction of movement of the camera; and a third determining subunit 843 configured to determine, based on the calibration parameter, a pixel position corresponding to the second coordinate in the road surface disease prediction image as the first pixel position.
The operations performed by the units 810-850 and the sub-unit 841-843 of the image processing apparatus 800 for intelligent transportation are similar to the operations performed by the steps S401-S407 in the image processing method 400 for intelligent transportation, and are not described herein again.
According to some embodiments, as shown in fig. 9, there is also provided an image processing apparatus 900 for intelligent transportation for a binocular camera, which includes an acquisition unit 910, a first determination unit 920, a first recognition unit 930, a second determination unit 940, and a removal unit 950. The first image frame and the second image frame are captured by a first camera of a binocular camera. The second determination unit 940 may include: a fourth determining subunit 941, configured to determine a scene depth corresponding to a second edge position of the first image frame based on the first image frame, a reference image frame corresponding to the first image frame and captured by a second camera of the binocular cameras, and the calibration parameter, wherein the second edge position is any one pixel position on a bottom edge of the first image frame; a fifth determining sub-unit 942 configured to determine, based on the scene depth, the calibration parameter, and the correspondence, a third coordinate of a third edge position of the road surface disease prediction image corresponding to the second edge position in the world coordinate system; a sixth determining subunit 943 configured to determine a fourth coordinate in the world coordinate system, the fourth coordinate being a real-world distance from the third coordinate along the moving direction of the camera; and a seventh determining subunit 944, configured to determine, based on the calibration parameter, a pixel position corresponding to the fourth coordinate in the road surface disease prediction image as the first pixel position.
The operations performed by the units 910 to 950 and the sub-units 941 to 944 in the image processing apparatus 900 for intelligent transportation are similar to the operations performed by the steps S501 to S508 in the image processing method 500 for intelligent transportation, and are not described herein again.
According to some embodiments, the bottom edge of the first image frame is perpendicular to a moving direction of the camera, and wherein the removing unit comprises: an eighth determining subunit configured to determine, in the road surface damage prediction image, a straight line that is parallel to a bottom edge of the road surface damage prediction image and passes through the first pixel position; and a removal subunit configured to remove the road surface defect prediction result in the image area above the straight line from the road surface defect prediction image.
According to some embodiments, the camera may be arranged such that the scene depth at the bottom edge of the first and second image frames is less than the scene depth at the top edge of the first and second image frames.
According to some embodiments, the first identification unit may comprise: and the first identification subunit is configured to identify the pavement damage from the first image frame by using a pavement damage identification model.
According to some embodiments, the image processing apparatus for intelligent transportation described above may further include: a second recognition unit configured to recognize a plurality of lane regions in the first image frame; and a second acquisition unit configured to acquire a road surface disease prediction result in an area of the deduplication image corresponding to at least one of the plurality of lane areas, based on a correspondence between pixel positions of the first image frame and pixel positions of the deduplication image.
According to some embodiments, the second identification unit may comprise: a second recognition subunit configured to recognize a plurality of lane regions in the first image frame using the lane line recognition model.
According to an embodiment of the present disclosure, there is also provided an electronic device, a readable storage medium, and a computer program product.
Referring to fig. 10, a block diagram of a structure of an electronic device 1000, which may be a server or a client of the present disclosure, which is an example of a hardware device that may be applied to aspects of the present disclosure, will now be described. Electronic device is intended to represent various forms of digital electronic computer devices, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 10, the electronic device 1000 includes a computing unit 1001 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)1002 or a computer program loaded from a storage unit 1008 into a Random Access Memory (RAM) 1003. In the RAM 1003, various programs and data necessary for the operation of the electronic apparatus 1000 can also be stored. The calculation unit 1001, the ROM 1002, and the RAM 1003 are connected to each other by a bus 1004. An input/output (I/O) interface 1005 is also connected to bus 1004.
A number of components in the electronic device 1000 are connected to the I/O interface 1005, including: input section 1006, output section 1007, storage section 1008, and communication section 1009. The input unit 1006 may be any type of device capable of inputting information to the electronic device 1000, and the input unit 1006 may receive input numeric or character information and generate key signal inputs related to user settings and/or function controls of the electronic device, and may include, but is not limited to, a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and/or a remote controller. Output unit 1007 may be any type of device capable of presenting information and may include, but is not limited to, a display, speakers, a video/audio output terminal, a vibrator, and/or a printer. The storage unit 1008 may include, but is not limited to, a magnetic disk, an optical disk. The communication unit 1009 allows the electronic device 1000 to pass through a computer network such as the internet and/or various telecommunication networksExchange information/data with other devices, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and/or a chipset, such as bluetoothTMDevices, 802.11 devices, WiFi devices, WiMax devices, cellular communication devices, and/or the like.
Computing unit 1001 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The computing unit 1001 performs the various methods and processes described above, such as the method 200. For example, in some embodiments, the method 200 may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program may be loaded and/or installed onto electronic device 1000 via ROM 1002 and/or communications unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the method 200 described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform the method 200 in any other suitable manner (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user may provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be performed in parallel, sequentially or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.
Although embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it is to be understood that the above-described methods, systems and apparatus are merely exemplary embodiments or examples and that the scope of the present invention is not limited by these embodiments or examples, but only by the claims as issued and their equivalents. Various elements in the embodiments or examples may be omitted or may be replaced with equivalents thereof. Further, the steps may be performed in an order different from that described in the present disclosure. Further, various elements in the embodiments or examples may be combined in various ways. It is important that as technology evolves, many of the elements described herein may be replaced with equivalent elements that appear after the present disclosure.

Claims (19)

1. An image processing method for intelligent transportation, comprising:
acquiring a first image frame and a second image frame which are continuously shot by a camera for a road surface in the process of moving along the road, wherein repeated road surface areas exist in the first image frame and the second image frame, and the bottom edge of the second image frame corresponds to the bottom boundary of the repeated road surface areas in the first image frame;
determining a real-world distance between the first image frame and the second image frame, the real-world distance being indicative of a distance between real-world locations at which the camera was located when capturing the first image frame and the second image frame, respectively;
performing pavement damage identification on the first image frame to obtain a pavement damage prediction image indicating a pavement damage prediction result in the first image frame, wherein pixel positions in the pavement damage prediction image have a corresponding relation with pixel positions in the first image frame;
determining a first pixel position corresponding to a first edge position of the second image frame in the pavement damage prediction image according to the real world distance, the calibration parameter of the camera and the corresponding relation, wherein the first edge position is any pixel position on the bottom edge of the second image frame; and
and removing the pavement damage prediction result in the repeated pavement area from the pavement damage prediction image based on the first pixel position to obtain a duplicate removal image.
2. The method according to claim 1, wherein the camera is a monocular camera, and wherein the determining that the first edge position of the second image frame is at the first pixel position corresponding to the road surface disturbance prediction image comprises:
determining a first coordinate of a second edge position of the pavement damage prediction image in a world coordinate system based on the calibration parameters and the corresponding relation, wherein the second edge position is any pixel position on the bottom edge of the pavement damage prediction image;
determining a second coordinate in the world coordinate system that is the real-world distance from the first coordinate along the direction of movement of the camera; and
and determining the pixel position corresponding to the second coordinate in the pavement damage prediction image as the first pixel position based on the calibration parameter and the corresponding relation.
3. The method of claim 1, wherein the camera is a binocular camera and the first image frame and the second image frame are captured by a first camera of the binocular camera, and wherein the determining a first pixel location corresponding to a first edge location of the second image frame in the road damage prediction image comprises:
determining a scene depth corresponding to a second edge position of the first image frame based on the first image frame, a reference image frame corresponding to the first image frame and captured by a second camera of the binocular cameras, and the calibration parameters, wherein the second edge position is any pixel position on the bottom edge of the first image frame;
determining a third coordinate of a third edge position, corresponding to the second edge position, of the pavement disease prediction image in a world coordinate system based on the scene depth, the calibration parameters and the corresponding relation;
determining a fourth coordinate in the world coordinate system, the fourth coordinate being the real-world distance from the third coordinate along the direction of movement of the camera; and
and determining the pixel position of the fourth coordinate in the pavement damage prediction image as the first pixel position based on the calibration parameter and the corresponding relation.
4. The method according to claim 1, wherein a bottom edge of the first image frame is perpendicular to a direction of movement of the camera, and wherein the removing of the pavement damage prediction result in the repeated pavement area from the pavement damage prediction image based on the first pixel position comprises:
determining a straight line in the pavement damage prediction image, wherein the straight line is parallel to the bottom edge of the pavement damage prediction image and passes through the first pixel position; and
and removing the road surface damage prediction result in the image area above the straight line from the road surface damage prediction image.
5. The method of claim 1, wherein the camera is arranged such that a scene depth at a bottom edge of the first and second image frames is less than a scene depth at a top edge of the first and second image frames.
6. The method of any of claims 1-5, wherein the identifying of the first image frame for pavement damage comprises:
and identifying the pavement diseases from the first image frame by using a pavement disease identification model.
7. The method of any of claims 1 to 5, further comprising:
identifying a plurality of lane regions in the first image frame; and
acquiring a road surface damage prediction result in an area of the deduplication image corresponding to at least one of the plurality of lane areas based on a correspondence between pixel positions of the first image frame and pixel positions of the deduplication image.
8. The method of claim 7, wherein the identifying a plurality of lane regions in the first image frame comprises:
a plurality of lane regions in the first image frame are identified using a lane line identification model.
9. An image processing apparatus for intelligent transportation, comprising:
a first acquisition unit configured to acquire a first image frame and a second image frame continuously captured by a camera for a road surface during movement along the road, the first image frame and the second image frame having a repeated road surface area therein, a bottom edge of the second image frame corresponding to a bottom boundary of the repeated road surface area in the first image frame;
a first determining unit configured to determine a real-world distance between the first image frame and the second image frame, the real-world distance indicating a distance between real-world positions at which the camera was located when the first image frame and the second image frame were captured, respectively;
a first identification unit configured to perform pavement damage identification on the first image frame to obtain a pavement damage prediction image indicating a pavement damage prediction result in the first image frame, wherein pixel positions in the pavement damage prediction image have a corresponding relationship with pixel positions in the first image frame;
a second determining unit, configured to determine, according to the real-world distance, the calibration parameter of the camera, and the correspondence, a first pixel position corresponding to a first edge position of the second image frame in the road surface damage prediction image, where the first edge position is any one pixel position on the bottom edge of the second image frame; and
a removing unit configured to remove the road surface damage prediction result in the repeated road surface area from the road surface damage prediction image based on the first pixel position to obtain a duplication removed image.
10. The apparatus of claim 9, wherein the camera is a monocular camera, and wherein the second determining unit comprises:
a first determining subunit, configured to determine, based on the calibration parameters and the corresponding relationship, a first coordinate of a second edge position of the road surface disease prediction image in a world coordinate system, where the second edge position is any pixel position on a bottom edge of the road surface disease prediction image;
a second determining subunit configured to determine a second coordinate in the world coordinate system, the second coordinate being the real-world distance from the first coordinate along the moving direction of the camera; and
and the third determining subunit is configured to determine, based on the calibration parameter, a pixel position corresponding to the second coordinate in the road surface disease prediction image as the first pixel position.
11. The apparatus of claim 9, wherein the camera is a binocular camera and the first image frame and the second image frame are captured by a first camera of the binocular camera, and wherein the second determining unit comprises:
a fourth determining subunit configured to determine a scene depth corresponding to a second edge position of the first image frame based on the first image frame, a reference image frame corresponding to the first image frame and captured by a second camera of the binocular cameras, and the calibration parameter, wherein the second edge position is any one of pixel positions on a bottom edge of the first image frame;
a fifth determining subunit, configured to determine, based on the scene depth, the calibration parameter, and the correspondence, a third coordinate of a third edge position of the road surface disease prediction image, which corresponds to the second edge position, in a world coordinate system;
a sixth determining subunit configured to determine a fourth coordinate in the world coordinate system, the fourth coordinate being the real-world distance from the third coordinate along the moving direction of the camera; and
a seventh determining subunit, configured to determine, based on the calibration parameter, a pixel position corresponding to the fourth coordinate in the road surface disease prediction image as the first pixel position.
12. The apparatus of claim 9, wherein a bottom edge of the first image frame is perpendicular to a moving direction of the camera, and wherein the removal unit comprises:
an eighth determining subunit configured to determine, in the road surface disturbance prediction image, a straight line that is parallel to a bottom edge of the road surface disturbance prediction image and passes through the first pixel position; and
a removing subunit configured to remove the road surface defect prediction result in the image area above the straight line from the road surface defect prediction image.
13. The apparatus of claim 9, wherein the camera is arranged such that a scene depth at a bottom edge of the first image frame and the second image frame is less than a scene depth at a top edge of the first image frame and the second image frame.
14. The apparatus according to any one of claims 9 to 13, wherein the first identifying unit comprises:
a first identifying subunit configured to identify a road surface fault from the first image frame using a road surface fault identification model.
15. The apparatus of any of claims 9 to 13, further comprising:
a second recognition unit configured to recognize a plurality of lane regions in the first image frame; and
a second acquisition unit configured to acquire a road surface disease prediction result in an area of the deduplication image corresponding to at least one of the plurality of lane areas, based on a correspondence relationship between pixel positions of the first image frame and pixel positions of the deduplication image.
16. The apparatus of claim 15, wherein the second identifying unit comprises:
a second recognition subunit configured to recognize a plurality of lane regions in the first image frame using a lane line recognition model.
17. An electronic device, comprising:
at least one processor; and
at least one memory communicatively coupled to the at least one processor; wherein
The at least one memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.
18. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-8.
19. A computer program product comprising a computer program, wherein the computer program realizes the method of any one of claims 1-8 when executed by a processor.
CN202210178793.0A 2022-02-25 2022-02-25 Image processing method and device for intelligent transportation, electronic equipment and medium Pending CN114549988A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210178793.0A CN114549988A (en) 2022-02-25 2022-02-25 Image processing method and device for intelligent transportation, electronic equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210178793.0A CN114549988A (en) 2022-02-25 2022-02-25 Image processing method and device for intelligent transportation, electronic equipment and medium

Publications (1)

Publication Number Publication Date
CN114549988A true CN114549988A (en) 2022-05-27

Family

ID=81679181

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210178793.0A Pending CN114549988A (en) 2022-02-25 2022-02-25 Image processing method and device for intelligent transportation, electronic equipment and medium

Country Status (1)

Country Link
CN (1) CN114549988A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117197412A (en) * 2023-10-09 2023-12-08 西安大地测绘股份有限公司 AR-based intelligent highway disease inspection system and method

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117197412A (en) * 2023-10-09 2023-12-08 西安大地测绘股份有限公司 AR-based intelligent highway disease inspection system and method
CN117197412B (en) * 2023-10-09 2024-04-05 西安大地测绘股份有限公司 AR-based intelligent highway disease inspection system and method

Similar Documents

Publication Publication Date Title
EP3008694B1 (en) Interactive and automatic 3-d object scanning method for the purpose of database creation
CN107748569B (en) Motion control method and device for unmanned aerial vehicle and unmanned aerial vehicle system
WO2020113423A1 (en) Target scene three-dimensional reconstruction method and system, and unmanned aerial vehicle
CN115147558B (en) Training method of three-dimensional reconstruction model, three-dimensional reconstruction method and device
CN115578433B (en) Image processing method, device, electronic equipment and storage medium
CN115631418B (en) Image processing method and device and training method of nerve radiation field
CN111721281B (en) Position identification method and device and electronic equipment
JP7351892B2 (en) Obstacle detection method, electronic equipment, roadside equipment, and cloud control platform
CN115239888B (en) Method, device, electronic equipment and medium for reconstructing three-dimensional face image
CN112967345A (en) External parameter calibration method, device and system of fisheye camera
CN113887400A (en) Obstacle detection method, model training method and device and automatic driving vehicle
CN114549988A (en) Image processing method and device for intelligent transportation, electronic equipment and medium
CN114627268A (en) Visual map updating method and device, electronic equipment and medium
CN113610702A (en) Picture construction method and device, electronic equipment and storage medium
CN115965939A (en) Three-dimensional target detection method and device, electronic equipment, medium and vehicle
CN113378605A (en) Multi-source information fusion method and device, electronic equipment and storage medium
CN114299192B (en) Method, device, equipment and medium for positioning and mapping
CN108564626B (en) Method and apparatus for determining relative pose angle between cameras mounted to an acquisition entity
CN113920174A (en) Point cloud registration method, device, equipment, medium and automatic driving vehicle
CN115578515A (en) Training method of three-dimensional reconstruction model, and three-dimensional scene rendering method and device
CN115578432B (en) Image processing method, device, electronic equipment and storage medium
CN113063421A (en) Navigation method and related device, mobile terminal and computer readable storage medium
CN107703954B (en) Target position surveying method and device for unmanned aerial vehicle and unmanned aerial vehicle
WO2020107487A1 (en) Image processing method and unmanned aerial vehicle
CN112449104B (en) Computer storage medium storing computer program

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination