CN113762272A

CN113762272A - Road information determination method and device and electronic equipment

Info

Publication number: CN113762272A
Application number: CN202111064513.5A
Authority: CN
Inventors: 马桥
Original assignee: Beijing Elite Road Technology Co ltd
Current assignee: Beijing Elite Road Technology Co ltd
Priority date: 2021-09-10
Filing date: 2021-09-10
Publication date: 2021-12-07

Abstract

The invention provides a method and a device for determining road information and electronic equipment, and relates to the technical field of artificial intelligence such as image video processing, intelligent parking and unmanned driving. The specific implementation scheme comprises the following steps: when determining the road information in the road to be detected, semantic segmentation processing can be firstly carried out on the image of the road to be detected to obtain a lane line region in the image; detecting key points of the image to obtain a plurality of parking space angular points of the road to be detected; therefore, the road information in the road to be detected can be determined jointly by combining the road line area and the multiple parking space angular points, and the accuracy of the determined road information is improved.

Description

Road information determination method and device and electronic equipment

Technical Field

The present disclosure relates to the field of image processing technologies, and in particular, to a method and an apparatus for determining road information, and an electronic device, and in particular, to the technical fields of image video processing, smart parking, unmanned driving, and other artificial intelligence.

Background

Determining road information is crucial to the safe driving of the vehicle. At present, under certain scenes, road information is shielded, and the like, so that the road information cannot be accurately determined.

Therefore, how to accurately determine the road information in the road is a problem that needs to be solved by those skilled in the art.

Disclosure of Invention

The disclosure provides a method and a device for determining road information and electronic equipment, which can accurately determine the road information in a road and improve the accuracy of the determined road information.

According to a first aspect of the present disclosure, there is provided a method of determining road information, which may include:

and performing semantic segmentation processing on the image of the road to be detected to obtain a lane line area in the image.

And detecting key points of the image to obtain a plurality of parking space angular points in the road to be detected.

And determining the road information in the road to be detected according to the lane line area and the plurality of parking space angular points.

According to a second aspect of the present disclosure, there is provided a road information determination device that may include:

and the segmentation unit is used for performing semantic segmentation processing on the image of the road to be detected to obtain a lane line area in the image.

And the detection unit is used for detecting key points of the image to obtain a plurality of parking space angular points in the road to be detected.

And the first determining unit is used for determining the road information in the road to be detected according to the lane line region and the plurality of parking space angular points.

According to a third aspect of the present disclosure, there is provided an electronic device, which may include:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of determining road information of the first aspect.

According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the method for determining road information of the first aspect.

According to a fifth aspect of the present disclosure, there is provided a computer program product comprising: a computer program stored in a readable storage medium, from which at least one processor of an electronic device can read the computer program, the at least one processor executing the computer program causing the electronic device to execute the method of determining road information according to the first aspect.

According to the technical scheme, the road information in the road can be accurately determined, and the accuracy of the determined road information is improved.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

fig. 1 is a schematic flowchart of a road information determination method provided according to a first embodiment of the present disclosure;

fig. 2 is a schematic diagram of an image of a road to be detected according to an embodiment of the disclosure;

fig. 3 is a schematic diagram of a parking space line region obtained by performing semantic segmentation processing on an image according to an embodiment of the present disclosure;

fig. 4 is a schematic view of a plurality of parking space angle points of a road to be detected according to an embodiment of the present disclosure;

fig. 5 is a schematic diagram of an image of another road to be detected according to the embodiment of the disclosure;

FIG. 6 is a schematic diagram of an image after processing according to an embodiment of the present disclosure;

fig. 7 is a flowchart illustrating a road information determination method according to a second embodiment of the present disclosure;

FIG. 8 is a schematic diagram of a target line segment provided by embodiments of the present disclosure;

FIG. 9 is a schematic diagram of a lateral vehicle-to-vehicle line obtained by fitting a straight line according to an embodiment of the present disclosure;

fig. 10 is a flowchart illustrating a road information determination method according to a third embodiment of the present disclosure;

fig. 11 is a schematic diagram of other parking space line pixel points except the parking space line pixel point corresponding to the lateral parking space line provided in the embodiment of the present disclosure;

fig. 12 is a schematic view of an area corresponding to a parking space on a road to be detected according to an embodiment of the present disclosure;

fig. 13 is a schematic view of an area corresponding to a parking space on another road to be detected according to the embodiment of the disclosure;

FIG. 14 is an image of a road collected in a practical application scenario provided by the embodiment of the present disclosure;

FIG. 15 is a schematic diagram of a lane in road detected in a practical application scenario according to an embodiment of the present disclosure;

fig. 16 is a schematic structural diagram of a road information determination device provided according to a fourth embodiment of the present disclosure;

fig. 17 is a schematic block diagram of an electronic device provided by an embodiment of the disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In embodiments of the present disclosure, "at least one" means one or more, "a plurality" means two or more. "and/or" describes the access relationship of the associated object, meaning that there may be three relationships, e.g., A and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone, wherein A and B can be singular or plural. In the description of the text of the present disclosure, the character "/" generally indicates that the former and latter associated objects are in an "or" relationship. In addition, in the embodiments of the present disclosure, "first", "second", "third", "fourth", "fifth", and "sixth" are only used to distinguish the contents of different objects, and have no other special meaning.

The technical scheme provided by the embodiment of the disclosure can be applied to the scene of parking space line detection. The parking spaces in the road can be accurately determined by detecting the parking space lines so as to effectively manage the parking spaces in the road. The parking space refers to a rectangular area which is planned in a road and used for parking vehicles, and the parking space line refers to a line segment used for marking the edge of the parking space area. In general, a parking space line comprises a lateral parking space line and a parking space dividing line; the lateral position lines are position lines parallel to the lane direction in the position lines, and the position dividing lines are position lines perpendicular to the lane direction in the position lines.

When determining a parking space area in a road, it is necessary to rely on road information in the road, such as a lane direction or a vehicle line, and therefore, in order to effectively manage the parking space in the road, the road information in the road may be determined first. At present, under certain scenes, road information is shielded, and the like, so that the road information cannot be accurately determined.

In order to accurately determine the road information in the road, the image of the road to be detected can be acquired first, and the serious shielding of the road information caused by the fact that the vehicle, the pedestrian, the stall or the parking space does not standardize the parking and other factors in some complex road scenes is considered, so that the complete road information is difficult to extract based on the acquired image. Based on this, the parking space angular points of the parking space lines are considered to be generally rarely blocked and can be well adapted to the complex road sections, so that the image of the road to be detected can be subjected to semantic segmentation processing to obtain the lane line area in the image, the road information of the road to be detected is jointly determined by combining a plurality of parking space angular points of the road to be detected, which are detected by the key points, and the accuracy of the determined road information is improved.

It can be understood that, in general, the region corresponding to the parking space is a rectangular frame region, and correspondingly, the corner point of the parking space may be understood as four vertexes of the rectangular frame, or may also be understood as four corners of the rectangular frame.

Based on the above technical concept, embodiments of the present disclosure provide a method for determining road information, and the method for determining road information provided by the present disclosure will be described in detail by specific embodiments. It is to be understood that the following detailed description may be combined with other embodiments, and that the same or similar concepts or processes may not be repeated in some embodiments.

Example one

Fig. 1 is a flowchart illustrating a road information determining method provided according to a first embodiment of the present disclosure, where the road information determining method may be performed by software and/or a hardware device, for example, the hardware device may be a terminal or a server. For example, referring to fig. 1, the method for determining the road information may include:

s101, performing semantic segmentation processing on the image of the road to be detected to obtain a lane line area in the image.

For example, when acquiring an image of a road to be detected, the image of the road to be detected may be acquired by road side equipment, or the image of the road to be detected sent by other electronic equipment may also be received, and the setting may be specifically performed according to actual needs.

After the image of the road to be detected is acquired, semantic segmentation processing can be performed on the image to obtain a lane line region in the image. For example, when performing semantic segmentation processing on the image, a deep learning semantic segmentation technology may be used to perform semantic segmentation processing on the image, or other semantic segmentation processing technologies may be used to perform semantic segmentation processing on the image, which may be specifically set according to actual needs. It can be understood that, by using the deep learning semantic segmentation technology, the stall line region can be used as a class of target construction segmenter, so that semantic segmentation processing is performed on the image based on the constructed segmenter.

For example, please refer to fig. 2, fig. 2 is a schematic diagram of an image of a road to be detected according to an embodiment of the present disclosure, the image shown in fig. 2 includes the road to be detected, and the road to be detected includes a plurality of parking spaces, because some parking spaces have vehicles parked therein, some parking spaces are blocked, and a semantic segmentation processing result, which is a parking space line region segmented from the image, is obtained by performing semantic segmentation processing on the image, as shown in fig. 3, fig. 3 is a schematic diagram of a parking space line region obtained by performing semantic segmentation processing on the image according to an embodiment of the present disclosure, it can be seen that the parking space line region shown in fig. 3 includes a plurality of parking space line pixel points, which include not only parking space pixel points of lateral parking space lines, but also comprises pixel points of parking space dividing lines.

It can be understood that, in the embodiment of the present disclosure, the image is subjected to semantic segmentation processing by using a deep learning semantic segmentation technology, and the parking space line pixel points in the image can be segmented more accurately to a certain extent.

Under some complex road scenes, the road information is seriously shielded due to the fact that vehicles, pedestrians, stalls or parking spaces are not standardized, and the like, and therefore, the parking space angular points of the parking space line are generally rarely shielded, and the method can be well suitable for complex road sections, and further key point detection can be carried out on images to obtain a plurality of parking space angular points of the road to be detected, so that the road information in the road to be detected can be determined jointly by combining the parking space angular points, and the following S102-S103 is executed:

s102, carrying out key point detection on the image to obtain a plurality of parking space angular points of the road to be detected.

For example, when performing key point detection on an image, a deep learning key point detection technology may be adopted to perform key point detection on the image, or other key point detection technologies may be adopted to perform key point detection on the image, which may be specifically set according to actual needs. It can be understood that, by using the deep learning key point detection technology, the parking space corner points can be used as a class of targets to construct a detector, so that key point detection is performed based on constructed detector images. For example, the target detection algorithm may be yolov4 algorithm, or fast-rcnn, and the like, and may be specifically set according to actual needs.

Under a normal condition, when the parking space line in the image is not shielded and the parking space in the image is a closed area, detecting key points of the image to obtain four complete parking space angular points; when the parking space line in the image is not shielded and the parking space in the image is a non-closed area, detecting key points of the image to obtain two complete parking space angular points; but when the parking space line in the image is shielded, the key point detection is carried out on the image, and partial parking space angular points can be obtained. The areas corresponding to the parking spaces can be accurately determined through the four parking space angular points; and the areas corresponding to the parking spaces cannot be accurately determined through the two parking space angular points.

By combining the image shown in fig. 1, after the image is subjected to key point detection, a plurality of parking space angular points of the road to be detected can be detected, for example, see fig. 4, where fig. 4 is a schematic diagram of a plurality of parking space angular points of the road to be detected provided by the embodiment of the present disclosure, and it can be seen that the detected parking space angular points are partial parking space angular points of parking spaces.

It should be noted that the semantic segmentation processing operation performed on the image of the road to be detected in S101 and the key point detection operation performed on the image in S102 may be implemented by one multitask neural network model, or may be implemented by two independent neural network models, which may be specifically set according to actual needs, and the embodiment of the present disclosure is not limited further herein. For example, assuming that fig. 5 is a schematic view of another image of a road to be detected provided in the embodiment of the present disclosure, a vehicle line region and a plurality of corresponding parking space corner points may be obtained through semantic segmentation processing operation performed on the image of the road to be detected and key point detection operation performed on the image, as shown in fig. 6, fig. 6 is a schematic view of the processed image provided in the embodiment of the present disclosure, each parking space shown in fig. 6 corresponds to four complete parking space corner points, and a region corresponding to the parking space is a closed region.

In addition, in the embodiment of the present disclosure, there is no sequence between the above S101 and the above S102, and the above S101 may be executed first, and then the above S102 may be executed; s102 may be executed first and then S101 may be executed, or S101 and S102 may be executed simultaneously, and may be specifically configured according to actual needs, and here, the embodiment of the present disclosure is only described by taking the example of executing S101 first and then executing S102, but the embodiment of the present disclosure is not limited thereto.

After the lane line region in the image of the road to be detected is acquired through the S101 and the plurality of parking space corner points in the image of the S102, the following S103 may be further executed:

s103, determining road information in the road to be detected according to the lane line area and the multiple parking space angular points.

For example, the road information may be a lane direction, an area corresponding to a parking space, or other road information, and may be specifically set according to actual needs. For example, the lane direction may be an east-west direction or a north-south direction, and may be specifically set according to actual needs, and herein, the embodiment of the present disclosure is not specifically limited.

It can be seen that, in the embodiment of the present disclosure, when determining the lane direction in the road to be detected, semantic segmentation processing may be performed on the image of the road to be detected first to obtain a lane line region in the image; detecting key points of the image to obtain a plurality of parking space angular points of the road to be detected; therefore, the road information in the road to be detected can be determined jointly by combining the road line area and the multiple parking space angular points, and the accuracy of the determined road information is improved.

Based on the embodiment shown in fig. 1, for example, when the road information includes the lane direction, in order to further understand how to determine the lane direction of the road to be detected according to the lane line region and the multiple corner points in S103, the following will describe in detail through the following embodiment two shown in fig. 7.

Example two

Fig. 7 is a flowchart illustrating a road information determining method according to a second embodiment of the disclosure, where the road information determining method may be implemented by software and/or a hardware device, for example, the hardware device may be a terminal or a server. For example, referring to fig. 7, the method for determining the road information may include:

s701, performing straight line fitting on the parking space line pixel points in the lane line area to obtain a target line segment.

For example, when performing linear fitting on the parking space line pixel points in the lane line region, a linear fitting technology may be adopted to perform linear fitting on the parking space line pixel points in the lane line region to obtain a target line segment.

With reference to the lane line region shown in fig. 2, after straight line fitting is performed on the parking space line pixel points in the lane line region, a corresponding target line segment may be obtained, as shown in fig. 8, fig. 8 is a schematic diagram of a target line segment provided in the embodiment of the present disclosure, and it can be seen that the direction indicated by the target line segment shown in fig. 8 may be a general lane direction.

In order to obtain a more accurate lane direction, after a target line segment is obtained by performing linear fitting on the parking space line pixel points in the lane line region, linear fitting may be further performed on a plurality of parking space angular points according to the direction indicated by the target line segment, so that a lateral parking space line of the road to be detected may be obtained, that is, the following S702 is performed:

s702, performing linear fitting on the plurality of parking space angular points according to the direction indicated by the target line segment to obtain a lateral parking space line of the road to be detected, wherein the direction indicated by the lateral parking space line is the lane direction of the road to be detected.

For example, when performing linear fitting on a plurality of parking space angular points, the confidence corresponding to each parking space angular point may be determined according to the lane line region; if a certain parking space angular point is located in the lane line area, the parking space angular point is determined to be a credible parking space angular point, and the corresponding confidence degree is greater than a preset threshold value; on the contrary, if a certain parking space angular point is located outside the lane line area, the parking space angular point is an untrusted parking space angular point, and the corresponding confidence coefficient of the parking space angular point is smaller than or equal to the preset threshold value. The value of the preset threshold may be set according to actual needs, and the embodiment of the present disclosure is not particularly limited to the value of the preset threshold.

It can be understood that, if it can be determined in advance that in S102, the key point detection is performed on the image to obtain the parking space angular points where the multiple parking space angular points of the road to be detected are all credible, when performing the linear fitting on the multiple parking space angular points according to the direction indicated by the target line segment, the linear fitting may be performed on the multiple parking space angular points where the confidence coefficient is greater than the preset threshold value, directly according to the direction indicated by the target line segment, without determining the parking space angular points where the confidence coefficient is greater than the preset threshold value according to the lane line region. In this scenario, the parking space angle points of the road to be detected obtained in S102 are all parking space angle points whose confidence degrees are greater than a preset threshold value.

For example, in combination with the target line segment shown in fig. 8 and the multiple parking space angle points shown in fig. 4, after performing straight line fitting on the multiple parking space angle points according to the direction indicated by the target line segment, a lateral lane line of the road to be detected may be obtained.

It can be seen that, in the embodiment of the present disclosure, when determining the lane direction of the road to be detected according to the lane line region and the plurality of parking spot angular points, straight line fitting may be performed on the parking spot line pixel points in the lane line region to obtain a target line segment; and carrying out linear fitting on the plurality of parking space angular points according to the direction indicated by the target line segment to obtain a lateral vehicle position line of the road to be detected.

Based on the embodiment shown in fig. 1, for example, when the road information includes the area corresponding to the parking space, in S103, how to determine the area corresponding to the parking space in the road to be detected according to the lane line area and the multiple parking space corner points, so that the roadside parking spaces in the road to be detected may be centrally managed based on the determined area corresponding to the parking space. Next, a detailed description will be given by the following second embodiment shown in fig. 10.

EXAMPLE III

Fig. 10 is a flowchart illustrating a road information determining method according to a third embodiment of the present disclosure, where the road information determining method may be implemented by software and/or a hardware device, for example, the hardware device may be a terminal or a server. For example, referring to fig. 10, the method for determining the road information may include:

s1001, performing straight line fitting on the parking space line pixel points in the lane line area to obtain a target line segment.

It should be noted that, in S1001, straight line fitting is performed on the parking space line pixel points in the lane line region to obtain an implementation manner of the target line segment, which is similar to the implementation manner of performing straight line fitting on the parking space line pixel points in the lane line region in S701 to obtain the target line segment, and reference may be made to the straight line fitting performed on the parking space line pixel points in the lane line region in S701 to obtain a description of the target line segment, which is not described herein again in this disclosure.

S1002, performing linear fitting on the plurality of parking space angular points according to the direction indicated by the target line segment to obtain a lateral parking space line of the road to be detected.

It should be noted that, in S1002, performing linear fitting on the multiple parking space angular points according to the direction indicated by the target line segment to obtain an implementation manner of the lateral vehicle position line of the road to be detected, which is similar to the implementation manner of performing linear fitting on the multiple parking space angular points according to the direction indicated by the target line segment in S702 to obtain the lateral vehicle position line of the road to be detected, reference may be made to performing linear fitting on the multiple parking space angular points according to the direction indicated by the target line segment in S702 to obtain a related description of the lateral vehicle position line of the road to be detected, which is not described herein again in this disclosure.

S1003, determining an area corresponding to a parking space in the road to be detected according to the lateral parking space line.

For example, when determining the area corresponding to the parking space in the road to be detected according to the lateral lane line, the other parking space line pixel points except the parking space line pixel point corresponding to the lateral lane line may be determined from the parking space line pixel points in the lane line area according to the lateral lane line; performing linear fitting on other parking space line pixel points to obtain a parking space parting line of the road to be detected; and determining the area corresponding to the parking space in the road to be detected according to the lateral parking space line and the parking space dividing line.

For example, according to the lateral vehicle position line, other parking space line pixel points except the parking space line pixel point corresponding to the lateral vehicle position line are determined from the parking space line pixel points in the lane line region, and in combination with the lateral vehicle position line shown in fig. 9, the parking space line pixel point corresponding to the lateral vehicle position line may be removed from the parking space line region shown in fig. 3 to obtain other parking space line pixel points except the parking space line pixel point corresponding to the lateral vehicle position line, for example, see fig. 11, where fig. 11 is a schematic diagram of other parking space line pixel points except the parking space line pixel point corresponding to the lateral vehicle position line provided in the embodiment of the present disclosure, and it can be seen that these other parking space pixel points may be understood as parking space pixel points on a parking space partition line.

The parking space division line of the road to be detected is obtained by fitting the other parking space line pixel points with straight lines by adopting a straight line fitting technology.

After the lateral position line and the parking space dividing line in the road to be detected are respectively determined, the area corresponding to the parking space in the road to be detected can be determined according to the lateral position line and the parking space dividing line. For example, when the image is the image shown in fig. 2, the determined area corresponding to the parking space in the road to be detected according to the lateral parking space line and the parking space dividing line can be shown in fig. 12, where fig. 12 is a schematic view of the area corresponding to the parking space in the road to be detected provided by the embodiment of the present disclosure, and it can be seen that the areas corresponding to the four parking spaces in the road to be detected can be accurately determined according to the lateral parking space line and the parking space dividing line. When the image is the image shown in fig. 4, the determined area corresponding to the parking space in the road to be detected can be shown in fig. 13 according to the lateral parking space line and the parking space dividing line, and fig. 13 is a schematic view of another area corresponding to the parking space in the road to be detected according to the embodiment of the present disclosure.

It can be seen that, in the embodiment of the present disclosure, when determining the area corresponding to the parking space in the road to be detected, other parking space line pixel points except the parking space line pixel point corresponding to the lateral parking space line may be determined from the parking space line pixel points in the lane line area according to the lateral parking space line; performing linear fitting on other parking space line pixel points to obtain a parking space parting line of the road to be detected; and then determining the area corresponding to the parking space in the road to be detected according to the lateral parking space line and the parking space dividing line, so that the area corresponding to the parking space in the road to be detected can be accurately determined, and then carrying out centralized management on roadside parking spaces in the road to be detected based on the determined area corresponding to the parking space.

In an actual application process, please refer to fig. 14 by way of example, fig. 14 is an image of a road collected in an actual application scene provided by the embodiment of the present disclosure, and by using the technical solution provided by the embodiment of the present disclosure, the image of the road shown in fig. 14 is processed to determine a lateral parking space line and a parking space dividing line of the road, where a parking space line formed by the lateral parking space line and the parking space dividing line may be referred to as fig. 15, and fig. 15 is a schematic view of a parking space line in the road detected in the actual application scene provided by the embodiment of the present disclosure.

Based on any one of the above embodiments, it can be understood that, when roadside parking spaces in a road to be detected are centrally managed based on an area corresponding to the parking spaces determined by the above embodiments, a position relationship between a vehicle and the parking spaces in the road to be detected needs to be detected, and the roadside parking spaces in the road to be detected are centrally managed based on the position relationship.

For example, when determining the position relationship between the parking spaces and the parking spaces in the road to be detected, the intersection area and the union area of the area corresponding to the vehicle and the area corresponding to the parking spaces in the road to be detected may be determined respectively; and determining the position relation between the vehicle and the parking space according to the ratio of the intersection area to the union area. For example, if the ratio of the intersection area to the union area is greater than or equal to the set threshold, it indicates that the vehicle is in the parking space; on the contrary, if the ratio of the intersection area to the union area is smaller than the set threshold, it indicates that the vehicle is located outside the parking space, so as to perform centralized management on roadside parking spaces in the road to be detected. The value of the set threshold may be set according to actual needs, and the embodiment of the present disclosure is not particularly limited to the value of the set threshold.

In addition, in view of prior art, when carrying out parking stall line detection, usually based on high-order video equipment, roadside device gathers the image of waiting to detect the road promptly, and operating personnel marks the side direction parking stall line in each parking stall in the roadside parking monitored control system according to the image of waiting to detect the road, can mark as initial side direction parking stall line to judge that the vehicle gets into and leaves the parking stall based on the initial side direction parking stall line of this mark, thereby carry out centralized management to the parking stall. However, when the roadside apparatus is deviated from the preset position and angle, the previously marked initial lateral parking space line becomes an invalid lateral parking space line, and it is impossible to determine whether the vehicle enters or leaves the parking space based on the previously marked initial lateral parking space line, and thus it is impossible to manage the parking space. For example, the high-order video device may be installed in a same-side installation manner, may also be installed in a side-to-side installation manner, may also be installed in a vertical installation manner, and may be specifically set according to actual needs.

Therefore, in a general situation, before the parking spaces are managed in a centralized manner through the pre-marked initial parking space lines, the pre-marked initial lateral parking space lines can be checked, so that whether the position of the road side equipment is shifted or not can be determined through the checking result. For example, the pre-marked initial lateral position line may be verified based on the determined lateral position line in the road to be detected in the embodiment of the present disclosure, so as to determine whether the position of the roadside device is shifted according to the verification result.

For example, when the pre-marked initial lateral position line is verified by the lateral position line in the road to be detected determined according to the embodiment of the disclosure, so as to determine whether the position of the roadside device is shifted according to the verification result, the determined lateral position line in the road to be detected can be matched with the pre-marked initial lateral position line of the roadside device; and determining whether the position of the road side equipment is shifted or not according to the matching result. For example, if the lateral parking space line approximately coincides with the initial lateral parking space line, it is determined that the position of the roadside device is not shifted; if the lateral vehicle position line and the initial lateral vehicle position line are not approximately overlapped, the position of the road side equipment is determined to be shifted, whether the position of the road side equipment is shifted or not can be accurately determined, the situation that the parking space cannot be accurately managed due to the fact that the position of the road side equipment is shifted is avoided, and therefore the accuracy of managing the parking space is improved.

It can be understood that when the determined lateral position line in the road to be detected is matched with the initial lateral position line marked in advance by the roadside device, in an ideal state, an operator completely fits the actual lateral position line through the initial lateral position line marked in advance by the roadside device, and by the technical scheme provided by the embodiment of the disclosure, the determined lateral position line is also completely accurate, so that the initial lateral position line marked in advance by the roadside device can be completely overlapped with the lateral position line; however, in the actual operation process, it is difficult to ensure that the initial lateral position line marked in advance by the roadside device by the operator completely fits the actual lateral position line, and the determined lateral position line cannot be completely accurate through the technical scheme provided by the embodiment of the disclosure, so that a deviation range within a certain degree can be tolerated during matching, and if the matching result is within the deviation range, the initial lateral position line and the lateral position line are approximately coincident; on the contrary, if the matching result is not within the deviation range, it is understood that the initial lateral lane and the lateral lane do not approximately coincide with each other. The value of the deviation range may be set according to actual needs, and the embodiment of the present disclosure is not particularly limited to the value of the deviation range.

Example four

Fig. 16 is a schematic structural diagram of a road information determining device 160 according to a fourth embodiment of the disclosure, and for example, referring to fig. 16, the road information determining device 160 may include:

the segmentation unit 1601 is configured to perform semantic segmentation processing on an image of a road to be detected to obtain a lane line region in the image.

The detecting unit 1602 is configured to perform key point detection on the image to obtain a plurality of parking space corner points in the road to be detected.

The first determining unit 1603 is configured to determine road information in a road to be detected according to a lane line area and a plurality of parking space angle points.

Optionally, the road information includes an area corresponding to the parking space.

Optionally, the first determining unit 1603 includes a first determining module, a second determining module, and a third determining module.

The first determining module is used for performing straight line fitting on the parking space line pixel points in the lane line area to obtain a target line segment.

And the second determining module is used for performing linear fitting on the plurality of parking space angular points according to the direction indicated by the target line segment to obtain a lateral parking space line of the road to be detected.

And the third determining module is used for determining road information in the road to be detected according to the lateral vehicle position line.

Optionally, the second determining module includes a first determining submodule and a second determining submodule.

And the first determining submodule is used for determining the confidence corresponding to each parking space angular point according to the lane line region.

And the second determining submodule is used for fitting the parking space angle points with the confidence degrees larger than the preset threshold value in the plurality of parking space angle points according to the direction indicated by the target line segment.

Optionally, the third determining module includes a third determining submodule, a fourth determining submodule and a fifth determining submodule.

And the third determining submodule is used for determining other parking space line pixel points except the parking space line pixel points corresponding to the lateral vehicle position line from the parking space line pixel points in the lane line area according to the lateral vehicle position line.

And the fourth determining submodule is used for performing linear fitting on the pixel points of other parking space lines to obtain a parking space dividing line of the road to be detected.

And the fifth determining submodule is used for determining road information in the road to be detected according to the lateral vehicle position line and the parking space dividing line.

Optionally, the device for determining road information further includes a second determining unit and a third determining unit.

And the second determining unit is used for respectively determining the areas corresponding to the vehicles in the road to be detected, and the intersection area and the union area of the areas corresponding to the parking spaces.

And the third determining unit is used for determining the position relation between the vehicle and the parking space according to the ratio of the intersection area to the union area.

Optionally, the device for determining road information further includes a matching unit and a fourth determining unit.

And the matching unit is used for matching the lateral position line with the initial lateral position line marked in advance by the road side equipment.

And the fourth determination unit is used for determining whether the position of the road side equipment is shifted or not according to the matching result.

Optionally, the fourth determining unit includes a fourth determining module and a fifth determining module.

And the fourth determining module is used for determining that the position of the road side equipment is not shifted if the lateral parking space line is approximately coincident with the initial lateral parking space line.

And the fifth determining module is used for determining that the position of the road side equipment is shifted if the lateral position line and the initial lateral position line are not approximately overlapped.

The road information determining device 160 provided in the embodiment of the present disclosure may implement the technical solution of the road information determining method shown in any one of the above embodiments, and the implementation principle and the beneficial effects thereof are similar to those of the road information determining method, and reference may be made to the implementation principle and the beneficial effects of the road information determining method, which is not described herein again.

The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

According to an embodiment of the present disclosure, the present disclosure also provides a computer program product comprising: a computer program, stored in a readable storage medium, from which at least one processor of the electronic device can read the computer program, the at least one processor executing the computer program causing the electronic device to perform the solution provided by any of the embodiments described above.

Fig. 17 is a schematic block diagram of an electronic device 170 provided by an embodiment of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 17, the apparatus 170 includes a computing unit 1701 that may perform various appropriate actions and processes in accordance with a computer program stored in a Read Only Memory (ROM)1702 or a computer program loaded from a storage unit 1708 into a Random Access Memory (RAM) 1703. In the RAM1703, various programs and data required for the operation of the device 170 can also be stored. The computing unit 1701, the ROM1702, and the RAM1703 are connected to each other through a bus 1704. An input/output (I/O) interface 1705 is also connected to bus 1704.

Various components in the device 170 are connected to the I/O interface 1705, including: an input unit 1706 such as a keyboard, a mouse, and the like; an output unit 1707 such as various types of displays, speakers, and the like; a storage unit 1708 such as a magnetic disk, optical disk, or the like; and a communication unit 1709 such as a network card, modem, wireless communication transceiver, etc. The communication unit 1709 allows the device 170 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

The computing unit 1701 may be a variety of general purpose and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 1701 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and the like. The calculation unit 1701 executes various methods and processes described above, such as a determination method of road information. For example, in some embodiments, the determination of road information may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 1708. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 170 via ROM1702 and/or communications unit 1709. When the computer program is loaded into the RAM1703 and executed by the computing unit 1701, one or more steps of the determination method of road information described above may be performed. Alternatively, in other embodiments, the computing unit 1701 may be configured in any other suitable manner (e.g., by means of firmware) to perform the determination method of road information.

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The Server can be a cloud Server, also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service ("Virtual Private Server", or simply "VPS"). The server may also be a server of a distributed system, or a server incorporating a blockchain.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A method of determining road information, comprising:

performing semantic segmentation processing on an image of a road to be detected to obtain a lane line area in the image;

detecting key points of the image to obtain a plurality of parking space angular points in the road to be detected;

2. The method of claim 1, wherein the road information comprises an area corresponding to a parking space.

3. The method according to claim 1 or 2, wherein the determining the road information in the road to be detected according to the lane line region and the plurality of parking point points comprises:

performing straight line fitting on the parking space line pixel points in the lane line area to obtain a target line segment;

performing linear fitting on the plurality of parking space angular points according to the direction indicated by the target line segment to obtain a lateral parking space line of the road to be detected;

and determining road information in the road to be detected according to the lateral vehicle position line.

4. The method of claim 3, wherein said fitting a straight line to the plurality of corner points according to the direction indicated by the target line segment comprises:

determining the confidence corresponding to each parking space angular point according to the lane line area;

and fitting the parking space angle points with the confidence degrees larger than a preset threshold value in the plurality of parking space angle points according to the direction indicated by the target line segment.

5. The method according to claim 3 or 4, wherein determining road information in the road to be detected according to the lateral lane comprises:

according to the lateral vehicle position line, determining other parking space line pixel points except the parking space line pixel point corresponding to the lateral vehicle position line from the parking space line pixel points in the lane line area;

performing linear fitting on the other parking space line pixel points to obtain a parking space parting line of the road to be detected;

and determining road information in the road to be detected according to the lateral parking space line and the parking space dividing line.

6. The method of claim 2, further comprising:

respectively determining an intersection area and a union area of areas corresponding to the vehicles in the road to be detected and the parking spaces;

and determining the position relation between the vehicle and the parking space according to the ratio of the intersection area to the union area.

7. The method of any of claims 3-5, further comprising:

matching the lateral position line with an initial lateral position line marked in advance by road side equipment;

and determining whether the position of the road side equipment is shifted or not according to the matching result.

8. The method of claim 7, wherein the determining whether the position of the roadside device is offset according to the matching result comprises:

if the lateral parking space line is approximately coincident with the initial lateral parking space line, determining that the position of the road side equipment is not shifted;

and if the lateral position line and the initial lateral position line are not approximately overlapped, determining that the position of the road side equipment is shifted.

9. An apparatus for determining road information, comprising:

the segmentation unit is used for performing semantic segmentation processing on the image of the road to be detected to obtain a lane line area in the image;

the detection unit is used for detecting key points of the image to obtain a plurality of parking space angular points in the road to be detected;

10. The apparatus of claim 9, wherein the road information comprises an area corresponding to a parking space.

11. The apparatus according to claim 9 or 10, wherein the first determining unit comprises a first determining module, a second determining module, and a third determining module;

the first determining module is used for performing straight line fitting on the parking space line pixel points in the lane line area to obtain a target line segment;

the second determining module is used for performing linear fitting on the plurality of parking space angular points according to the direction indicated by the target line segment to obtain a lateral parking space line of the road to be detected;

and the third determining module is used for determining the road information in the road to be detected according to the lateral vehicle position line.

12. The apparatus of claim 11, wherein the second determination module comprises a first determination submodule and a second determination submodule;

the first determining submodule is used for determining the confidence corresponding to each parking space angular point according to the lane line region;

and the second determining submodule is used for fitting the parking space angle points with the confidence degrees larger than a preset threshold value in the plurality of parking space angle points according to the direction indicated by the target line segment.

13. The apparatus of claim 11 or 12, the third determination module comprising a third determination submodule, a fourth determination submodule, and a fifth determination submodule;

the third determining submodule is used for determining other parking space line pixel points except the parking space line pixel point corresponding to the lateral parking space line from the parking space line pixel points in the lane line area according to the lateral parking space line;

the fourth determining submodule is used for performing linear fitting on the other parking space line pixel points to obtain a parking space dividing line of the road to be detected;

and the fifth determining submodule is used for determining the road information in the road to be detected according to the lateral parking space line and the parking space dividing line.

14. The apparatus of claim 10, further comprising a second determining unit and a third determining unit;

the second determining unit is used for respectively determining an area corresponding to a vehicle in the road to be detected, and an intersection area and a union area of the areas corresponding to the parking spaces;

the third determining unit is configured to determine a positional relationship between the vehicle and the parking space according to a ratio of the intersection area to the union area.

15. The apparatus according to any of claims 11-13, further comprising a matching unit and a fourth determining unit;

the matching unit is used for matching the lateral position line with an initial lateral position line marked in advance by road side equipment;

the fourth determination unit is configured to determine whether the position of the roadside apparatus is shifted according to a matching result.

16. The apparatus of claim 15, wherein the fourth determining unit comprises a fourth determining module and a fifth determining module;

the fourth determining module is configured to determine that the position of the roadside device is not shifted if the lateral parking space line is approximately coincident with the initial lateral parking space line;

the fifth determining module is configured to determine that the position of the roadside device is shifted if the lateral lane line and the initial lateral lane line do not approximately coincide.

17. An electronic device, comprising:

at least one processor; and

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of determining road information of any one of claims 1-8.

18. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the determination method of road information according to any one of claims 1 to 8.

19. A computer program product comprising a computer program which, when being executed by a processor, carries out the steps of the method of determining road information according to any one of claims 1 to 8.