CN117935225B - Shielding identification method of traffic signal lamp, intelligent equipment and storage medium - Google Patents
Shielding identification method of traffic signal lamp, intelligent equipment and storage medium Download PDFInfo
- Publication number
- CN117935225B CN117935225B CN202410282328.0A CN202410282328A CN117935225B CN 117935225 B CN117935225 B CN 117935225B CN 202410282328 A CN202410282328 A CN 202410282328A CN 117935225 B CN117935225 B CN 117935225B
- Authority
- CN
- China
- Prior art keywords
- single lamp
- frame
- lamp
- state
- detection frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 238000001514 detection method Methods 0.000 claims abstract description 134
- 230000000007 visual effect Effects 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 10
- 230000016776 visual perception Effects 0.000 abstract 1
- 230000000903 blocking effect Effects 0.000 description 24
- 238000010586 diagram Methods 0.000 description 5
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 239000003086 colorant Substances 0.000 description 2
- 230000001186 cumulative effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000013475 authorization Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008033 biological extinction Effects 0.000 description 1
- 230000033228 biological regulation Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 235000014510 cooky Nutrition 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Landscapes
- Traffic Control Systems (AREA)
Abstract
The application relates to the technical field of automatic driving, in particular to a shielding identification method of a traffic signal lamp, intelligent equipment and a storage medium, and aims to solve the problem of identifying whether the traffic signal lamp is shielded or not. For this purpose, the method provided by the application comprises the steps of determining the type of an image frame relative to a single lamp according to a first detection frame of the single lamp and a second detection frame of an obstacle in the image frame of the traffic signal lamp, wherein the type comprises an occlusion frame and a visible frame; determining that the single lamp is in an occlusion state when the occlusion integrated time of the single lamp is greater than a first time threshold, wherein the occlusion integrated time is the time when the image frame type is continuously determined as an occlusion frame; after that, if the visible integrated time of the single lamp is greater than the second duration threshold, the normal state is modified, and the visible integrated time is continuously determined as the time of the visible frame by the image frame type. Based on the method, whether the traffic signal lamp is blocked or not can be judged in a pure visual perception mode without depending on a high-precision map.
Description
Technical Field
The application relates to the technical field of automatic driving, in particular to a shielding identification method of a traffic signal lamp, intelligent equipment and a storage medium.
Background
When the vehicle is automatically driven, the state of the traffic signal lamp needs to be accurately perceived, and then the vehicle is controlled to automatically drive according to the perceived result.
The conventional traffic signal lamp sensing method mainly utilizes a high-precision map to locate the area where the traffic signal lamp is located, and then carries out traffic signal lamp identification on the image of the area to obtain the state of the traffic signal lamp. For example, when identifying whether a traffic light is obscured by a dynamic obstacle, the traffic light position is queried via a high-precision map, and based on the perceived dynamic obstacle position, it is determined whether the traffic light is obscured by the dynamic obstacle. If the shielding occurs, the vehicle needs to be controlled to take corresponding driving measures in time, so that the running safety of the vehicle is ensured.
However, the method is severely dependent on a high-precision map, and if the high-precision map is not adopted, whether the traffic signal lamp is blocked by a dynamic obstacle cannot be accurately identified.
Accordingly, there is a need in the art for a new solution to the above-mentioned problems.
Disclosure of Invention
The present application has been made to overcome the above-mentioned drawbacks, and aims to solve or at least partially solve the technical problem of accurately recognizing whether a traffic signal lamp is blocked by a dynamic obstacle without using a high-precision map.
In a first aspect, there is provided a method of identifying occlusion of a traffic signal, comprising:
acquiring a first detection frame of a single lamp in an image frame of the traffic signal lamp and a second detection frame of an obstacle in the image frame;
Determining the type of the image frame relative to the single lamp according to the first detection frame and the second detection frame, wherein the type comprises an shielding frame and a visual frame, the shielding frame indicates that the single lamp is shielded by the obstacle, and the visual frame indicates that the single lamp is not shielded by the obstacle;
When the shielding accumulated time of the single lamp is larger than a first time threshold, determining that the state of the single lamp is a shielding state, wherein the shielding accumulated time is the time when an image frame of the traffic signal lamp is continuously determined to be the shielding frame relative to the type of the single lamp;
After determining the shielding state, if the visible accumulated time of the single lamp is larger than a second duration threshold value, modifying the state of the single lamp from the shielding state to a normal state, otherwise, maintaining the shielding state, wherein the visible accumulated time is the time when the image frame of the traffic signal lamp is continuously determined as the visible frame relative to the type of the single lamp.
In one technical scheme of the shielding identification method of the traffic signal lamp, when the traffic signal lamp comprises a plurality of single lamps, the method further comprises:
after determining the state of each single lamp, acquiring the semantic meaning of a target single lamp, wherein the target single lamp is a single lamp in a shielding state;
And if the single lamps with the semantics in the plurality of single lamps are all in the shielding state, determining that the state of the traffic signal lamp under the semantics is the shielding state.
In one technical scheme of the method for identifying the shielding of the traffic signal lamp, the method for acquiring the semantics of the target single lamp comprises the following steps:
Acquiring a corresponding relation between a preset lamp cap shape and semantics;
and acquiring the semantics of the target single lamp according to the corresponding relation between the lamp cap shape of the target single lamp and the corresponding relation.
In one technical scheme of the shielding recognition method of the traffic signal lamp, the method further comprises the step of updating the color of the single lamp in the shielding state by the following steps:
Acquiring a normal color of the single lamp in a normal state before the single lamp is determined to be in a shielding state;
Setting the color of the single lamp to be the normal color and maintaining the normal color for a preset time period;
after the preset duration is maintained, if the state of the single lamp is still in a shielding state, setting the color of the single lamp to be an unknown color.
In one technical solution of the above method for identifying shielding of a traffic signal lamp, the determining, according to the first detection frame and the second detection frame, the type of the image frame relative to the single lamp includes:
acquiring the overlapping degree between the first detection frame and the second detection frame;
And if the overlapping degree is larger than a set threshold value, the type of the image frame relative to the single lamp is the shielding frame, otherwise, the type of the image frame relative to the single lamp is the visual frame.
In one technical scheme of the shielding recognition method of the traffic signal lamp, the method further comprises the following steps:
acquiring the color of the single lamp in the image frame;
And if the overlapping degree is larger than a set threshold value and the color is a preset non-effective color, the type is the occlusion frame, otherwise, the type is the visual frame.
In one technical scheme of the method for identifying shielding of a traffic signal lamp, the acquiring the overlapping degree between the first detection frame and the second detection frame includes:
Performing outer expansion on the second detection frame to obtain an outer expansion detection frame;
and acquiring the overlapping degree between the first detection frame and the expansion detection frame.
In one technical scheme of the method for identifying shielding of a traffic signal lamp, the acquiring the overlapping degree between the first detection frame and the second detection frame includes:
acquiring the cross-over ratio between the first detection frame and the second detection frame;
and acquiring the overlapping degree according to the intersection ratio.
In a second aspect, a computer readable storage medium is provided, in which a plurality of program codes are stored, the program codes being adapted to be loaded and run by a processor to perform the method according to any one of the above-mentioned aspects of the traffic light obstruction recognition method.
In a third aspect, a smart device is provided, the smart device comprising at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores a computer program, and the computer program when executed by the at least one processor implements the method according to any one of the above-mentioned methods for identifying traffic light shielding.
The technical scheme provided by the application has at least one or more of the following beneficial effects:
In the technical scheme of implementing the shielding recognition method of the traffic signal lamp, a first detection frame of a single lamp in an image frame of the traffic signal lamp and a second detection frame of an obstacle in the image frame can be obtained, and according to the first detection frame and the second detection frame, the type of the image frame relative to the single lamp is determined, wherein the type comprises a shielding frame and a visible frame, the shielding frame indicates that the single lamp is shielded by the obstacle, and the visible frame indicates that the single lamp is not shielded by the obstacle; when the shielding accumulated time of the single lamp is larger than a first time threshold, determining that the state of the single lamp is a shielding state, wherein the shielding accumulated time is the time when an image frame of the traffic signal lamp is continuously determined as a shielding frame relative to the type of the single lamp; after the single lamp is determined to be in the shielding state, if the visible accumulated time of the single lamp is larger than the second duration threshold value, the state of the single lamp is modified from the shielding state to the normal state, otherwise, the shielding state is continuously maintained, and the visible accumulated time is the time when the image frame of the traffic signal lamp is continuously determined to be the visible frame relative to the type of the single lamp.
When the obstacle is a dynamic obstacle (such as a vehicle), the relative position relationship between the obstacle and the traffic signal lamp in the process of passing the traffic signal lamp can be as follows: the obstacle does not block the single lamp (stage 1), the obstacle blocks a part of the area of the single lamp (stage 2), the obstacle blocks all the area of the single lamp (stage 3), the obstacle blocks a part of the area of the single lamp (stage 4), and the obstacle does not block the single lamp (stage 5). Based on the above embodiment, it can be determined that the single lamp is in the shielding state in the stage 2, and after entering the stage 3, the first detection frame of the single lamp cannot be detected from the image because the single lamp is completely shielded. However, in the above embodiment, the state of the single lamp is changed from the blocking state to the normal state only when the visible cumulative time of the single lamp is greater than the second time threshold, and therefore, even if the first detection frame of the single lamp cannot be detected from the image, the state of the single lamp is maintained in the blocking state in the stage 3. Therefore, whether the partial area is blocked or the whole area is blocked, the single lamp can be determined to be in a blocking state, and the embodiment does not need to use a high-precision map, so that the defect that whether the traffic signal lamp is blocked by a dynamic obstacle or not can be accurately judged only by using the high-precision map in the prior art is overcome.
Drawings
The present disclosure will become more readily understood with reference to the accompanying drawings. As will be readily appreciated by those skilled in the art: the drawings are for illustrative purposes only and are not intended to limit the scope of the present application. Wherein:
FIG. 1 is a flow chart of the main steps of a method for identifying occlusion of a traffic signal according to one embodiment of the present application;
FIG. 2 is a schematic diagram of the positional relationship between a single light A and a dynamic obstacle according to one embodiment of the application;
FIG. 3 is a second schematic diagram of the positional relationship between a single light A and a dynamic obstacle according to one embodiment of the application;
FIG. 4 is a flow chart of the main steps for obtaining the status of a traffic signal under a plurality of single lights of the same semantic meaning when the traffic signal has the single lights according to one embodiment of the present application;
FIG. 5 is a flow chart of the main steps of updating the color of a single lamp of an occlusion state according to one embodiment of the present application;
FIG. 6 is a flow chart of the main steps of a method for identifying occlusion of a traffic signal according to another embodiment of the present application;
Fig. 7 is a schematic diagram of the main structure of a smart device according to an embodiment of the present application.
Reference numerals:
11: a memory; 12: a processor.
Detailed Description
Some embodiments of the application are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are merely for explaining the technical principles of the present application, and are not intended to limit the scope of the present application.
In the description of the present application, a "processor" may include hardware, software, or a combination of both. The processor may be a central processor, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and/or signal processing functions. The processor may be implemented in software, hardware, or a combination of both. The computer readable storage medium includes any suitable medium that can store program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, and the like.
The personal information of the related user possibly related in each embodiment of the application is personal information which is actively provided by the user or generated by using the product/service in the process of using the product/service and is obtained by authorization of the user, and is processed based on the reasonable purpose of the business scene according to legal, legal and necessary principles strictly according to the requirements of laws and regulations.
The personal information of the user processed by the application can be different according to specific product/service scenes, and the personal information of the user can relate to account information, equipment information, driving information, vehicle information or other related information of the user according to the specific scene of using the product/service by the user. The present application treats the user's personal information and processes it with a high diligence.
The application is very important to the safety of the personal information of the user, and adopts reasonable and feasible safety protection measures which accord with the industry standard to protect the information of the user and prevent the personal information from unauthorized access, disclosure, use, modification, damage or loss.
The following describes an embodiment of a traffic signal shielding recognition method provided by the application. Referring to fig. 1, fig. 1 is a schematic flow chart of main steps of a traffic light shielding recognition method according to an embodiment of the present application. As shown in fig. 1, the traffic light blocking identification method in the embodiment of the present application mainly includes the following steps S101 to S104.
Step S101: a first detection frame of a single lamp in an image frame of a traffic signal lamp and a second detection frame of an obstacle in the image frame are obtained.
The image frame can be obtained by image acquisition of the traffic signal lamp through a visual sensor (such as a camera) arranged on the intelligent device. The intelligent device may include a driving device, an intelligent vehicle, a robot, and the like. After the image frame of the traffic signal lamp is obtained, the image frame can be input into a preset detection model, and the single lamp of the traffic signal lamp in the image frame is detected through the detection model to obtain a first detection frame of the single lamp, namely a boundary frame of the single lamp in the image frame. In addition, the detection model can also be used for detecting the obstacle in the image frame to obtain a second detection frame of the obstacle, and the second detection frame is namely a boundary frame of the obstacle in the image frame. In this embodiment, a conventional model construction method may be used to construct the above detection model, as long as it has the capability of detecting single lights and obstacles for image frames.
The obstacle in the present embodiment may be a dynamic obstacle, and for example, the obstacle may include an obstacle that can move, for example, a vehicle, a pedestrian, or the like.
Step S102: and determining the type of the image frame relative to the single lamp according to the first detection frame and the second detection frame, wherein the type comprises an occlusion frame and a visual frame.
The occlusion frame indicates that a single light in the image frame is occluded by an obstacle in the image frame, and the visual frame indicates that a single light in the image frame is not occluded by an obstacle in the image frame.
The first detection frame can represent the boundary frame of the single lamp in the image frame, and the second detection frame can represent the boundary frame of the obstacle in the image frame, so that the position relation between the single lamp and the obstacle can be obtained according to the first detection frame and the second detection frame, whether the single lamp is blocked by the obstacle can be judged according to the position relation, and the type of the image frame relative to the single lamp can be determined according to the judging result.
A traffic signal may contain multiple individual lights and thus there may be multiple individual lights in an image frame of the traffic signal. These individual lights may be simultaneously obscured by an obstacle or only a portion may be obscured, and thus the image frames may be different or the same type with respect to each individual light. For example, one traffic signal includes single light a, single light B, single light C, and an obstacle obscures only single light a, then the image frame is an occlusion frame with respect to the type of single light a and a visual frame with respect to the type of single light B, C.
Step S103: and when the shielding accumulated time of the single lamp is larger than the first time threshold, determining that the state of the single lamp is the shielding state, wherein the shielding accumulated time is the time when the image frame of the traffic signal lamp is continuously determined as the shielding frame relative to the type of the single lamp.
If the shielding accumulated time is larger than the first time threshold, the single lamp is continuously shielded by the obstacle for a longer time, so that the single lamp can be determined to be in a shielding state. The value of the first time length threshold can be flexibly set by a person skilled in the art according to actual requirements, and the embodiment is not particularly limited.
Step S104: after the state of the single lamp is determined to be the shielding state, if the visible accumulated time of the single lamp is larger than the second duration threshold value, the state of the single lamp is modified from the shielding state to the normal state, otherwise, the shielding state is continuously maintained, and the visible accumulated time is continuously determined as the time of the visible frame relative to the type of the single lamp.
If the visible accumulated time is larger than the second duration threshold, the single lamp is indicated to last longer without being blocked by the obstacle, so that the single lamp can be determined to be in a normal state without being blocked, the blocking state of the single lamp can be interrupted, and the blocking state is modified to be in a normal state. The value of the second duration threshold may be flexibly set by a person skilled in the art according to actual needs, which is not particularly limited in this embodiment.
Based on the methods described in the steps S101 to S104, no need to use a high-precision map, no matter whether the single lamp is in a partial area or a whole area is blocked, the single lamp can be accurately identified to be in a blocking state by using an image frame, and the accuracy of blocking identification is effectively improved.
The technical effects of the present embodiment will be briefly described with reference to fig. 2 and 3. Fig. 2 is a schematic diagram of a positional relationship between the single lamp a and the dynamic obstacle, fig. 3 is a schematic diagram of a positional relationship between the single lamp a and the dynamic obstacle, fig. 2 illustrates a positional relationship between the single lamp a and the dynamic obstacle in stages 1, 2 and 3, and fig. 3 illustrates a positional relationship between the single lamp a and the dynamic obstacle in stages 4 and 5, wherein the dynamic obstacle is a vehicle.
Stage 1: the vehicle is traveling in a direction approaching the traffic light, at which stage the vehicle has not traveled to the traffic light. Taking a single lamp A in a traffic signal lamp as an example, after detecting an image frame of the traffic signal lamp, a first detection frame of the single lamp A and a second detection frame of a vehicle can be obtained, and according to the position relation between the first detection frame and the second detection frame, determining that the single lamp A is not blocked by the vehicle, namely that the current image frame is a visible frame relative to the type of the single lamp A, and recording the visible accumulated time of the single lamp A. And when the visible accumulated time of the single lamp A is larger than the second duration threshold value, determining that the single lamp A is in a normal state.
Stage 2: after the image frames of the traffic signal lamp are detected, a first detection frame of the single lamp A and a second detection frame of the vehicle can be obtained, the single lamp A is determined to be blocked by the vehicle according to the position relation between the first detection frame and the second detection frame, namely, the current image frame is a blocking frame relative to the type of the single lamp A, the blocking accumulated time of the single lamp A is recorded, and when the blocking accumulated time of the single lamp A is larger than a first time threshold, the single lamp A is determined to be in a blocking state. Further, since the first detection frame of the single lamp a can be detected, it can be determined that the type of the shielding state is that the partial area is shielded.
Stage 3: when detecting the image frame of the traffic light, the first detection frame of the single lamp a cannot be obtained because the single lamp a is completely blocked by the vehicle, but the second detection frame of the vehicle can be obtained. Since the first detection frame of the single lamp a is not obtained, the type of the current image frame with respect to the single lamp a cannot be determined. However, in the stage 2, the state of the single lamp A is determined to be the shielding state, and after the state is determined to be the shielding state, the shielding state is interrupted and modified to be the normal state only when the visible accumulated time of the single lamp is required to be larger than the second duration threshold value. Since the type of the current image frame relative to the single lamp a cannot be determined, and thus the visible accumulated time cannot be obtained, the occlusion state is not interrupted, and therefore the single lamp a is still in the occlusion state in the stage 3. Further, since the first detection frame of the single lamp a is not detected, the type of the blocking state is that the entire area is blocked.
Stage 4: the first detection frame of the single lamp A and the second detection frame of the vehicle can be detected at the same time, the single lamp A is blocked by the vehicle according to the position relation between the first detection frame and the second detection frame, namely, the current image frame is a blocking frame relative to the type of the single lamp A, and the blocking accumulated time of the single lamp A is recorded. Similar to phase 3, after determining the occlusion state, the occlusion state is interrupted when the visible cumulative time of the single light is required to be greater than the second duration threshold. Because the visible accumulated time cannot be obtained or is smaller than or equal to the second duration threshold, the shielding state is not interrupted, and therefore the single lamp A is still in the shielding state in the stage 4. Further, since the first detection frame of the single lamp a can be detected, the type of the shielding state is that a partial area is shielded.
Stage 5: in this stage, the first detection frame of the single lamp a and the second detection frame of the vehicle can be detected at the same time, the single lamp a is determined to be not shielded by the vehicle according to the position relationship between the first detection frame and the second detection frame, namely, the current image frame is a visual frame relative to the type of the single lamp a, and the visual accumulated time of the single lamp a is recorded. And when the visible accumulated time of the single lamp A is larger than the second duration threshold value, determining that the single lamp A is in a normal state.
According to the analysis, whether the single lamp is in a shielding state or not (stages 2 and 4) or all the single lamp is in a shielding state (stage 3), the single lamp can be accurately identified by utilizing the image frame, and the accuracy and convenience of shielding identification are improved.
Step S102 is further described below.
In some embodiments of the above step S102, when determining the type of the image frame with respect to the single lamp according to the first detection frame of the single lamp and the second detection frame of the obstacle, the determination may be performed by the following steps S1021 to S1024.
Step S1021: the overlapping degree between the first detection frame and the second detection frame is obtained.
Specifically, the overlapping area between the first detection frame and the second detection frame may be acquired, and the degree of overlap may be acquired according to the overlapping area. The larger the overlap area, the greater the degree of overlap.
In some embodiments, a cross-over ratio (Intersection over Union) between the first detection frame and the second detection frame may be obtained, and the degree of overlap is obtained from the cross-over ratio. For example, the overlap ratio may be directly used as the degree of overlap. By the embodiment, the overlapping degree between the first detection frame and the second detection frame can be conveniently and accurately obtained.
In some embodiments, the second detection frame may be subjected to expansion to obtain an expanded detection frame, and then the degree of overlap between the first detection frame and the expanded detection frame is obtained, and the degree of overlap is used as the degree of overlap between the first detection frame and the second detection frame.
In practical applications, the second detection frame of the obstacle may be detected inaccurately, if the size of the second detection frame is smaller than the actual boundary frame of the obstacle, the calculation of the overlapping degree may be inaccurate, and the single lamp may be blocked by the obstacle, but due to the inaccurate overlapping degree, the single lamp is misjudged to be not blocked by the obstacle. Therefore, by performing the outer expansion of the second detection frame, this can be avoided to the greatest extent possible. In addition, even if the size of the second detection frame is relatively close to the actual boundary frame of the obstacle, the second detection frame may be expanded to enlarge the second detection frame. After the expansion, the overlapping degree of the first detection frame and the second detection frame is increased, and the single lamp is possibly not shielded by the obstacle, but the shielding of the single lamp by the obstacle can be judged in advance by utilizing the increased overlapping degree, which is equivalent to the logic of shielding judgment entered in advance, and the single lamp can be timely identified to be in a shielding state.
In this embodiment, the second detection frame may be subjected to expansion by a conventional detection frame expansion method, and the detection frame expansion method is not specifically limited in this embodiment. In addition, the person skilled in the art can flexibly set the expansion range according to the actual requirement, and the present embodiment is not particularly limited.
Step S1022: judging whether the overlapping degree between the first detection frame and the second detection frame is larger than a set threshold value or not; if yes, go to step S1023; if not, go to step S1024.
Step S1023: the image frame is determined to be an occlusion frame with respect to the type of single light.
Step S1024: the image frame is determined to be a visual frame with respect to the type of single light.
If the overlapping degree is larger than the set threshold value, the overlapping area between the first detection frame and the second detection frame is larger, and the single lamp is shielded by the obstacle, so that the type of the image frame is a shielding frame; conversely, the type of image frame is a visual frame. The values of the above-mentioned setting threshold values can be flexibly set by those skilled in the art according to actual needs, and this embodiment is not particularly limited.
Based on the methods described in the steps S1021 to S1024, it can be rapidly and accurately determined whether the single lamp is blocked by the obstacle by using the overlapping degree between the first and second detection frames, so as to obtain the type of the image frame relative to the single lamp.
In some embodiments of step S102 described above, when determining the type of the image frame relative to the single lamp according to the first detection frame of the single lamp and the second detection frame of the obstacle, the determination may be performed by the following steps S1025 to S1029.
Step S1025: the overlapping degree between the first detection frame and the second detection frame is obtained.
This step is the same as the method of step S1021, and will not be described here again.
Step S1206: the color of a single lamp in an image frame is acquired.
In this embodiment, the color of the single lamp in the image frame may be detected by a preset detection model, to obtain the color of the single lamp.
Step S1027: and judging whether the overlapping degree between the first detection frame and the second detection frame is larger than a set threshold value or not, and whether the color of the single lamp is a preset non-effective color or not.
If the overlapping degree is greater than the set threshold, and the color of the single lamp is a preset inactive color, go to step S1028; otherwise, go to step S1029.
In this embodiment, which color is a preset inactive color may be set in advance. In the present embodiment, a predetermined effective color may be set in advance, and a color other than the predetermined effective color may be set as a predetermined non-effective color.
The preset effective color is a color capable of traffic guidance, and the preset non-effective color is a color not capable of traffic guidance. In some embodiments, the preset effective colors may include green, red, and yellow, the green indicating that traffic is allowed, the red indicating that traffic is forbidden, and the yellow indicating that traffic is warned; the preset inactive color may be black. If the single lamp displays an effective color, the single lamp is lightened; if the single lamp shows a non-valid color, it indicates that the single lamp is off.
Step S1028: the image frame is determined to be an occlusion frame with respect to the type of single light.
Step S1029: the image frame is determined to be a visual frame with respect to the type of single light.
If the overlapping degree is larger than the set threshold value, the overlapping area between the first detection frame and the second detection frame is larger, and the single lamp is shielded by the obstacle. Since the first detection frame can also be detected, this indicates that the single lamp is now partially blocked. In this case, if the color of the single lamp is a preset inactive color, indicating the extinction of the single lamp, traffic indication cannot be performed according to the single lamp. Thus, it may be determined that this single light in the current image frame is inactive, and the type of image frame may be determined as an occlusion frame. However, if the color of the single lamp is a preset effective color, indicating the lighting of the single lamp, traffic indication may be performed according to the single lamp. Thus, even if a partial region of this single lamp is occluded, it is still valid, and the type of image frame can be determined as a visible frame.
Based on the methods described in the steps S1025 to S1029, the degree of overlap between the first and second detection frames and the color of the single lamp can be used simultaneously to accurately obtain the type of the image frame relative to the single lamp, and compared with the method of determining the type of the image frame only according to the degree of overlap, the accuracy of the type of the image frame can be further improved.
An embodiment of the traffic light blocking recognition method is described below.
By the method described in the foregoing steps S101 to S104, the state of one single lamp can be accurately identified. When the traffic signal includes a plurality of individual lamps, the state of each individual lamp in the traffic signal can be obtained by the above method, and after the states of all individual lamps are obtained, the state of the entire traffic signal can be obtained by the following steps S105 to S106 shown in fig. 4.
Step S105: after the states of the single lamps are determined, the semantics of the target single lamps are acquired, and the target single lamps are single lamps in shielding states.
The meaning of a single light is the traffic indication direction indicated by the single light, such as straight, left turn, dropped head, etc. When the single lamp is in a normal state, the single lamp is not shielded by the obstacle, and the semantics of the single lamp can be acquired at the moment. Since the semantics of the single lamp are usually fixed and not mutated, even if the single lamp is in a shielding state, the semantics obtained in a normal state can be utilized.
Step S106: if the single lamp with the semantic meaning is in a shielding state in the plurality of single lamps included in the traffic signal lamp, determining that the state of the traffic signal lamp under the semantic meaning is in the shielding state.
When there are a plurality of single lamps with the same meaning, as long as at least one single lamp is in a normal state, traffic indication can be performed on the meaning according to the colors of the single lamps. Therefore, only when all single lamps with the same semantics are in the shielding state, the fact that traffic indication cannot be carried out on the semantics is determined, and further the condition that the traffic signal lamp is in the shielding state under the semantics is determined.
For example, if there are two single lamps with straight line semantics, if only one single lamp is in a normal state, then it can be determined whether to allow straight line according to the color of the single lamp, and if only two single lamps are in a blocking state, it cannot be determined whether to allow straight line, at this time, the state of the traffic signal lamp under straight line is in a blocking state.
Based on the methods described in the above steps S105 to S106, when the traffic signal includes a plurality of single lights with the same semantics, the state of the traffic signal under each semantics can be accurately determined.
Step S105 is further described below.
In some embodiments of step S105 described above, in acquiring the semantics of the target single lamp, the acquisition may be performed by the following steps S1051 to S1052.
Step S1051: and acquiring a corresponding relation between the preset lamp cap shape and the semantics.
The lamp cap shape refers to the lamp cap shape of a single lamp, and the lamp cap shape comprises, but is not limited to, a round cake, a straight arrow, a left-turn arrow, a right-turn arrow, a turning arrow and the like. In addition, the shape of the base can also be understood as a pattern that the individual lamps exhibit when lit. The lighting means that the color of the single lamp is a preset effective color, and the preset effective color can comprise red, green and yellow.
Different lamp cap shapes can represent different semantics, and in the embodiment, the corresponding relation between each lamp cap and the semantics can be preset. For example, the semantics of the cookie may include three traffic directions of straight, left turn, and turn around.
Those skilled in the art can flexibly set different shapes of the lamp cap according to actual requirements, and can represent different semantics, which are not particularly limited in this embodiment.
Step S1052: and acquiring the semantics of the target single lamp according to the corresponding relation between the lamp cap shape of the target single lamp and the above.
In this embodiment, the shape of the base of the single lamp in the image frame can be detected by a preset detection model, so as to obtain the shape of the base of the single lamp. For the target single lamp, the lamp cap shape of the target single lamp can be matched with the corresponding relation so as to determine the semantics corresponding to the lamp cap shape of the target single lamp, and the semantics are taken as the semantics of the target single lamp.
Based on the methods described in the above steps S1051 to S1052, the semantics of the target single lamp can be obtained conveniently and accurately by using the shape of the base of the target single lamp.
An embodiment of the traffic light blocking recognition method is described below.
By the method described in the foregoing steps S101 to S104, the state of one single lamp can be accurately identified. If the single lamp is in a normal state, the color of the single lamp in the image frame can be directly detected, and traffic indication is carried out according to the color of the single lamp. However, if the single lamp is in the blocked state, the color of the single lamp in the image frame cannot be directly detected, and for this, in some embodiments of the present application, the color of the single lamp in the blocked state may be updated through the following steps S201 to S203 shown in fig. 5.
Step S201: the normal color at the normal state before the single lamp is determined as the blocked state is acquired. This normal color is the color of the single lamp when it is lit, i.e. this normal color is a preset effective color. For example, the normal color is red.
Step S202: after the single lamp is determined to be in the blocking state, the color of the single lamp is set to the normal color and maintained for a preset period of time.
Step S203: after the preset time period is maintained, if the state of the single lamp is still in the shielding state, the color of the single lamp is set to be unknown.
In practical applications, a single lamp is usually turned on for a period of time and then turned off. Therefore, the time maintained when the single lamp is lighted can be utilized to carry out color reasoning on the single lamp in the shielding state, namely: the color of the single lamp after shielding can be the same as the color of the single lamp before shielding in a period of time. In this regard, the present embodiment sets a preset time period, and after the single lamp is determined to be in the blocking state, the color of the single lamp is kept the same as the normal color before blocking for this preset time period. After exceeding this preset time period, if the single lamp is still in the blocking state, the color of the single lamp is not kept the same as the normal color before blocking, but is set to an unknown color, that is, the color of the single lamp cannot be determined, and the color is unknown.
The value of the preset time period may be set according to the maintenance time of the single lamp on in the present embodiment. For example, the preset duration may be less than or equal to the sustain time for a single lamp to be lit.
Based on the above-mentioned methods in step S201 to step S203, the color of the single lamp can be determined as accurately as possible when the single lamp is in the shielding state, so that the state of the traffic indication direction indicated by the single lamp (such as allowing or prohibiting traffic) can be obtained accurately according to the color of the single lamp.
An embodiment of the traffic signal shielding recognition method provided by the application is briefly described again with reference to fig. 6.
Referring to fig. 6, the single lamp detection is performed on the image frame of the traffic signal lamp, so that a first detection frame of the single lamp 1, the single lamp 2 and the single lamp 3 in the traffic signal lamp can be obtained, and the obstacle detection is performed on the image frame, so that an obstacle detection frame, namely a second detection frame, can be obtained.
Taking the single lamp 1 as an example, the first detection frame of the single lamp 1 and the second detection frame of the obstacle are matched, namely: and determining the type of the image frame relative to the single lamp according to the first detection frame and the second detection frame, wherein the type comprises an occlusion frame and a visual frame. Then, according to the matching result, whether the shielding state is activated is determined, namely: judging whether the shielding accumulated time of the single lamp 1 is larger than a first time length threshold value or not; if the state of the single lamp 1 is larger than the shielding state, determining the state of the single lamp 1 as the shielding state; otherwise, the current state is maintained.
The shielding recognition process of the single lamp 2 and the single lamp 3 is similar to that of the single lamp 1, and is not repeated here. After the current image frame is subjected to shielding recognition, the state of the single lamp 1 can be determined to be a shielding state, the states of the single lamps 2 and 3 are normal states, and the single lamps 2 and 3 are both lighted single lamps.
After the states of the single lamp 1, the single lamp 2 and the single lamp 3 are determined, the states of the three single lamps are fused, and the state of the traffic signal lamp is determined.
Assume that: the shapes of the lamp caps of the single lamp 1, the single lamp 2 and the single lamp 3 are round cakes, the single lamp 2 and the single lamp 3 are red in the current image frame, the single lamp 1 is red in the previous image frame, namely, the normal color of the single lamp 1 before being blocked is red. Because the lamp caps of the three single lamps have the same shape, the semantics of the three single lamps are the same, wherein the semantics of the round cakes comprise straight movement, left rotation and turning around. That is, the traffic signal has three single lamps of the same meaning, and two single lamps are in a normal state, so the state of the traffic signal under this meaning is a normal state. If all three single lamps are in the shielding state, the state of the traffic signal lamp under the semantic meaning is the shielding state.
It should be noted that, although the foregoing embodiments describe the steps in a specific order, it will be understood by those skilled in the art that, in order to achieve the effects of the present application, the steps are not necessarily performed in such an order, and may be performed simultaneously (in parallel) or in other orders, and those solutions after these adjustments belong to equivalent solutions to those described in the present application, and therefore will also fall within the scope of the present application.
It will be appreciated by those skilled in the art that the present application may implement all or part of the above-described methods according to the above-described embodiments, or may be implemented by means of a computer program for instructing relevant hardware, where the computer program may be stored in a computer readable storage medium, and where the computer program may implement the steps of the above-described embodiments of the method when executed by a processor. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable storage medium may include: any entity or device, medium, usb disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier wave signals, telecommunications signals, software distribution media, and the like capable of carrying the computer program code. It should be noted that the computer readable storage medium may include content that is subject to appropriate increases and decreases as required by jurisdictions and by jurisdictions in which such computer readable storage medium does not include electrical carrier signals and telecommunications signals.
Another aspect of the application also provides a computer-readable storage medium.
In an embodiment of a computer readable storage medium according to the present application, the computer readable storage medium may be configured to store a program for performing the traffic light obstruction identification method of the above-described method embodiment, which program may be loaded and executed by a processor to implement the traffic light obstruction identification method described above. For convenience of explanation, only those portions of the embodiments of the present application that are relevant to the embodiments of the present application are shown, and specific technical details are not disclosed, please refer to the method portions of the embodiments of the present application. The computer readable storage medium may be a storage device including various electronic devices, and optionally, the computer readable storage medium in the embodiments of the present application is a non-transitory computer readable storage medium.
The application further provides intelligent equipment.
In an embodiment of a smart device according to the application, the smart device may comprise at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores a computer program which, when executed by the at least one processor, implements the method of any of the embodiments described above. The intelligent equipment provided by the application can comprise driving equipment, intelligent vehicles, robots and other equipment. Referring to fig. 7, memory 11 and processor 12 are illustratively shown in fig. 7 as being communicatively coupled via a bus.
In some embodiments of the application, the smart device may further comprise at least one sensor for sensing information. The sensor is communicatively coupled to any of the types of processors referred to herein. Optionally, the smart device may further comprise an autopilot system for guiding the smart device to drive itself or assist in driving. The processor communicates with the sensors and/or the autopilot system for performing the method of any one of the embodiments described above.
Thus far, the technical solution of the present application has been described in connection with one embodiment shown in the drawings, but it is easily understood by those skilled in the art that the scope of protection of the present application is not limited to these specific embodiments. Equivalent modifications and substitutions for related technical features may be made by those skilled in the art without departing from the principles of the present application, and such modifications and substitutions will fall within the scope of the present application.
Claims (10)
1. A method for identifying occlusion of a traffic signal, the method comprising:
Acquiring a first detection frame of a single lamp in an image frame of the traffic signal lamp and a second detection frame of an obstacle in the image frame, wherein the image frame is obtained by acquiring images of the traffic signal lamp through a vision sensor arranged on intelligent equipment, and the obstacle is a dynamic obstacle;
Determining the type of the image frame relative to the single lamp according to the first detection frame and the second detection frame, wherein the type comprises an shielding frame and a visual frame, the shielding frame indicates that the single lamp is shielded by the obstacle, and the visual frame indicates that the single lamp is not shielded by the obstacle;
When the shielding accumulated time of the single lamp is larger than a first time threshold, determining that the state of the single lamp is a shielding state, wherein the shielding accumulated time is the time when an image frame of the traffic signal lamp is continuously determined to be the shielding frame relative to the type of the single lamp;
After determining the shielding state, if the visible accumulated time of the single lamp is larger than a second duration threshold value, modifying the state of the single lamp from the shielding state to a normal state, otherwise, maintaining the shielding state, wherein the visible accumulated time is the time when the image frame of the traffic signal lamp is continuously determined as the visible frame relative to the type of the single lamp.
2. The method of claim 1, wherein when the traffic signal comprises a plurality of individual lights, the method further comprises:
after determining the state of each single lamp, acquiring the semantic meaning of a target single lamp, wherein the target single lamp is a single lamp in a shielding state;
And if the single lamps with the semantics in the plurality of single lamps are all in the shielding state, determining that the state of the traffic signal lamp under the semantics is the shielding state.
3. The method of claim 2, wherein the obtaining the semantics of the target single lamp comprises:
Acquiring a corresponding relation between a preset lamp cap shape and semantics;
and acquiring the semantics of the target single lamp according to the corresponding relation between the lamp cap shape of the target single lamp and the corresponding relation.
4. The method of claim 1, further comprising updating the color of the single light of the occlusion state by:
Acquiring a normal color of the single lamp in a normal state before the single lamp is determined to be in a shielding state;
Setting the color of the single lamp to be the normal color and maintaining the normal color for a preset time period;
after the preset duration is maintained, if the state of the single lamp is still in a shielding state, setting the color of the single lamp to be an unknown color.
5. The method of claim 1, wherein determining the type of the image frame relative to the single light based on the first detection frame and the second detection frame comprises:
acquiring the overlapping degree between the first detection frame and the second detection frame;
And if the overlapping degree is larger than a set threshold value, the type of the image frame relative to the single lamp is the shielding frame, otherwise, the type of the image frame relative to the single lamp is the visual frame.
6. The method of claim 5, wherein the method further comprises:
acquiring the color of the single lamp in the image frame;
And if the overlapping degree is larger than a set threshold value and the color is a preset non-effective color, the type is the occlusion frame, otherwise, the type is the visual frame.
7. The method of claim 5, wherein the obtaining the degree of overlap between the first detection frame and the second detection frame comprises:
Performing outer expansion on the second detection frame to obtain an outer expansion detection frame;
and acquiring the overlapping degree between the first detection frame and the expansion detection frame.
8. The method of claim 5, wherein the obtaining the degree of overlap between the first detection frame and the second detection frame comprises:
acquiring the cross-over ratio between the first detection frame and the second detection frame;
and acquiring the overlapping degree according to the intersection ratio.
9. A computer readable storage medium, in which a plurality of program codes are stored, characterized in that the program codes are adapted to be loaded and run by a processor to perform the method of identifying occlusion of a traffic light according to any of claims 1 to 8.
10. An intelligent device, comprising:
at least one processor;
And a memory communicatively coupled to the at least one processor;
Wherein the memory has stored therein a computer program which, when executed by the at least one processor, implements the method of identifying occlusion of a traffic light of any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410282328.0A CN117935225B (en) | 2024-03-13 | 2024-03-13 | Shielding identification method of traffic signal lamp, intelligent equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410282328.0A CN117935225B (en) | 2024-03-13 | 2024-03-13 | Shielding identification method of traffic signal lamp, intelligent equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117935225A CN117935225A (en) | 2024-04-26 |
| CN117935225B true CN117935225B (en) | 2024-06-21 |
Family
ID=90757800
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410282328.0A Active CN117935225B (en) | 2024-03-13 | 2024-03-13 | Shielding identification method of traffic signal lamp, intelligent equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117935225B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113343873A (en) * | 2021-06-17 | 2021-09-03 | 湖北亿咖通科技有限公司 | Signal lamp identification method, device, equipment, medium and product |
| CN115578712A (en) * | 2022-09-09 | 2023-01-06 | 阿波罗智联(北京)科技有限公司 | Image stabilizing method and device, electronic equipment and storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110390833B (en) * | 2019-06-03 | 2020-12-29 | 杭州电子科技大学 | Traffic indicating facility shielding detection method |
| CN112949661B (en) * | 2021-05-13 | 2021-08-06 | 北京世纪好未来教育科技有限公司 | Detection frame self-adaptive external expansion method and device, electronic equipment and storage medium |
| CN113822146A (en) * | 2021-08-02 | 2021-12-21 | 浙江大华技术股份有限公司 | Target detection method, terminal device and computer storage medium |
| CN117078919A (en) * | 2023-09-06 | 2023-11-17 | 浙江大华技术股份有限公司 | Method and device for expanding detection frame |
-
2024
- 2024-03-13 CN CN202410282328.0A patent/CN117935225B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113343873A (en) * | 2021-06-17 | 2021-09-03 | 湖北亿咖通科技有限公司 | Signal lamp identification method, device, equipment, medium and product |
| CN115578712A (en) * | 2022-09-09 | 2023-01-06 | 阿波罗智联(北京)科技有限公司 | Image stabilizing method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117935225A (en) | 2024-04-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111212772B (en) | Method and device for determining a driving strategy of a vehicle | |
| TWI493513B (en) | Lane departure warning system and lane identification apparatus and related method thereof | |
| US9349070B2 (en) | Vehicle external environment recognition device | |
| JP5100159B2 (en) | Automatic switching control method for in-vehicle projector | |
| CN101908271B (en) | Signal lamp color identification method and device | |
| US20150039212A1 (en) | Vehicle exterior environment recognition device | |
| CN111815959A (en) | Vehicle violation detection method and device and computer readable storage medium | |
| US20150091715A1 (en) | Vehicle external environment recognition device | |
| CN108765945A (en) | A kind of vehicle peccancy recognition methods and system | |
| CN108407806A (en) | Auxiliary driving method and device | |
| TWI549102B (en) | Lane departure warning system and lane identification apparatus and related method thereof | |
| SE541589C2 (en) | Method and system for controlling driving of a vehicle along a road | |
| CN110634324A (en) | Vehicle-mounted terminal based reminding method and system for courtesy pedestrians and vehicle-mounted terminal | |
| CN111967384A (en) | Vehicle information processing method, device, equipment and computer readable storage medium | |
| CN117935225B (en) | Shielding identification method of traffic signal lamp, intelligent equipment and storage medium | |
| KR100355993B1 (en) | An apparatus for warning lane deviation and a method for warning the same | |
| CN115690441A (en) | Traffic signal lamp area detection method, device and equipment | |
| EP3287940A1 (en) | Intersection detection system for a vehicle | |
| CN111066024A (en) | Method and device for recognizing lane, driver assistance system and vehicle | |
| CN114494938A (en) | Non-motor vehicle behavior identification method and related device | |
| CN114359233A (en) | Image segmentation model training method and device, electronic equipment and readable storage medium | |
| JP6378594B2 (en) | Outside environment recognition device | |
| CN117475411B (en) | Signal lamp countdown identification method, computer readable storage medium and intelligent device | |
| CN112970030A (en) | Method and device for detecting traffic light | |
| CN118230287A (en) | Fault identification method of traffic signal lamp, intelligent equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |