WO2020098004A1 - Lane traffic status reminder method and device - Google Patents

Abstract

Provided in the present application are a lane traffic status reminder method and device. The method comprises: acquiring a front-view image of a vehicle, and extracting image features of the front-view image; identifying lane information and signal light information in the front-view image based on the image features; determining the current traffic status of the vehicle according to the lane information and the signal light information; and generating reminder information according to the traffic status, and displaying the reminder information. By means of the front-view image obtained by the device, the present application can, according to the front-view image, intelligently identify and remind a driver whether the vehicle can pass safely at the intersection, thereby improving safety while driving.

Description

Target tracking processing method and electronic equipment Technical field

The invention relates to the field of data processing, in particular to a target tracking processing method and electronic equipment.

Background technique

The identification and tracking of targets (vehicles, pedestrians, etc.) are the key technologies for various ADAS applications. Among them, target recognition is responsible for detecting the types and positions of various targets in each frame of the scene, and tracking is to associate the detected targets in the two frames before and after, giving an identity, and the detection and tracking cooperation can also be more accurate Estimate the target location.

However, the existing related algorithms have a large amount of calculation in the process of simultaneously identifying and tracking the target, which makes the data processing efficiency of the device low and cannot deal well with complex scenes.

Summary of the invention

The invention provides a target tracking processing method, which can improve the data processing efficiency of the device.

An embodiment of the present invention provides a target tracking processing method, which is applied to an electronic device including at least three processing units. The method includes:

Through the first processing unit, acquire a plurality of corner points of the target image, track the corresponding positions of the corner points in the target image in multiple frames, and obtain tracking data of the corner points;

The second processing unit detects the target object in the target image and determines the image range corresponding to the target object;

The third processing unit determines the target range of the target object according to the tracking data of the corner point and the image range of the target object.

Optionally, the acquiring multiple corner points of the target image and tracking the corresponding positions of the corner points in the multi-frame target image to obtain the tracking data of the corner points include:

Acquiring a preset area of the target image;

Construct an image pyramid of the preset area;

Determine multiple corner points according to the image pyramid;

Tracking the corresponding position of the corner point in the target image in multiple frames to obtain tracking data of the corner point.

Optionally, the acquiring multiple corner points of the target image and tracking the corresponding positions of the corner points in the target image in multiple frames includes:

Perform forward pyramid LK tracking and reverse pyramid LK tracking on the corner points respectively to obtain preliminary tracking data;

Perform cross-validation on the preliminary tracking data to obtain tracking data of corner points that satisfy preset conditions among the plurality of corner points.

Optionally, the determining the target range of the target object according to the tracking data of the corner point and the image range of the target object includes:

Determine the target corner point corresponding to the target object according to the tracking data of the corner point and the image range of the target object; determine the target according to the tracking data of the target corner point and the image range of the target object The target range of the object.

Optionally, the tracking data of the corner point includes a tracking range corresponding to each corner point in the corner point set;

The determining the target range of the target object according to the tracking data of the target corner point and the image range of the target object includes:

Determining the confidence of the tracking range of the corner point and the image range of the target object;

According to the confidence of the tracking range and the image range, the target range of the target object is determined according to a preset weighting value, respectively.

Optionally, the determining the target corner point corresponding to the target object according to the tracking data of the corner point and the image range of the target object includes:

Acquiring a preset mapping relationship between the target image and the image range;

According to the mapping relationship, determine a plurality of mapped pixels of the image range in the target image;

Determining the pixel range enclosed by the image range in the target image according to the mapped pixels;

The corner point located within the pixel range is taken as the target corner point.

Optionally, before acquiring the preset mapping relationship between the target image and the image range, the method further includes: constructing, by the first processing unit, pixels in the target image and the image range Mapping relationship.

Optionally, the first processing unit is a vector operation processor.

The invention also provides an electronic device. The electronic device includes at least three processing units, wherein:

The first processing unit is configured to acquire multiple corner points of the target image, track the corresponding positions of the corner points in the target image in multiple frames, and obtain tracking data of the corner points;

The second processing unit is configured to detect a target object in the target image and determine an image range corresponding to the target object;

The third processing unit is configured to determine the target range of the target object according to the tracking data of the corner point and the image range of the target object.

Optionally, the first processing unit is a vector operation processor.

It can be seen from the above that by allocating tracking data and image range recognition to different processing units for processing and fusing them to obtain the target range of the target object, the performance of the multi-core processor can be effectively used to improve the data processing efficiency and better Cope with complex scenes.

BRIEF DESCRIPTION

FIG. 1 is an implementation flowchart of a target tracking processing method provided by an embodiment of the present invention.

FIG. 2 is a flowchart of implementing corner tracking data provided by an embodiment of the present invention.

FIG. 3 is a flowchart of determining a target range provided by an embodiment of the present invention.

FIG. 4 is a flowchart of determining a target corner provided by an embodiment of the present invention.

FIG. 5 is a structural frame diagram of an electronic device provided by an embodiment of the present invention.

detailed description

The following describes the preferred embodiments of the present invention in detail with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, so as to make the protection scope of the present invention more clearly defined.

Please refer to FIG. 1, which shows an implementation process of a target tracking processing method provided by an embodiment of the present invention.

As shown in FIG. 1, a target tracking processing method is applied to an electronic device. The electronic device includes at least three processing units. The method includes:

101. Through the first processing unit, acquire a plurality of corner points of the target image, track the corresponding positions of the corner points in the multi-frame target image, and obtain tracking data of the corner points.

The corner point may be a feature point, that is, an image part with certain characteristics in the image, such as a person's face or hand in the image, which can be set as required.

Among them, the target image may be acquired by an image sensor of an electronic device, such as a camera.

In some embodiments, multiple corner points of the target image are obtained, and feature parts in the image can be extracted through a preset algorithm, and these feature parts are matched through a database to determine the corner point positions. For example, the corner point is identified by the fast9 corner detection algorithm.

In some embodiments, tracking the corresponding position of the corner point in the multi-frame target image to obtain the tracking data of the corner point, first of all, it is possible to determine the position of a corner point in the target image of a frame (such as the first frame), and then pass The preset target tracking algorithm tracks the change of the relative position of the corner point in other frames to obtain the tracking data of the corner point. Wherein, the tracking data of the corner point may be parameters such as the movement trajectory and relative displacement distance of the corner point.

Alternatively, the target tracking algorithm may use a pyramid LK optical flow tracking algorithm, by acquiring the image pyramid of the target image, and then tracking the optical flow of the corner points in the image pyramid through the pyramid LK optical flow tracking algorithm to track the movement of the corner points.

This step may be implemented in the first processing unit of the electronic device, and the first processing unit may be a processing unit for vector operations, such as a full-time vector operation processor for vector operations, such as a GPU (Graphics Processing Unit, visual processor) ). Because the tracking algorithm is relatively simple, but there are a large number of pixel-level operations, so when the 101 step is executed by the vector operation processor, the operation efficiency can be greatly improved. Of course, in addition to vector operation processors, this step can also use CPU (Central Processing Unit, core processor) or DSP (Digital Signal Processing, digital signal processor) and other devices to run calculations.

102. Through the second processing unit, detect and identify the target object in the target image, and determine the image range corresponding to the target object.

The target object may be one or more, such as a human body, a vehicle, or other obstacles. Identifying the target object can be achieved by a preset recognition algorithm, such as a neural network algorithm, etc. The specific recognition algorithm can be selected according to the actual situation.

In some embodiments, to determine the image range corresponding to the target object, the target object can be marked in advance through a recognition algorithm, and the outline range of the marked object or the relative position in the target image can be framed with a detection frame To determine the image range corresponding to the target object.

This step may be implemented in a second processing unit of the electronic device, the second processing unit and the first processing unit are relatively independent of each other, so that step 102 and step 101 are in different process states, and the second processing unit may be General-purpose computing processor, such as CPU (Central Processing Unit, core processor) or DSP (Digital Signal Processing, digital signal processor) and other devices to run calculations

103. The third processing unit determines the target range of the target object according to the tracking data of the corner points and the image range of the target object.

The target range may be the range occupied by the target object in the image. For example, if the target range is a human body, the target range frames the maximum width and maximum height of the human body.

In some embodiments, through the tracking data of the corner points, the displacement of each corner point in the target image can be obtained. Combining the detected image range of the target object with the displacement of each corner point, the corner points in the image range can be selected through the image range frame of the target object, and then the corner points of the target object in the image range The displacement situation.

Then, the image range of the target object and the displacement of the corner points within the target object range are weighted to obtain the target range of the target object.

This step may be implemented in the third processing unit of the electronic device, the third processing unit is relatively independent of the second processing unit and the first processing unit, so that the step 103, the step 102 and the step 101 are in different process states To distribute tasks to different cores to improve data processing efficiency. Of course, the third processing unit and the second processing unit may also be the same processing unit.

In the prior art, if it is necessary to obtain the corner points within the range of the target object, it is necessary to separately construct image pyramids for different target objects, and combine the image range of the target object for corner recognition, so that the algorithm needs to construct the target multiple times Image pyramid of images. If a single thread is used to identify the target object, then the image pyramid of the target image is constructed, and the tracking data is obtained through the image pyramid, the time complexity is O (N * Log (M) * (M * n ^ 2 + n ^ 3)), where N is the number of detected target objects, M is the average size of the target objects, and n is generally 2. From this, we can find the relationship of the product, so when there are many objects in the image, the pyramid construction time-consuming has a greater impact on the overall time complexity.

However, when the above steps 101-103 are executed by the multi-core processing unit, only an image pyramid needs to be constructed once for the target image, and the overall corner recognition of the image pyramid of the target image is performed, using the image range frame of the identified target object Select the desired corner point for tracking, and the time complexity can be O (Log (W) * (W * n ^ 2 + n ^ 3)) + O (W), where N is detected The number of target objects, M is the average size of the target object, n is generally 2, W is the size of the ROI area, it can be seen that the time complexity is reduced, and the data processing efficiency is greatly improved.

It can be seen from the above that by allocating tracking data and image range recognition to different processing units for processing and fusing them to obtain the target range of the target object, the performance of the multi-core processor can be effectively used to improve the data processing efficiency and better Cope with complex scenes.

Please refer to FIG. 2, which shows an implementation process of obtaining corner tracking data provided by an embodiment of the present application.

As shown in FIG. 2, the acquiring multiple corner points of the target image, tracking the corresponding positions of the corner points in the target image in multiple frames, and obtaining the tracking data of the corner points include:

201. Acquire a preset area of a target image.

202. Construct an image pyramid of a preset area.

203. Determine multiple corner points according to the image pyramid.

204. Track the corresponding position of the corner point in the multi-frame target image to obtain the tracking data of the corner point.

Wherein, the preset area may be a certain area or the entire area in the target image manually preset to reduce redundant data of the target image and improve algorithm efficiency. For example, the preset area may be an image area that is two-thirds of the height / width in the middle of the target image. Or by identifying the possible moving direction of the host vehicle in the target image, and determining the position of the preset area in the target image according to the moving direction. Of course, in addition to the above preset method, the size, position, etc. of the area can also be set according to actual needs.

In the process of constructing an image pyramid, an image pyramid of 4 or more layers can be constructed for the target image, and the corner points of the images in these pyramids are extracted by an algorithm to obtain the position of the corner point. Then track the corresponding positions of the above corner points in the multi-frame target image to obtain the tracking data of the corner points.

Among them, the tracking algorithm may be a fast9 corner tracking algorithm to provide more accurate tracking data. Of course, other tracking algorithms can be used in addition.

In some embodiments, tracking the corresponding position of the corner point in the multi-frame target image to obtain the tracking data of the corner point includes:

The forward pyramid LK tracking and the reverse pyramid LK tracking are performed on the corner points respectively to obtain preliminary tracking data; the preliminary tracking data is cross-validated to obtain the tracking data of corner points satisfying preset conditions among multiple corner points.

For example, if the detection loss of a corner lasts for several frames, the tracking is considered invalid and the relevant parameters of the corner are deleted.

The above method can greatly improve the accuracy of tracking data of corners. By cross-validating preliminary tracking data, corners with poor tracking effects can be removed, which further reduces waste of resources and improves data processing efficiency.

In some embodiments, please refer to FIG. 3, which shows an implementation process of determining a target range provided by an embodiment of the present application.

As shown in FIG. 3, the determination of the target range of the target object based on the tracking data of the corner points and the image range of the target object includes:

301. Determine the target corner corresponding to the target object according to the tracking data of the corner and the image range of the target object.

By combining the tracking data of all corner points in the target image with the image range of the target object, the corner points in the image range of the target object can be determined, and the corner points in the image range are defined as the target corner points.

For example, the image range A of the target object is acquired, and if the target image has a corner point B and a corner point C within the image range A, the corner points B and C can be used as the target corner point. If there are image ranges of N target objects, the image ranges of the N target objects may be associated with corner points in the corresponding ranges.

302. Determine the target range of the target object according to the tracking data of the target corner point and the image range of the target object.

After obtaining the target corner, you can determine the more accurate movement and shape change of the target object (such as vehicle turning, sideways walking, etc.) according to the tracking of the corner and the position change of the image range, and then determine the target object Target range.

Specifically, the target object can be selected by a tracking frame representing the target range. The displacement value of the tracking frame can be obtained from the average displacement of the corner coordinates of the next frame and the corner coordinates of the previous frame, and the size change of the tracking frame can be obtained from the average distance between the target corner points of the next frame.

For example, if the image range A is the approximate position of the human body, and the corner points B and C are the characteristic points of the left and right hands of the human body, respectively, the position of the image range A can be combined with the position change of the corner points B and C to use The width represents the current posture and movement of the human body. The length / width of the tracking frame can be changed according to the change of the posture and movement of the human body.

Based on the image range of the identified target object, the desired corner point is selected as the target corner point, and the target corner point is tracked, so that the device does not need to perform image range recognition multiple times and diagonal points within the image range The recognition action greatly reduces the time complexity of this embodiment and improves the data processing efficiency.

In some embodiments, in order to improve the accuracy of the target range, step 302 may include:

Determine the confidence of the tracking range of the corner point and the image range of the target object; according to the confidence of the tracking range and the image range, determine the target range of the target object according to the preset weighting value.

Among them, the confidence can be obtained by conventional algorithms. The tracking range of the corner point is the approximate area that the corner point occupies in the target image. The approximate area can be framed by the tracking frame.

Specifically, if the confidence of the tracking range of the corner point is high and the confidence of the image range of the target object is low, the tracking range of the corner point may be used as a basis to determine the target range of the target object in combination with the preset weighting value; If the confidence of the tracking range of the corner point is low, and the confidence of the image range of the target object is high, the target range of the target object can be determined in combination with the preset weighting value based on the image range of the target object.

Understandably, the specific weighting method can be determined according to the actual situation.

Combining the tracking data of the target corner points with the image range of the target object, you can use the image range frame to select the desired corner point, without having to frame each target object and then perform complicated corner point recognition, making the embodiment The time complexity is the linear time complexity, which improves the operation efficiency.

Please refer to FIG. 4, which shows an implementation process of determining a target corner point provided by an embodiment of the present application.

As shown in FIG. 4, the determination of the target corner corresponding to the target object based on the tracking data of the corner point and the image range of the target object includes:

401. Acquire a preset mapping relationship between the target image and the image range.

402. According to the mapping relationship, determine a plurality of mapped pixels whose image range is in the target image.

403. Determine the pixel range enclosed by the image range in the target image according to the mapped pixels.

404. Use the corner points within the pixel range as the target corner points.

The image range can be a parameter representing the relative position, including the approximate outline range of the target object. By establishing a mapping relationship between the image range and the pixel position of the target image, the outline or relative position of the target object can be determined in the target image The pixel location being mapped.

When the mapped pixels corresponding to the image range are obtained, the pixel range enclosed by the image range in the target image can be obtained therefrom, so as to better determine the position of the target object in the target image, and then use the target object in the target image Select the corresponding corner point as the target corner point in the rough outline range box in to achieve the combination of the image range and the tracking data of the corner point to obtain the tracking data of the corner point in the image range of the target object. The embodiment in FIG. 4 quickly determines the corner points within the image range through the mapping relationship, so that only a small amount of resources is required to obtain a better tracking effect on the target object and improve the calculation efficiency.

In some embodiments, in order to further improve efficiency, before the preset mapping relationship between the acquisition target image and the image range, the method further includes:

Through the first processing unit, a mapping relationship between pixels in the target image and the image range is constructed. The first processing unit is a vector operation processor, and the mapping relationship between the pixels in the target image and the image range is constructed by the vector operation processor, and the processing efficiency can be greatly improved by taking advantage of the parallel operation.

Please refer to FIG. 5, which shows a structural framework of an electronic device provided by an embodiment of the present application.

As shown in FIG. 5, the electronic device 50 includes at least three processing units, wherein:

The first processing unit 51 is configured to acquire multiple corner points of the target image, track the corresponding positions of the corner points in the multi-frame target image, and obtain tracking data of the corner points;

The second processing unit 52 is used to detect the target object in the target image and determine the image range corresponding to the target object;

The third processing unit 53 is used to determine the target range of the target object based on the tracking data of the corner point and the image range of the target object.

In the electronic device 50, the electronic device 50 may be an in-vehicle electronic device 50, such as an ADAS (Advanced Driver Assistance System, advanced driving assistance system). The electronic device 50 may also include a memory. The processing unit is electrically connected to the memory.

The processing unit is the control center of the electronic device 50, which uses various interfaces and lines to connect the various parts of the entire electronic device 50, executes the electronic device 50 by running or loading computer programs stored in the memory, and calling data stored in the memory Various functions and processing data to monitor the electronic device 50 as a whole.

In some embodiments, the first processing unit 51 is a vector operation processor.

In this embodiment, the processor unit in the electronic device 50 will load the instructions corresponding to the process of one or more computer programs into the memory according to the following steps, and the processing unit runs the computer stored in the memory Program to achieve various functions, such as:

Through the first processing unit 51, acquire a plurality of corner points of the target image, and track the corresponding positions of the corner points in the target image in multiple frames to obtain tracking data of the corner points; through the second processing Unit 52, detecting and identifying the target object in the target image to determine the image range corresponding to the target object; through the third processing unit 53, according to the tracking data of the corner point and the image range of the target object, Determine the target range of the target object.

In some embodiments, a storage medium is also provided. The storage medium stores a plurality of instructions. The instructions are suitable to be loaded by the processing unit to perform any of the above target tracking processing methods, for example:

Through the first processing unit 51, acquire a plurality of corner points of the target image, and track the corresponding positions of the corner points in the target image in multiple frames to obtain tracking data of the corner points; through the second processing Unit 52, detecting and identifying the target object in the target image to determine the image range corresponding to the target object; through the third processing unit 53, according to the tracking data of the corner point and the image range of the target object, Determine the target range of the target object.

A person of ordinary skill in the art may understand that all or part of the steps in the various methods of the foregoing embodiments may be completed by instructing relevant hardware through a program. The program may be stored in a computer-readable storage medium, and the storage medium may include: Read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.

In the embodiment of the present application, the electronic device and the target tracking processing method in the above embodiment belong to the same concept, and any method steps provided in the target tracking processing method embodiment may be run on the electronic device For the specific implementation process, please refer to the embodiment of the target tracking processing method, and any combination can be used to form an optional embodiment of the present application, which will not be repeated here.

The embodiments of the present invention have been described in detail above with reference to the drawings, but the present invention is not limited to the above-mentioned embodiments. Within the scope of knowledge possessed by those of ordinary skill in the art, various embodiments can be made without departing from the spirit of the present invention Kind of change.

Name of invention: Method and equipment for reminding traffic lanes

TECHNICAL FIELD [0001] The present application relates to the field of traffic data processing, and in particular, to a method and device for reminding traffic lane status.
BACKGROUND [0002] With the increasing number of cars, traffic accidents and violations become more and more frequent, most of which are at the intersection of traffic lights. Therefore, signal lights are permanently installed at intersections to reduce traffic accidents.
[0003] In order to facilitate the driver's viewing, signal lights are generally installed at intersections at high places or on the side of the road where it is easier to observe. However, if the driver's attention is not concentrated when the vehicle passes by the intersection, or the vehicle's sight is blocked due to the vehicle following behind the big car, it will often cause the driver to fail to receive the signal light information in time, which will cause great driving safety Big threat.
[0004] Application content The present application provides a lane traffic state reminding method and device, which can remind the driver of the current signal lights.
[0005] The present application provides a lane passing state reminding method, which is applied to an electronic device and includes: acquiring a forward-looking image of a vehicle, extracting image features of the forward-looking image; identifying the forward-looking image based on the image features Lane information and signal light information in; determining the current traffic state of the vehicle according to the lane information and signal light information; generating reminder information according to the traffic state, and displaying the reminder information.
[0006] Optionally, identifying lane information and signal information in the forward-looking image based on the image features includes: acquiring a lane reference feature; comparing the image feature with the lane reference feature to determine Whether there is a target feature matching the lane reference feature among the image features, wherein the target feature includes a lane direction feature; if so, extracting information corresponding to the target feature to obtain the lane information, where The lane information includes lane direction information.
[0007] Optionally, the target feature further includes a lane line feature; the image feature is compared with the lane reference feature to determine whether there is a match between the image feature and the lane reference feature The target features include: judging whether the lane line feature exists in the image feature; if the lane line feature exists, determining the lane area of the lane based on the lane line feature; judging in the lane area Whether the lane direction feature exists among the image features.
[0008] Optionally, the lane direction feature is related to the shape feature of the driving arrow of the lane.
[0009] Optionally, the signal light information includes the lane direction type of the signal light and the corresponding signal type; determining the current traffic state of the vehicle according to the lane information and the signal light information includes: determining according to the lane direction information The direction of traffic that the vehicle is about to pass; acquiring the signal type of the signal light corresponding to the direction of traffic; determining the current traffic state of the vehicle according to the signal type, where the traffic state includes a traffic allowed state and a traffic prohibited state.
[0010] Optionally, the traffic state includes a normal driving state and an attention reminding state, and the current traffic state of the vehicle is determined according to the lane information and the signal light information, including: if none of the lane information can be identified and When the signal light information, the vehicle is currently in a driving state; if the lane information cannot be identified, and the signal light information is present, the vehicle is currently in a state of attention and reminder. If the lane information is present, and cannot When the signal light information is recognized, the vehicle is currently in a driving state.
[0011] Optionally, generating reminder information according to the traffic state and displaying the reminder information includes: acquiring voice information corresponding to the traffic state, playing the voice information; The text or pattern information corresponding to the traffic state is displayed on the text or pattern information; or a level signal corresponding to the traffic state is acquired, and the preset indicator is turned on and off through the level signal.
[0012] The present application also provides a lane passing state reminding device, including a forward-looking camera, a processing circuit electrically connected to the forward-looking camera, and a reminding module electrically connected to the processing circuit, wherein: the forward-looking camera , For acquiring the forward-looking image of the vehicle; the processing circuit, for extracting image features of the forward-looking image; identifying lane information and signal information in the forward-looking image based on the image features; according to the Lane information and signal light information determine the current traffic state of the vehicle; generate reminder information according to the traffic state; and the reminder module, which is used to display the reminder information.
[0013] Optionally, the processing circuit is specifically configured to: obtain a lane reference feature; compare the image feature with the lane reference feature, and determine whether the image feature exists with the A target feature matching a lane reference feature, where the target feature includes a lane line feature and a lane direction feature; if so, extract information corresponding to the target feature to obtain the lane information, wherein the lane information includes the lane direction information.
[0014] Optionally, the reminder module includes one of a sound generator, a display, or an indicator light; the sound generator is used to play voice information, and the voice information corresponds to the traffic state; A display for displaying text or pattern information, the text or pattern information corresponding to the traffic state;
The indicator light is used to turn on and off under the control of the level signal of the reminder module, and the level signal corresponds to the traffic state.
[0015] As can be seen from the above, the image features in the forward-looking image are extracted, and lane information and signal information are identified based on the image features, and the current traffic state of the vehicle is determined based on the lane information and signal information, and reminder information is generated according to the traffic state And show it. In this application, the forward-looking image obtained by the device can be intelligently recognized at the signal lamp intersection according to the forward-looking image and remind the driver whether the vehicle can pass safely at this time, thereby improving the safety during driving.
BRIEF DESCRIPTION OF THE DRAWINGS [0016] FIG. 1 is a flowchart of an implementation method of a lane traffic state reminding method provided by an embodiment of the present application.
[0017] FIG. 2 is an application scenario diagram of a lane passing state reminding method provided by an embodiment of the present application.
[0018] FIG. 3 is an implementation flowchart of image feature recognition provided by an embodiment of the present application.
[0019] FIG. 4 is a flowchart of obtaining lane information according to an embodiment of the present application.
[0020] FIG. 5 is a flowchart of an implementation of determining a passing state provided by an embodiment of the present application.
[0021] FIG. 6 is a schematic diagram of a functional structure of a passing state provided by an embodiment of the present application.
DETAILED DESCRIPTION [0022] The following describes the preferred embodiments of the present application in detail with reference to the accompanying drawings, so that the advantages and features of the present application can be more easily understood by those skilled in the art, thereby making the protection scope of the present application more clearly defined.
[0023] Please refer to FIG. 1, which shows an implementation process of a method for reminding a lane traffic state provided by an embodiment of the present application.
[0024] The lane passing state reminding method is applied to an electronic device, and the electronic device may be an on-board electronic device installed on a car. The vehicle-mounted electronic device may include a front-view camera, a processing circuit, and a reminder module.
[0025] The forward-looking camera may be directly in front of the vehicle to obtain a forward-looking image in front of the vehicle.
[0026] The processing circuit may perform analysis processing on the visual field image to determine the traffic state of the vehicle based on the forward-looking image.
[0027] The reminder module may be a sounder, a display, or an indicator light, etc., to issue a corresponding reminder according to the traffic state.
[0028] Please refer to FIG. 1, which shows an implementation process of the lane traffic state reminding method provided by the embodiment of the present application.
[0029] As shown in FIG. 1, a lane passing state reminding method is applied to an electronic device. The electronic device may be the electronic device described in the above embodiment. The method includes: 101. Obtaining a forward-looking image of a vehicle. Extract the image features of the forward-looking image.
[0030] The forward-looking image may be acquired by a forward-looking camera provided on the vehicle.
[0031] Among them, the image feature may be the shape, color, position and other features of each object in the image.
[0032] In some embodiments, the image features may be extracted by a preset image processing algorithm. Specifically, the image processing process may include steps such as graying, image filtering, image edge enhancement, image edge detection, and feature extraction of the forward-looking image. The specific implementation method may be determined according to the actual situation and different algorithms .
[0033] 102. Identify the lane information and the signal light information in the forward-looking image based on the image features.
[0034] Wherein, the lane information may include lane direction information and lane line information, etc., so as to obtain the lane position and lane travel direction related information through the above lane information.
[0035] The signal light information may include the lane direction type of the signal light and the corresponding signal type, so as to obtain the lane direction associated with the signal light and the corresponding signal type through the signal light information.
[0036] Specifically, after the image features of the forward-looking image are obtained, the features of the lane and the traffic light can be identified, and the lane information and the traffic light information obtained after the recognition can be obtained.
[0037] In some embodiments, the lane information and the signal light information may be obtained by a feature recognition algorithm. For example, through the feature recognition algorithm to identify the shape and contour characteristics of the signal light, in order to know whether the object is a signal light, and can know the specific position of the signal light; then, in order to obtain the signal light information, you can indicate the signal light The direction and whether the signal is red light, green light, or yellow light are determined. The obtained information is all signal light information.
[0038] 103, according to the lane information and signal information to determine the current traffic state of the vehicle.
[0039] Wherein, the current traffic state of the vehicle refers to whether there is a signal light during the current driving of the vehicle; if there is a signal light, combined with the signal light information and lane information to determine whether the traffic conditions are met at this time.
[0040] In some embodiments, the traffic state needs to be determined according to the lane direction of the current lane and the corresponding signal light type of the lane direction.
[0041] Please refer to FIG. 2, which illustrates an application scenario of a lane traffic state reminding method provided by an embodiment of the present application.
[0042] As shown in FIG. 2, the application scenario in the figure shows a forward-looking image, which includes a lane 11 and a signal light 12 in the forward-looking image. Wherein, the lane 11 includes a lane line 111 and a driving arrow 112, wherein the direction of the lane of the lane is shown by the driving arrow 112 provided on the road, for example, the right turn arrow 112 in the figure. The signal lamp 12 may include a plurality of lane direction types, such as a forward signal lamp in the forward direction shown in the figure and a right turn signal lamp 121 in the right turn direction.
[0043] After identifying the lane information and signal information in the forward-looking image, you can first determine the lane direction of the lane 11 is the right turn direction, and then turn the right turn signal 121 (that is, used to control the right turn direction lane Signal lights) to determine the signal type.
[0044] If the signal type of the right turn signal light 121 is red light at this time, the traffic state of the vehicle is a prohibited traffic state; if the signal type of the right turn signal light 121 is green light at this time, the traffic state of the vehicle is permitted traffic status.
[0045] It can be understood that, in addition to the above implementation manners, different judgment manners may be performed according to actual conditions, for example, if there is no right turn signal light.
[0046] After the passing state is obtained, a corresponding reminder operation may be performed according to the passing state.
[0047] 104. Generate reminder information according to the passing state, and display the reminder information.
[0048] Among them, the reminder information may be voice information related to the passing state of the vehicle, for example, if the front is a red light, the voice prompts "red light ahead, please wait", or text or graphics related to the passing state of the vehicle Or reminder information such as a control instruction that controls the indicator light to turn on and off, and the specific implementation manner of the reminder information may be determined according to actual conditions.
[0049] Displaying the reminder information may be to obtain voice information corresponding to the passing state and play the voice information. Or it is to obtain text and pattern information corresponding to the passing state and display the text or pattern information. Or it is to obtain a level signal corresponding to the passing state, and use the level signal to control the on and off of the preset indicator light. Of course, the specific display method can be designed according to the needs.
[0050] Through the above display method, the driver can timely know the traffic state of the current traffic and road conditions, making it difficult for the driver to ignore the reminder information.
[0051] It can be seen from the above that the image features in the forward-looking image are extracted, the lane information and the signal light information are identified based on the image features, and the current traffic state of the vehicle is determined according to the lane information and the signal light information, and reminder information is generated according to the traffic state And show it. In this application, the forward-looking image obtained by the device can be intelligently recognized at the signal lamp intersection according to the forward-looking image and remind the driver whether the vehicle can pass safely at this time, thereby improving the safety during driving.
[0052] Please refer to FIG. 3, which shows an implementation process of image feature recognition provided by an embodiment of the present application.
[0053] As shown in FIG. 3, the recognition of the lane information and the signal light information in the forward-looking image based on the image features includes: 201. Acquiring a lane reference feature.
[0054] The lane reference feature may be a preset feature parameter, and the feature parameter may be stored in a feature database at a specific location.
[0055] 202. Compare the image feature with the lane reference feature to determine whether there is a target feature that matches the lane reference feature in the image feature.
[0056] Wherein, the target feature includes a lane direction feature, and the lane direction feature is related to an object or graphic feature to which the forward direction of the lane belongs.
[0057] In some embodiments, the lane direction feature may be related to the shape feature of the driving arrow of the lane. For example, by extracting the shape feature of the driving arrow in the forward-looking image, and determining whether the shape feature is consistent with the left-turn arrow, the forward arrow, or the right-turn arrow, the direction of travel of the lane can be determined.
[0058] Of course, the driving arrow may be a driving arrow set on the road of the lane, or may be a driving arrow indicated on a street sign. The present application does not limit the location of the driving arrow.
[0059] 203. If yes, extract information corresponding to the target feature to obtain lane information, where the lane information includes lane direction information.
[0060] Wherein, information corresponding to the target feature is extracted, and information having a mapping relationship with the target feature may be extracted from an information database related to the target feature.
[0061] For example, if the target feature is a lane feature, and the feature determines that the corresponding information in the information database is a "right turn lane", then the "right turn lane" is the information corresponding to the target feature. Of course, the "right-turn lane" information indicating the forward direction of the lane may be the lane direction information.
[0062] It can be known from the above that by acquiring the lane reference feature and comparing the image feature with the lane reference feature, the lane information of the lane can be quickly and accurately obtained.
[0063] Please refer to FIG. 4, which shows an implementation process of obtaining lane information provided by an embodiment of the present application.
[0064] Under normal circumstances, the forward-looking image contains a large number of image features. If the image features need to be processed in real time during the vehicle forward process, a large amount of data calculation is required, resulting in a large delay. In order to improve the recognition rate of lane features, reduce the amount of data calculation in the recognition process, thereby saving data calculation time, and improve the reaction speed, the image features are compared with the lane reference features to determine whether the image features There are target features that match the lane reference features, including: 301. Determine whether there are lane line features among the image features.
[0065] Wherein, the lane line is a line used for dividing the lane, and may be a line segment.
[0066] 302. If there is a lane line feature, the lane area of the lane is determined according to the lane line feature.
[0067] If the lane line feature corresponding to the lane line is detected, the lane area separated by the lane line can be determined.
[0068] With reference to FIG. 2, two lane lines 111 are shown in the figure, and the area enclosed by the lane line 111 is a lane area.
[0069] 303. In the lane area, determine whether there is a lane direction feature among the image features.
[0070] After the lane area of the lane is determined, the object features outside the lane area can be ignored, that is, the object features outside the lane area are not recognized, so that the number of required recognition features can be greatly reduced and the data of the recognition process can be reduced The amount of calculation saves the calculation time of data and further improves the reaction speed.
[0071] Please refer to FIG. 5, which shows an implementation process of determining a passing state provided by an embodiment of the present application.
[0072] In some embodiments, the signal light information includes the lane direction type of the signal light and the corresponding signal type.
[0073] With reference to FIG. 2, wherein the current traffic state of the vehicle is determined based on the lane information and the signal light information, including: 401. Determine the traffic direction of the vehicle that is about to pass according to the lane direction information.
[0074] For example, if the obtained lane direction information is "right-turn lane", it may be determined that the traffic direction of the vehicle in the lane is right-turn.
[0075] 402. Acquire a signal type of a signal light corresponding to a traffic direction.
[0076] Among them, it is possible to find a signal lamp corresponding to the direction of traffic through the preset relationship in the signal lamp information, for example, a right turn signal lamp 121, and determine whether the signal type of the corresponding signal lamp of the right turn signal lamp 121 is a red light or Green light.
[0077] 403. Determine the current traffic state of the vehicle according to the signal type, where the traffic state includes a traffic allowed state and a traffic prohibited state.
[0078] If it is a red light at this time, the current traffic state of the vehicle is a prohibited traffic state; if it is a green light at this time, the current traffic state of the vehicle is a permitted traffic state.
[0079] As can be seen from the above, using the obtained lane direction information and combining it with the signal type of the signal light, the current traffic state of the vehicle can be judged more accurately, so as to remind the user of the traffic state more accurately and ensure the traffic of the vehicle Safety.
[0080] In some embodiments, the traffic state may also include a normal driving state and attention reminder state, based on the lane information and signal information to determine the current traffic state of the vehicle, including: if neither lane information nor signal information can be identified , The vehicle is currently in the driving state; if the lane information cannot be recognized and the signal information is present, the vehicle is currently in the caution state; if the lane information is present and the signal information cannot be recognized, the vehicle is currently in the driving state.
[0081] Judging according to the obtained different lane information and signal light information, it is possible to further give the driver sufficient prompts under different driving environments, thereby ensuring safer passage of the vehicle.
[0082] Please refer to FIG. 5, which shows a lane passing state reminding device provided by an embodiment of the present application.
[0083] As shown in FIG. 5, the lane passing state reminding device 5 includes a forward-looking camera 51, a processing circuit 52 electrically connected to the forward-looking camera 51, and a reminding module 53 electrically connected to the processing circuit 52, wherein : The front-view camera 51 is used to obtain a front-view image of the vehicle.
[0084] Wherein, the front-view camera 51 may be provided with a CCD (Charged Coupled Device, charge affinity device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor, complementary metal oxide semiconductor) image sensor, the specific type is not limited in this application.
[0085] The processing circuit 52 is used to extract image features of the forward-looking image; identify lane information and signal information in the forward-looking image based on the image features; determine the current traffic state of the vehicle based on the lane information and signal information; according to the traffic state Generate reminder information.
[0086] Wherein, the processing circuit 52 may include a processor, a memory and a corresponding circuit function module, the processor is electrically connected to the memory.
[0087] Specifically, the memory may be used to store computer programs and data. The computer program stored in the memory contains instructions executable in the processor. By calling the computer program stored in the memory, the processor can execute the method for reminding the lane traffic state as described above.
[0088] In some embodiments, the processing circuit 52 is specifically configured to: obtain a lane reference feature; compare the image feature with the lane reference feature, and determine whether there is a A target feature that matches the lane reference feature, where the target feature includes a lane line feature and a lane direction feature; if so, extract information corresponding to the target feature to obtain the lane information, where the lane information includes Lane direction information.
[0089] The reminder module 53 is used to display reminder information.
[0090] In some embodiments, the reminder module may include one of a sounder, a display, or an indicator light.
[0091] The sound generator is used for playing voice information, and the voice information corresponds to a passing state. For example, if the signal light in front is a red light, the generator can play the voice message "Red light in front, please wait".
[0092] The display is used to display text or pattern information, and the text or pattern information corresponds to a passing state.
For example, if the signal light in front is a red light, the text "Red light ahead, please wait" or a pattern representing that meaning is displayed.
[0093] The indicator light is used to turn on and off under the control of the level signal of the reminder module, and the level signal corresponds to the passing state. For example, there are three indicator lights corresponding to different signal lights. If the signal light in front is a red light, the indicator light corresponding to the red light is controlled to light up, and so on.
[0094] As can be seen from the above, the lane traffic state reminding device uses image features in the forward-looking image to extract lane information and traffic light information based on the image features, and determines the current traffic state of the vehicle based on the lane information and traffic light information. Generate reminder information and display it according to traffic status. In this application, the forward-looking image obtained by the device can be intelligently recognized at the signal lamp intersection according to the forward-looking image and remind the driver whether the vehicle can pass safely at this time, thereby improving the safety during driving.
[0095] Persons of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, the storage medium Can include: read-only memory (ROM, ReadOnlyMemory), random access memory (RAM, RandomAccessMemory), magnetic disk or optical disk, etc.
[0096] In the embodiment of the present application, the lane traffic state reminding device belongs to the same concept as the lane traffic state reminding method in the above embodiment, and the lane traffic state can be run on the lane traffic state reminding device For any method steps provided in the reminding method embodiment, the specific implementation process is described in detail in the lane passing state reminding method embodiment, and any combination may be used to form an optional embodiment of the present application, which will not be repeated here.
[0097] The embodiments of the present application have been described in detail above with reference to the drawings, but the present application is not limited to the above-mentioned embodiments, and within the scope of knowledge possessed by those of ordinary skill in the art, it can also be on the premise of not departing from the purpose of the application Make various changes.

Claims (11)

1. A target tracking processing method, applied to an electronic device including at least three processing units, characterized in that the method includes:

   Through the first processing unit, acquire a plurality of corner points of the target image, track the corresponding positions of the corner points in the target image in multiple frames, and obtain tracking data of the corner points;

   Through the second processing unit, detect and identify the target object in the target image to determine the image range corresponding to the target object;

   The third processing unit determines the target range of the target object according to the tracking data of the corner point and the image range of the target object.
2. The target tracking processing method according to claim 1, wherein the multiple corners of the target image are acquired, and the corresponding positions of the corner points in the target image in multiple frames are tracked to obtain Tracking data for the corner points, including:

   Acquiring a preset area of the target image;

   Construct an image pyramid of the preset area;

   Determine multiple corner points according to the image pyramid;

   Tracking the corresponding position of the corner point in the target image in multiple frames to obtain tracking data of the corner point.
3. The target tracking processing method according to claim 2, wherein the tracking the corresponding position of the corner point in the multi-frame target image to obtain the tracking data of the corner point includes:

   Perform forward pyramid LK tracking and reverse pyramid LK tracking on the corner points respectively to obtain preliminary tracking data;

   Perform cross-validation on the preliminary tracking data to obtain tracking data of corner points that satisfy preset conditions among the plurality of corner points.
4. The target tracking processing method according to claim 1, wherein the determining the target range of the target object according to the tracking data of the corner point and the image range of the target object includes:

   Determine the target corner point corresponding to the target object according to the tracking data of the corner point and the image range of the target object; determine the target according to the tracking data of the target corner point and the image range of the target object The target range of the object.
5. The target tracking processing method according to claim 4, wherein the tracking data of the corner points includes a tracking range corresponding to each of the corner points in the corner point set;

   The determining the target range of the target object according to the tracking data of the target corner point and the image range of the target object includes:

   Determining the confidence of the tracking range of the corner point and the image range of the target object;

   According to the confidence of the tracking range and the image range, the target range of the target object is determined according to a preset weighting value, respectively.
6. The target tracking processing method according to claim 4, wherein the determining the target corner point corresponding to the target object according to the tracking data of the corner point and the image range of the target object includes:

   Acquiring a preset mapping relationship between the target image and the image range;

   According to the mapping relationship, determine a plurality of mapped pixels of the image range in the target image;

   Determining the pixel range enclosed by the image range in the target image according to the mapped pixels;

   The corner point located within the pixel range is taken as the target corner point.
7. The target tracking processing method according to claim 6, wherein before the acquiring the preset mapping relationship between the target image and the image range, the method further comprises:

   Through the first processing unit, a mapping relationship between pixels in the target image and the image range is constructed.
8. The target tracking processing method according to any one of claims 1-7, wherein the first processing unit is a vector operation processor.
9. An electronic device, characterized in that the electronic device includes at least three processing units, wherein:

   The first processing unit is configured to acquire multiple corner points of the target image, track the corresponding positions of the corner points in the target image in multiple frames, and obtain tracking data of the corner points;

   The second processing unit is configured to detect a target object in the target image and determine an image range corresponding to the target object;

   The third processing unit is configured to determine the target range of the target object according to the tracking data of the corner point and the image range of the target object.
10. The electronic device according to claim 9, wherein the first processing unit is a vector operation processor.
11. 1. A lane passing state reminding method, applied to an electronic device, characterized by comprising: acquiring a forward-looking image of a vehicle, extracting image features of the forward-looking image; identifying the forward-looking image based on the image features Lane information and signal light information in; determining the current traffic state of the vehicle according to the lane information and signal light information; generating reminder information according to the traffic state, and displaying the reminder information.
    2. The lane traffic state reminding method according to claim 1, wherein identifying lane information and signal information in the forward-looking image based on the image features includes: obtaining a lane reference feature; converting the image The feature is compared with the lane reference feature to determine whether there is a target feature matching the lane reference feature among the image features, wherein the target feature includes a lane direction feature; if so, the target feature is mapped Information is extracted to obtain the lane information, where the lane information includes lane direction information.
    3. The lane traffic state reminding method according to claim 2, wherein the target feature further includes a lane line feature; and the image feature is compared with the lane reference feature to determine Whether there is a target feature matching the lane reference feature in the image feature includes: determining whether the lane line feature exists in the image feature; if the lane line feature exists, determining the lane feature according to the lane line feature The lane area of the lane; in the lane area, it is determined whether the lane direction feature exists among the image features.
    4. The lane traffic state reminding method according to claim 3, wherein the lane direction feature is related to the shape feature of the driving arrow of the lane.
    5. The method for reminding lane traffic status according to claim 2, wherein the signal light information includes a lane direction type of the signal light and a corresponding signal type to determine the current traffic of the vehicle according to the lane information and the signal light information The status includes: determining the traffic direction of the vehicle that is about to pass according to the lane direction information; acquiring the signal type of the signal light corresponding to the traffic direction; determining the current traffic state of the vehicle according to the signal type, wherein the The pass state includes the pass-through state and the pass-through state.
    6. The lane traffic state reminding method according to claim 2, wherein the traffic state includes a normal driving state and an attention reminding state, and the current traffic state of the vehicle is determined according to the lane information and the signal light information, Including: if none of the lane information and the signal information can be identified, the vehicle is currently in a driving state;
    If the lane information cannot be recognized, and the signal light information is present, the vehicle is currently in a cautionary state. If the lane information is present, and the signal light information cannot be recognized, the vehicle is currently driving status.
    The method for reminding the lane traffic status according to any one of claims 1-6, wherein the generating of the reminder information according to the traffic status and displaying the reminder information includes: acquiring and Voice information corresponding to the state, playing the voice information; obtaining text and pattern information corresponding to the traffic state, displaying the text or pattern information; or obtaining a level signal corresponding to the traffic state, The level signal controls the lighting of the preset indicator.
    8. A lane passing state reminding device, characterized by comprising a forward-looking camera, a processing circuit electrically connected to the forward-looking camera, and a reminding module electrically connected to the processing circuit, wherein: the forward-looking camera, Used to obtain a forward-looking image of the vehicle; the processing circuit is used to extract image features of the forward-looking image; identify lane information and signal information in the forward-looking image based on the image features; according to the lane The information and the signal light information determine the current traffic state of the vehicle; generate reminder information according to the traffic state; and the reminder module for displaying the reminder information.
    9. The lane traffic state reminding device according to claim 8, wherein the processing circuit is specifically used to: obtain a lane reference feature; compare the image feature with the lane reference feature, the Determine whether there is a target feature matching the lane reference feature among the image features, wherein the target feature includes a lane line feature and a lane direction feature: if so, extract information corresponding to the target feature to obtain The lane information, wherein the lane information includes lane direction information.
    10. The lane passing state reminding device according to claim 8 or 9, wherein the reminding module includes one of a sound generator, a display, or an indicator light; the sound generator is used to play voice information , The voice information corresponds to the traffic state: the display is used to display text or pattern information, the text or pattern information corresponds to the traffic state; the indicator light is used to receive the reminder The level signal of the module is controlled to turn on and off, and the level signal corresponds to the passing state.
    

Images