JP7258078B2 - Real scene navigation icon display method, apparatus, equipment and medium - Google Patents

Description

TECHNICAL FIELD Embodiments of the present application relate to computer technology, specifically to real-world navigation technology, and more particularly to a real-world navigation icon display method, apparatus, apparatus and medium.

Real scene navigation is a combination of navigation technology and augmented reality (AR), and can provide users with clearer and more intuitive navigation guidance so that they do not get lost.

However, in the process of real-world navigation, especially when making a curve or changing the road, the display of the navigation icons is not smooth with the change of the vehicle position, and the display of the icons is likely to deviate from the road entity. Therefore, how to optimize the display of real-world navigation icons during the steering process of a vehicle is an issue that needs to be considered at present.

Embodiments of the present application disclose a method, device, apparatus and medium for displaying real-world navigation icons to optimize the display effect of navigation icons in the process of real-world navigation steering.

In a first aspect, an embodiment of the present application discloses a method for displaying real-world navigation icons performed by a real-world navigation icon display device, comprising :
determining a road traveled by the vehicle by matching the positioning coordinates of the vehicle with an electronic map;
determining a steering navigation path for the vehicle based on the road of travel;
performing a fitting process on the steering navigation path and generating an instruction icon based on the fitted steering navigation path;
determining a target lane of the road on which the vehicle is traveling on a real scene image;
and displaying the instruction icon on the actual scene image based on the positional relationship between the target lane and the instruction icon ;
When the traveling road is curved,
Determining a target lane of the road on which the vehicle is traveling on a real scene image includes:
using a preset image recognition algorithm to recognize a real scene image of the vehicle in motion to obtain an initial recognition result of a target lane on the driving road;
obtaining at least one of a deflection angle of a vehicle body and road coordinates of the travel road on the electronic map;
using at least one of the obtained vehicle body deflection angle and the road coordinates to modify the initial recognition result of the target lane to obtain a target recognition result of the target lane.

In a second aspect, embodiments of the present application disclose a live view navigation icon display device,
a driving road determination module for determining a driving road of the vehicle by matching the positioning coordinates of the vehicle with an electronic map;
a steering navigation path determination module for determining a steering navigation path of the vehicle based on the driving road;
an icon generation module for performing a fitting process on the steering navigation path and generating an instruction icon based on the fitted steering navigation path;
a target lane determination module for determining a target lane of the road on which the vehicle is running on a real scene image;
an icon display module for displaying the instruction icon on the actual scene image based on the positional relationship between the target lane and the instruction icon ;
When the traveling road is curved,
Determining a target lane of the road on which the vehicle is traveling on a real-world image includes:
Using a preset image recognition algorithm, recognize the real scene image while the vehicle is running, obtain the initial recognition result of the target lane on the driving road, and obtain the vehicle body deflection angle and the electronic obtaining at least one of road coordinates on a map; using at least one of the obtained vehicle body deflection angle and the road coordinates to modify an initial recognition result of the target lane; Get the target recognition result of the target lane.

In a third aspect, embodiments of the present application further disclose an electronic device,
at least one processor;
a memory communicatively coupled to the at least one processor;
The memory stores instructions executable by the at least one processor, the instructions comprising: executed by the at least one processor;

In a fourth aspect, embodiments of the present application further disclose a non-transitory computer-readable storage medium having computer instructions stored thereon, the computer instructions being stored in the computer according to any of the embodiments of the present application. Execute the described method for displaying real-world navigation icons.
In a fifth aspect, the embodiments of the present application further disclose a computer program, said computer program causing a computer to execute the in-scene navigation icon display method according to any of the embodiments of the present application.

According to the technical solution of the embodiment of the present application, in the steering process of the vehicle, after obtaining the current steering navigation path of the vehicle, the fitting process is performed on the steering navigation path, and the determined target lane and the indication icon are displayed. According to the positional relationship of , the pointing icon is displayed on the real scene image to optimize the overall display effect of the navigation icon in the real scene navigation process.

However, the descriptions in this section are not intended to limit the key or critical features of the embodiments of the disclosure, nor are they intended to limit the scope of the disclosure. Other features of the present disclosure will become easier to understand with the following specification.

The drawings are for better understanding of the present technical solution and do not limit the present application.
4 is a flow chart of a method for displaying real-world navigation icons disclosed by an embodiment of the present application; 4 is a flow chart of another method for displaying real-world navigation icons disclosed by an embodiment of the present application; FIG. 5 is a schematic diagram of the display effect of the real-scene navigation icons disclosed by the embodiments of the present application; 4 is a flow chart of another method for displaying real-world navigation icons disclosed by an embodiment of the present application; 1 is a schematic configuration diagram of a real-scene navigation icon display device disclosed by an embodiment of the present application; FIG. 1 is a block diagram of an electronic device disclosed by an embodiment of the present application; FIG.

Illustrative embodiments of the present application are described below with reference to the drawings, and various details of the embodiments of the present application are included for ease of understanding and should be considered as exemplary only. be. Accordingly, those skilled in the art should appreciate that various changes and modifications can be made to the example embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and brevity, the following description omits descriptions of well-known functions and constructions.

FIG. 1 is a flow chart of a method for displaying real-world navigation icons disclosed in an embodiment of the present application, which can be applied to real-time display of navigation icons in the process of real-world navigation such as curves and lane changes. can. The method of the present embodiment can be implemented by a real-world navigation icon display device, which can be implemented in software and/or hardware, and can be integrated into any electronic device with computing power, such as an in-vehicle device. Including but not limited to.

As shown in FIG. 1, the real-scene navigation icon display method disclosed by this embodiment can include the following steps.

S101: Determine the driving road of the vehicle by matching the positioning coordinates of the vehicle with the electronic map.

The positioning coordinates of the vehicle can be obtained by a positioning device of the vehicle. For example, the global positioning system (GPS) obtains the latitude and longitude coordinates of the vehicle in real time. Since the electronic map data includes the coordinates of each road, the road on which the vehicle is currently traveling can be determined by matching the positioning coordinates and the electronic map while the vehicle is traveling.

Selectably, the step of determining the road traveled by the vehicle by matching the positioning coordinates of the vehicle with the electronic map includes obtaining a set of positioning coordinates of the vehicle within a preset time period, and for the set of positioning coordinates: data filtering; and matching the filtered set of positioning coordinates with the electronic map to determine the road the vehicle is traveling on.

The preset length of time can be flexibly set according to needs, for example, while the vehicle is running, according to the time cycle, the vehicle positioning data is acquired periodically, and the positioning coordinate set is obtained. can be obtained and then used for road matching. Using the set of positioning coordinates within a set time to determine the road on which the vehicle travels contributes to improving the stability of road matching. By filtering the positioning coordinate set, some positioning points with large deviation can be removed, which contributes to improving the accuracy of road matching and further ensures that the steering navigation path is obtained accurately. Here, as for the filtering algorithm, this embodiment is not specifically limited, and any available algorithm in the prior art can be used as long as it can implement data filtering.

S102: Determine a steering navigation path for the vehicle based on the road on which it is traveling.

A steering navigation path is a portion of a complete navigation path obtained using any available path planning algorithm based on the road the vehicle is currently traveling on and the vehicle's destination. Based on the road on which the vehicle is currently driving, it is possible to obtain the steering navigation path of the vehicle and determine the road or carriageway that the vehicle is about to enter, i.e. the steering (driving direction) referred to in this embodiment. changes) include situations such as vehicle curves and vehicle lane changes.

S103: Fitting is performed on the steering navigation path, and an instruction icon is generated based on the fitted steering navigation path.

The pointing icon is a navigation icon that indicates the inner lane of the vehicle during the steering process of the vehicle, and can realistically indicate the lane that the vehicle is currently approaching or going to cross. Generate an instruction icon based on the current steering navigation path, match the instruction icon with the current steering navigation path, for example, match the shape of the two, ensure the accuracy of the navigation guidance, and include it in the steering navigation path The world coordinates and image coordinates of the pointing icon can be determined based on the position coordinates provided. A smooth curve can be obtained by fitting the steering navigation path, and the current indication icon generated based on the smooth curve ensures a certain display smoothness in subsequent displays, To avoid the occurrence of some jumpy coordinate points in the steering navigation path and affect the display effect of the pointing icon.

Here, the fitting algorithm used in performing the fitting process for the steering navigation path after ensuring that a smooth curve is obtained is not specifically limited in this embodiment, and Any available algorithm can be flexibly selected. Illustratively, a Bessel curve algorithm is used to fit the steering navigation path.

A Bessel curve is a basic tool in computer graphics image modeling, where graphics are created by controlling four points on a curve: a start point, an end point, and two intermediate points that are separated from each other. and can be edited, where an important role is played by the control line located in the middle of the curve. This control line is imaginary, intersecting the Bessel curve in the middle and having control endpoints at both ends. When moving the endpoints at both ends, when the Bessel curve changes the curvature of the curve, when moving the middle point, i.e. when moving the virtual control line, the Bessel curve changes when the start and end points are locked. move uniformly to The Bessel curve algorithm is usually the preferred algorithm during the graphics editing process because all control points, nodes on the Bessel curve are editable.

In addition, as the vehicle runs in real time, the steering navigation path of the vehicle also changes, and the instruction icon generated based on the fitted steering navigation path also changes. Also called flow icons, they can show dynamically changing effects while the vehicle is in motion. Both the length of the curve corresponding to the pointing icon and the width and height of the pointing icon can be flexibly set. Selectably, the pointing icon may be composed of a plurality of preset icon units, the preset icon units may be, for example, V-shaped icon units, each preset icon unit are arranged in order with the smooth curve obtained by the fitting process as the center line.

S104: Determine the target lane of the road on which the vehicle is running on the actual scene image.

In the process of real-world navigation, the camera installed in the vehicle can be used to collect real-time images of the vehicle driving environment, and then the lane recognition technology can be used to recognize the lanes in the real-world image; Lane recognition techniques can include, but are not limited to, image processing-based recognition techniques, model training-based recognition techniques, and the like. The specific lane recognition process may be performed locally in the local device or performed in the cloud server, and is not specifically limited in this embodiment. By recognizing the real scene image, the finally recognized lane may include at least one lane, where the target lane is the lane of the vehicle during a steering process such as when the vehicle turns or changes lanes. Point to the inside lane. In the lane recognition process, the target lane may be determined in combination with the driving direction that the vehicle intends to turn, and the vehicle steering judgment may be determined by analyzing the deflection angle of the vehicle body. For example, when the vehicle is about to turn right, the closest lane on the right side of the vehicle on the real scene image is determined as the target lane; Determine the lane as the target lane.

S105: Display the instruction icon on the actual scene image based on the positional relationship between the target lane and the instruction icon.

The pointing icon indicates the target lane to the user during the steering process of the vehicle, and the pointing icon is displayed on the actual scene image based on the positional relationship between the two, for example, the image coordinate relationship between the two on the actual scene image, thereby facilitating navigation. To obtain a display effect in which an icon fits a real scene.

It should be noted that the operations S101 to S103 and operation S104 are not limited in execution order, and the operation order shown in FIG. 1 should not be understood as a specific limitation of the present embodiment.

According to the technical solution of the present embodiment, during the steering process of the vehicle, after obtaining the current steering navigation path of the vehicle, the fitting process is performed on the steering navigation path to obtain a smooth curve, and then the smooth curve is obtained. Generate the current instruction icon based on a smooth curve to avoid the phenomenon of affecting the display effect of the instruction icon due to the appearance of some jumpy coordinate points on the steering navigation path while the vehicle is running, To ensure the smoothness of the icon display, solve the problem of uneven display of the real-world navigation icon during the steering process of the vehicle, and according to the positional relationship between the determined target lane and the indicator icon, By displaying the instruction icon on the real scene image, it is possible to ensure the close contact between the instruction icon and the lane, and avoid the display effect of the instruction icon deviating from the lane, so that the overall display effect of the navigation icon in the process of the real scene navigation is achieved. optimized for

FIG. 2 is a flowchart of another real-world navigation icon display method disclosed by an embodiment of the present application, which can be further optimized and extended based on the above technical solution and combined with the above optional embodiments. As shown in FIG. 2, the method can include the following steps.

S201: Determine the driving road of the vehicle by matching the positioning coordinates of the vehicle with the electronic map.

S202: Determining a steering navigation path for the vehicle according to the driving road.

S203: Perform fitting processing on the steering navigation path.

S204: Based on the shape of the fitted steering navigation path, use a three-dimensional icon drawing method to draw the pointing icon.

Here, a three-dimensional (3D) icon drawing method can be realized with a three-dimensional icon drawing tool such as OpenGL. Illustratively, the instruction icon may be composed of a plurality of preset icon units, and the preset icon units are all 3D icon units, for example, 3D V-shaped icon units. Each preset icon unit is arranged in order with the smooth curve obtained by the fitting process as the center line.

S205: Determine the world coordinates corresponding to the pointing icon according to the position coordinates included in the fitted steering navigation path.

The pointing icon matches the shape of the steering navigation path and may be part of the steering navigation path, so that the position coordinates (i.e. world coordinates) contained in the fitted steering navigation path are correspondingly pointing It may be the world coordinates of the icon, and the world coordinates of the pointing icon further include the coordinates of the icon centerline along the length direction. After the coordinates of the center line of the icon are determined, the world coordinates corresponding to each outline of the pointing icon can be determined in combination with the preset width and height of the pointing icon. Naturally, within a preset floating display range, i.e., for the entire position coordinate in the fitted steering navigation path, the floating change in world coordinates does not significantly adversely affect the final display effect of the navigation icons. It can also be used to perform the appropriate translational transformations and then determine the world coordinates corresponding to the pointing icon.

S206: Using the transformation relationship between the world coordinate system and the image coordinate system, determine the image coordinates of the pointing icon on the real scene image.

Here, the transformation relationship between the world coordinate system and the image coordinate system may be obtained based on the internal parameters and external parameters of the vehicle-mounted camera, and the description thereof is omitted in this embodiment.

S207: Determine the target lane of the road on which the vehicle is running on the real scene image.

S208: Display the pointing icon on the real scene image along the steering direction based on the respective image coordinates of the target lane and the pointing icon on the real scene image.

In the process of recognizing the target lane, its image coordinates can also be determined. The pointing icon is used to direct the user to the target lane, therefore, after obtaining the image coordinates of both, the pointing icon is displayed on the real scene image along the current steering direction, so that the navigation icon fits perfectly on the real scene. display effect can be obtained. It should be noted that the operations S201-S206 and operation S207 are not limited in execution order, and the operation order shown in FIG. 2 should not be understood as a specific limitation of this embodiment.

According to the technical solution of the present embodiment, during the steering process of the vehicle, after obtaining the current steering navigation path of the vehicle, the fitting process is performed on the steering navigation path to obtain a smooth curve, and then the smooth curve is obtained. Generate the current instruction icon based on a smooth curve to avoid the phenomenon of affecting the display effect of the instruction icon due to the appearance of some jumpy coordinate points on the steering navigation path while the vehicle is running, Ensure the smoothness of the icon display, solve the problem of uneven display of the real-world navigation icons during the vehicle steering process. In addition, based on the image coordinates, the instruction icon is displayed along the current steering direction on the actual scene image to ensure the degree of close contact between the instruction icon and the lane, and the indication that the instruction icon is out of the lane. To avoid the effect, thereby optimizing the overall display effect of navigation icons in the process of real-world navigation.

Based on the above technical scheme, after the step of drawing the pointing icon using a three-dimensional icon drawing method, based on the shape of the steering navigation path that is selectively fitted, the method of the present embodiment is three-dimensional. using an icon drawing scheme to draw a steering icon that is the same as the direction of the fitted steering navigation path;
Accordingly, after the step of displaying the pointing icon on the scene image, the example method further includes displaying a steering icon above the pointing icon.

A steering icon indicates that the vehicle is about to change direction. Both the pointing icon and the steering icon are displayed in a 3D style, exhibiting good stereoscopic effect and realism, improving the visual effect of the real-scene navigation icon display, and providing good navigation guidance to the user.

FIG. 3 is a schematic diagram showing the display effect of real-world navigation icons, taking the vehicle changing lanes as an example. In FIG. 3, the steering icon is displayed above the instruction icon, both the steering icon and the instruction icon are 3D icons, and the instruction icon is displayed in the real scene image along the current steering direction, i.e., the direction to the target lane. be done. FIG. 3 is not to be understood as a specific limitation to this embodiment as a simple overview of the navigation icon display effect in this embodiment. A local view of the navigation path may be displayed on the real-world image, and the vehicle speed, current road name, road name to be entered, traveled distance, traveled time, and orbis, etc., may be displayed as shown in the lower right corner of FIG. Additional information may be displayed and is not specifically limited in this embodiment.

FIG. 4 is a flow chart of another real-world navigation icon display method disclosed by an embodiment of the present application, which can be further optimized and extended based on the above technical solution and combined with the above optional embodiments. Specifically, FIG. 4 exemplifies a method of displaying a real-scene navigation icon according to the present embodiment, taking a curve scene as an example. As shown in FIG. 4, the method may include the following steps.

S301: Determine the driving road of the vehicle by matching the positioning coordinates of the vehicle with the electronic map.

S302: Determine a steering navigation path of the vehicle according to the road on which it is traveling.

S303: Fitting is performed on the steering navigation path, and an instruction icon is generated based on the fitted steering navigation path.

S304: Using the preset image recognition algorithm, recognize the real scene image while the vehicle is running, and obtain the initial recognition result of the target lane on the driving road.

A preset image recognition algorithm is used to recognize lanes on the image, and the specific implementation of the algorithm is not specifically limited in this embodiment.

S305: Obtain at least one of the deflection angle of the vehicle body and the road coordinates on the electronic map of the traveling road.

S306: Using at least one of the obtained vehicle body deflection angle and road coordinates to modify the initial recognition result of the target lane to obtain the target recognition result of the target lane.

In the curve scene, the vehicle body deflection angle is constantly changing with the change of the road bend, so the change of the body deflection angle can be used to correct the curve lane recognition result. The body deflection angle can be obtained using the vehicle's inertial measurement unit (IMU). Road coordinates are the world coordinates stored in the electronic map of the road on which the vehicle is currently traveling. Changes in road coordinates can reflect changes in road geometry, and the combination of road coordinates and road width can be used to estimate the world coordinates of lanes on the current road. Similarly, curved lane recognition results can be modified.

If the vehicle deflection angle and road coordinates are used simultaneously for lane recognition result correction, a weighted fusion process can be performed on both for lane recognition result correction. Weight assignment may be flexibly set after securing the correction probability, and is not specifically limited in this embodiment. Additionally, an extended Kalman filtering algorithm may be used to modify lane recognition results.

Correcting the initial recognition result of the lane using at least one of the deflection angle of the vehicle body and the road coordinates to ensure the accuracy of the final recognition result of the target lane, and the accuracy of the display position of the instruction icon. to ensure that the navigation icons have a realistic display effect.

S307: Display the instruction icon on the actual scene image based on the positional relationship between the target lane and the instruction icon.

According to the technical solution of this embodiment, in the steering process of the vehicle, first, the current steering navigation path of the vehicle is obtained, and then the fitting process is performed on the steering navigation path to obtain a smooth curve, Further, the instruction icon is generated based on the smooth curve, and the appearance of some jumping coordinate points on the steering navigation path while the vehicle is running avoids the phenomenon affecting the display effect of the instruction icon. To ensure the smoothness of icon display, solve the problem of uneven display of real-world navigation icons during vehicle steering process, and then at least one of the obtained vehicle body deflection angle and road coordinates. using one to correct the initial recognition result of the target lane to ensure the accuracy of the final recognition result of the target lane, furthermore to ensure the accuracy of the display position of the instruction icon, and similarly, the lane recognition result is not accurate and stable, so that the display of the real-world navigation icon is not smooth. , to ensure the close contact between the indicator icon and the lane, avoid the display effect of the indicator icon deviating from the lane, and thereby optimize the overall display effect of the navigation icon in the process of real-world navigation.

In the normal case, the image recognition algorithm can satisfy most lane recognition situations, and compared with the lane recognition algorithm based on the training model, the computational complexity involved in the image recognition algorithm is relatively low, and the navigation device also consumes relatively less computational resources, so it can be the preferred lane recognition method. However, in some special cases, such as night scenes, due to the low image acquisition quality, there may exist large errors when using image recognition algorithms to recognize lanes, and at this time, based on the training model Lane recognition algorithms can be used to improve recognition accuracy.

Based on the above technical solution, the step of selectively determining the target lane of the road on which the vehicle is running on the real scene image includes:
using a pre-trained neural network model to determine a target lane of a road on which the vehicle is traveling on a real scene image, wherein the neural network model is based on sample images of different image parameters; and the image parameters include image brightness and image contrast.

For example, a large number of sample images with different image parameters taken at night may be collected as inputs for model training, and the lane marking results in the sample images may be used as outputs to train a neural network model. Usable neural network structures are not specifically limited in this embodiment, and can be flexibly selected as required. For example, the neural network structure may be divided into a front-end network and a back-end network. Encoder-Decoder network of, but not limited to. Multi-layer network links can enhance the model's expressive ability, further improve the model's robustness and adaptability to night scenes, output highly accurate lane recognition results, and have good real-world navigation effects even in night scenes. can be displayed.

Of course, for lane recognition, the local device may transmit the actual scene image to the cloud server after acquiring the actual scene image, and the cloud server may perform lane recognition processing. Recognize real-world images using pre-trained neural network models, or use pre-configured image recognition algorithms to determine the target lane of the road the vehicle is currently traveling on and then feeding back the lane recognition result to the local device, thereby reducing the resource consumption of the local device.

FIG. 5 is a schematic configuration diagram of a real-scene navigation icon display device disclosed in an embodiment of the present application, and this embodiment is applied to real-time display of navigation icons during real-scene navigation processes such as curves and lane changes. be able to. The device may be implemented in software and/or hardware, and may be integrated into any electronic device with computing power, including but not limited to vehicle-mounted devices and the like.

As shown in FIG. 5, the real-scene navigation icon display device 400 disclosed by this embodiment includes a driving road determination module 401, a steering navigation path determination module 402, an icon generation module 403, a target lane determination module 404, an icon display module 405, where:
The travel road determination module 401 determines the travel road of the vehicle by matching the positioning coordinates of the vehicle and the electronic map;
a steering navigation path determination module 402 for determining a steering navigation path for the vehicle based on the driving road;
The icon generation module 403 performs fitting processing on the steering navigation path, generates an instruction icon based on the fitted steering navigation path,
A target lane determination module 404 determines a target lane of a road on which the vehicle is traveling on the real scene image;
The icon display module 405 displays the instruction icon on the actual scene image based on the positional relationship between the target lane and the instruction icon.

Selectably, icon generation module 403 may:
a fitting processing unit for fitting the steering navigation path;
an indication icon drawing unit for drawing an indication icon using a three-dimensional icon drawing method based on the shape of the fitted steering navigation path;
a world coordinate determination unit for determining world coordinates corresponding to the pointing icon based on the position coordinates included in the fitted steering navigation path;
an image coordinate determination unit for determining the image coordinates of the pointing icon on the real scene image using the transformation relationship between the world coordinate system and the image coordinate system.

Selectably, the icon display module 405 specifically:
The pointing icon is displayed on the actual scene image along the steering direction based on the respective image coordinates of the target lane and the pointing icon on the actual scene image.

Optionally, the icon generation module 403 further includes a steering icon drawing unit, which uses a three-dimensional icon drawing method based on the shape of the steering navigation path to which the pointing icon drawing unit is fitted. , after performing the operation of drawing a pointing icon, use a three-dimensional icon drawing method to draw a steering icon that is the same as the direction of the fitted steering navigation path;
Accordingly, the icon display module 405 includes a pointing icon display unit and a steering icon display unit, wherein:
The instruction icon display unit displays the instruction icon on the actual scene image based on the positional relationship between the target lane and the instruction icon,
The steering icon display unit displays the steering icon above the instruction icon after the instruction icon display unit performs the operation of displaying the instruction icon on the actual scene image.

Selectably, icon generation module 403 may:
a fitting processing unit for fitting to the steering navigation path using a Bessel curve algorithm;
an indicator icon generation unit for generating an indicator icon based on the fitted steering navigation path.

Selectably, the driving road determination module 401:
a data filtering unit for obtaining a set of positioning coordinates of the vehicle within a preset time and performing data filtering on the set of positioning coordinates;
a driving road determination unit for matching the filtered set of positioning coordinates with the electronic map to determine the driving road of the vehicle.

Selectably, if the road of travel is curved, the target lane determination module 404 may:
an initial recognition result determination unit for recognizing a real scene image while the vehicle is running using a preset image recognition algorithm to obtain an initial recognition result of the target lane on the driving road;
a deflection angle and road coordinate acquisition unit for acquiring at least one of the deflection angle of the vehicle body and the road coordinates on the electronic map of the driving road;
a recognition result modification unit for using at least one of the obtained vehicle body deflection angle and road coordinates to modify the initial recognition result of the target lane to obtain the target recognition result of the target lane. .

Selectably, the target lane determination module 404 specifically:
Using a pre-trained neural network model to determine the target lane of the road on which the vehicle is running on the real-world image;
A neural network model is trained based on sample images of different image parameters, the image parameters including image brightness and image contrast.

The real-world navigation icon display device 400 disclosed by the embodiments of the present application can implement any of the real-world navigation icon display methods disclosed by the embodiments of the present application, and the functional modules and beneficial functions corresponding to executing the methods can be implemented. have a great effect. For details not described in this embodiment, reference may be made to the description contained in any method embodiment of the present application.

According to embodiments of the present application, embodiments of the present application further provide an electronic device and a readable storage medium.
According to an embodiment of the present application, the present application provides a computer program, the computer program causing a computer to execute the real-world navigation icon display method provided by the present application.

As shown in FIG. 6, it is a block diagram of an electronic device of a method for displaying real-world navigation icons according to an embodiment of the present application. Electronic equipment is intended to represent various forms of digital computers such as, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronics can also represent various forms of mobile devices such as personal digital processors, mobile phones, smart phones, wearable devices, and other similar computing devices. The components, their connections and relationships, and their functionality illustrated herein are merely examples and are not intended to limit the implementation of the embodiments of the application as described and/or required herein.

As shown in FIG. 6, the electronic device includes one or more processors 501, memory 502, and interfaces for connecting components including high speed and low speed interfaces. Each component is interconnected by a different bus and can be mounted on a common motherboard or otherwise mounted as desired. The processor can process instructions executed within the electronic device, said instructions being transmitted in or on memory to an external input/output device (e.g., a display device coupled to the interface) to a graphical user interface ( Graphical User Interface (GUI) contains instructions for displaying graphical information. In other embodiments, multiple processors and/or multiple buses can be used along with multiple memories, if desired. Similarly, multiple devices can be connected, each electronic device providing some required operation, for example as a server bank, a group of blade servers, or a multi-processor system. In FIG. 6, one processor 501 is taken as an example.

Memory 502 is a non-transitory computer-readable storage medium according to embodiments of the present application. Here, the memory stores instructions executable by at least one processor, such that the at least one processor can execute the method for displaying real-world navigation icons according to an embodiment of the present application. A non-transitory computer-readable storage medium of an embodiment of the present application stores computer instructions for causing a computer to execute a method for displaying real-world navigation icons according to an embodiment of the present application.

The memory 502, as a non-temporary computer-readable storage medium, stores program instructions/modules (e.g., the driving road determination module 401 shown in FIG. 5, the steering navigation path Stores non-transitory software programs, non-transitory computer-executable programs and modules, such as determination module 402, icon generation module 403, target lane determination module 404 and icon display module 405). The processor 501 performs various functional applications and data processing of the electronic device by executing non-transitory software programs, instructions and modules stored in the memory 502, namely, the real-world navigation of the above method embodiments. Implement an icon display method.

The memory 502 can include a program storage area and a data storage area, where the program storage area can store an operating system, application programs required for at least one function, and the data storage area can: It can store data and the like created by using an electronic device. Memory 502 may also include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. is. In some embodiments, memory 502 may include memory located remotely to processor 501. These remote memories may be networked to electronic devices for implementing the in-scene navigation icon display method of the present embodiment. may be connected via Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The electronic device for implementing the method for displaying real-world navigation icons of the embodiments of the present application may further include an input device 503 and an output device 504 . The processor 501, the memory 502, the input device 503 and the output device 504 can be connected via a bus or other methods, and the connection via a bus is taken as an example in FIG.

The input device 503 may also receive input numeric or character information and generate key signal inputs related to user settings and function control of electronic devices for implementing the live view navigation icon display method of the embodiments of the present application. It is often an input device such as a touch screen, keypad, mouse, trackpad, touchpad, indicator stick, one or more mouse keys, trackball, joystick, or the like. The output device 504 may include a display device, an auxiliary lighting device, a haptic feedback device, etc., where the auxiliary lighting device is, for example, a Light Emitting Diode (LED), and the haptic feedback device is, for example, a vibramotor. be. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), an LED display, and a plasma display. In some embodiments, the display device can be a touch screen.

Various embodiments of the systems and techniques described herein may be digital electronic circuit systems, integrated circuit systems, Application Specific Integrated Circuits (ASICs), computer hardware, firmware, software, and/or Can be implemented in combination.These various embodiments can include being embodied in one or more computer programs, the one or more computer programs running on at least one programmable processor. The programmable processor, which may be an application specific or general purpose programmable processor, interprets data and instructions from a storage system, at least one input device, and at least one output device. It can be received and transmitted to the storage system, the at least one input device, and the at least one output device.

These computer programs (also called programs, software, software applications, or code) contain machine instructions for programmable processors and are implemented in a high-level procedural and/or object-oriented programming language and/or assembly/machine language. can do. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or apparatus for providing machine instructions and/or data to a programmable processor. , for example, magnetic disks, optical disks, memories, programmable logic devices (PLDs), including machine-readable media for receiving machine instructions, which are machine-readable signals. The term "machine-readable signal" is any signal for providing machine instructions and/or data to a programmable processor.

To provide interaction with a user, the systems and techniques described herein can be implemented on a computer, which includes a display device for displaying information to the user, such as a cathode ray tube (Cathode Ray Tube). Ray Tube, CRT) or LCD monitor, and a keyboard and pointing device, such as a mouse or trackball, through which a user can provide input to the computer. Other types of devices may be used to provide interaction with the user. For example, the feedback provided to the user may be any form of sensing feedback, e.g., visual, auditory, or tactile feedback, including acoustic, audio, and tactile input. It can receive input from the user in the form.

The systems and techniques described herein may be computing systems that include back-end components (eg, data servers), or computing systems that include middleware components (eg, application servers), or computing systems that include front-end components. A system (e.g., a user computer having a graphical user interface or web browser, through which a user interacts with embodiments of the systems and techniques described herein), or such a backend component , middleware components, and front-end components. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include Local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system can include a client side and a server. A client side and server are generally remote from each other and typically interact through a communication network. A client-server relationship is created by computer programs running on corresponding computers and having a client-server relationship to each other.

According to the technical solution of the embodiment of the present application, in the steering process of the vehicle, after obtaining the current steering navigation path of the vehicle, the steering navigation path is subjected to a fitting process to obtain a smooth curve; The indicator icon is generated according to the smooth curve to avoid the phenomenon of affecting the display effect of the indicator icon due to the appearance of some jumpy coordinate points on the steering navigation path while the vehicle is running; to ensure the display smoothness of the vehicle, solve the problem of uneven display of the real-world navigation icons during the steering process of the vehicle, and then, based on the determined positional relationship between the target lane and the indicator icon, By displaying the instruction icon on the real scene image, it is possible to ensure the close contact between the instruction icon and the lane, and avoid the display effect of the instruction icon deviating from the lane, so that the overall display effect of the navigation icon in the process of the real scene navigation is achieved. optimized for

It should be appreciated that steps may be reordered, added, or deleted using the various forms of flow shown above. For example, each step described in this application may be executed in parallel, sequentially, or in a different order, but the technical solution disclosed in this application There is no limitation herein as long as the desired result can be achieved.

The above specific embodiments do not limit the protection scope of the present application. Those skilled in the art can make various modifications, combinations, subcombinations, and permutations depending on design requirements and other factors. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall all fall within the protection scope of the present application.

Claims (11)

A real-scene navigation icon display method performed by a real-scene navigation icon display device , comprising:
determining a road traveled by the vehicle by matching the positioning coordinates of the vehicle with an electronic map;
determining a steering navigation path for the vehicle based on the road of travel;
performing a fitting process on the steering navigation path and generating an instruction icon based on the fitted steering navigation path;
determining a target lane of the road on which the vehicle is traveling on a real scene image;
and displaying the instruction icon on the actual scene image based on the positional relationship between the target lane and the instruction icon;
When the traveling road is curved,
Determining a target lane of the road on which the vehicle is traveling on a real scene image includes:
using a preset image recognition algorithm to recognize a real scene image of the vehicle in motion to obtain an initial recognition result of a target lane on the driving road;
obtaining at least one of a deflection angle of a vehicle body and road coordinates of the travel road on the electronic map;
using at least one of the obtained vehicle body deflection angle and the road coordinates to modify the initial recognition result of the target lane to obtain a target recognition result of the target lane;
A real-scene navigation icon display method, characterized by: generating an indication icon based on the fitted steering navigation path;
drawing the pointing icon using a three-dimensional icon drawing scheme based on the shape of the fitted steering navigation path;
determining world coordinates corresponding to the pointing icon based on location coordinates included in the fitted steering navigation path;
determining image coordinates of the pointing icon on the real-world image using a transformation relationship between a world coordinate system and an image coordinate system;
2. The method of claim 1, wherein: The step of displaying the instruction icon on the actual scene image based on the positional relationship between the target lane and the instruction icon,
displaying the pointing icon on the real-scene image along a steering direction based on image coordinates of the target lane and the pointing icon on the real-world image, respectively;
3. The method of claim 2, wherein: After drawing the pointing icon using a three-dimensional icon drawing scheme based on the shape of the fitted steering navigation path, the method comprises:
further comprising using the 3D icon drawing scheme to draw a steering icon that is the same as the direction of the fitted steering navigation path;
Accordingly, after displaying the pointing icon on the scene image, the method further comprises displaying the steering icon above the pointing icon;
3. The method of claim 2, wherein: The step of performing a fitting process on the steering navigation path includes:
fitting the steering navigation path using a Bessel curve algorithm;
2. The method of claim 1, wherein: determining a road on which the vehicle travels by matching the positioning coordinates of the vehicle with an electronic map;
obtaining a set of positioning coordinates of the vehicle within a preset time and performing data filtering on the set of positioning coordinates;
and matching a filtered set of positioning coordinates with the electronic map to determine a road on which the vehicle travels.
2. The method of claim 1, wherein: Determining a target lane of the road on which the vehicle is traveling on a real scene image includes:
using a pre-trained neural network model to determine a target lane of the road on which the vehicle is traveling on a real-world image of the vehicle, comprising:
wherein said neural network model is trained based on sample images of different image parameters, said image parameters including image brightness and image contrast;
2. The method of claim 1, wherein: A real-view navigation icon display device,
a driving road determination module for determining a driving road of the vehicle by matching the positioning coordinates of the vehicle with an electronic map;
a steering navigation path determination module for determining a steering navigation path of the vehicle based on the driving road;
an icon generation module for performing a fitting process on the steering navigation path and generating an instruction icon based on the fitted steering navigation path;
a target lane determination module for determining a target lane of the road on which the vehicle is running on a real scene image;
an icon display module for displaying the instruction icon on the actual scene image based on the positional relationship between the target lane and the instruction icon;
When the traveling road is curved,
Determining a target lane of the road on which the vehicle is traveling on a real-world image includes:
Using a preset image recognition algorithm, recognize the real scene image while the vehicle is running, obtain the initial recognition result of the target lane on the driving road, and obtain the vehicle body deflection angle and the electronic obtaining at least one of road coordinates on a map; using at least one of the obtained vehicle body deflection angle and the road coordinates to modify an initial recognition result of the target lane; get the target recognition result of the target lane,
A real-scene navigation icon display device characterized by: at least one processor;
a memory communicatively coupled to the at least one processor;
Instructions executable by the at least one processor are stored in the memory, and the instructions enable the at least one processor to execute the real-world navigation icon display method according to any one of claims 1 to 7 . , executed by the at least one processor;
An electronic device characterized by: A non-transitory computer-readable storage medium having computer instructions stored thereon,
The computer instructions cause the computer to execute the real-scene navigation icon display method according to any one of claims 1 to 7 ,
A non-transitory computer-readable storage medium characterized by: A computer program,
The computer program causes a computer to execute the real-scene navigation icon display method according to any one of claims 1 to 7 ,
A computer program characterized by:

Images