WO2020155075A1

WO2020155075A1 - Navigation apparatus and method, and related device

Info

Publication number: WO2020155075A1
Application number: PCT/CN2019/074310
Authority: WO
Inventors: 郭菁睿; 李令言; 姜浩然; 莫浩桔
Original assignee: 华为技术有限公司
Priority date: 2019-01-31
Filing date: 2019-01-31
Publication date: 2020-08-06
Also published as: CN113330278A; CN113330278B

Abstract

A navigation apparatus and method, and a related device, wherein the navigation apparatus (10) comprises a first processor (101), and a neural network processor (104) coupled to the first processor. The first processor (101) is used to acquire a plurality of images on a road segment collected by a camera module (102) and instruct the neural network processor (104) to identify the plurality of images, wherein the navigation apparatus (10) moves along the road segment, and the road segment comprises a navigation target position. The neural network processor (104) is used to identify a plurality of images to obtain a plurality of signs, and each sign is used to mark a position on the road segment. The first processor (101) is further used to determine a plurality of route reference points in the road segment according to the plurality of signs, and generate a navigation route of the road segment on the basis of the plurality of route reference points, and the navigation route is used to guide a user towards the navigation target position. The navigation apparatus may guide the user to find a vehicle.

Description

Navigation device, method and related equipment

Technical field

This application relates to the field of navigation technology, and in particular to a navigation device, method and related equipment.

Background technique

In the existing navigation system, the user can usually obtain the navigation route between the navigation start address and the navigation target address through online map or offline map navigation, and conduct real-time navigation based on the navigation route and current location information To reach the navigation destination address. However, in some scenarios or environments, the server may not be able to provide a navigation map, or it can provide but cannot provide an accurate navigation map. For example, in large-scale shopping malls, office buildings, stations, parking lots, amusement parks, schools and other places, users are often unable to find the target address quickly due to reasons such as large places, inconspicuous markers, and inaccurate positioning on maps. Unable to quickly return to a certain target address to find storage items or deal with related affairs, which makes the user experience poor.

Take the application scenario of a user looking for a car in a parking lot as an example. Most of the newly-built residential quarters, shopping malls, office buildings and other large buildings include parking lots in the construction plan. The building area of the parking lot is also getting bigger and bigger, and the number of parking spaces is easily increased. thousand. Due to the rapid development speed, it is difficult for the management facilities inside the parking lot to keep up. There is a problem that the direction is difficult to distinguish due to the large internal space of the parking lot and the similar environment and markers, which often makes it difficult for car owners to find quickly when returning to the parking lot. Own vehicle has given birth to a new "difficult to find a car" problem. In this case, many parking lots have introduced a reverse car search method based on the divided parking area and parking space code, such as manual car search, swiping card search, monitoring car search and other measures to help car owners find cars. However, the above-mentioned car-finding methods are all supported by a large amount of manpower or a large amount of hardware facilities, which cost a large amount of money, and for users, it is still troublesome to use, and the effect is not good. The user's "active memory behavior" (active swiping card) is inseparable from the swiping card to find a car. Once the owner gets off the car in a hurry, he does not pay attention to the parking position after getting off the car, and does not actively swipe the card, so he can only blindly search around based on memory .

Therefore, there is an urgent need to provide a navigation method with strong functionality and practicability to solve the problem that the user cannot quickly find the navigation target address in some environments or places.

Summary of the invention

The embodiment of the present invention provides a navigation device, a method and related equipment to guide a user to find a navigation target location.

In the first aspect, an embodiment of the present invention provides a navigation device, which may include: a first processor, and a neural network processor coupled to the first processor, wherein the first processor is configured to Acquiring multiple images on a road segment collected by the camera module, and instructing the neural network processor to recognize the multiple images, wherein the navigation device moves along the road segment, and the road segment includes a navigation target position; The neural network processor is configured to recognize the multiple images to obtain multiple signs, and each sign is used to mark a position on the road section; the first processor is also configured to , Determining a plurality of route reference points in the road section, and generating a navigation route of the road section based on the plurality of route reference points, and the navigation route is used to guide the user to the navigation target location.

In the embodiment of the present invention, the first processor in the navigation device acquires multiple images on the road segment moved by the navigation device collected by the camera module, and instructs the neural network processor to recognize the multiple images on the road segment, Finally, the first processor determines the route reference points in the road section according to the signs recognized by the neural network processor, and generates a navigation route based on the route reference points, so as to help the user to easily find the navigation target location according to the navigation route when returning. That is, the embodiment of the present invention realizes the navigation function of finding the target address only by using the navigation device, without a lot of manpower and material resources, no network, and no need to lay out a lot of hardware equipment, and the cost is low; and the whole process does not require user's cumbersome operation and does not rely on The user's active memory, map data of the site environment, network signals, etc. are convenient and feasible, which greatly improves the user's navigation experience and effects.

In a possible implementation, the device further includes the camera module, configured to collect the multiple images on the road section. In a possible implementation manner, the device further includes a memory coupled to the first processor and the camera module; the memory is configured to store the multiple images collected by the camera module.

In the embodiment of the present invention, multiple images on the road section moved by the navigation device are collected through the camera module in the navigation device, and stored in the memory. The first processor instructs the neural network processor to perform the processing on the multiple images on the road section. Finally, the first processor determines the route reference point in the road segment according to the signs recognized by the neural network processor, and generates a navigation route based on the route reference point to help the user to easily find the navigation route according to the navigation route when returning Navigate to the target location. That is, the embodiment of the present invention realizes the navigation function of finding the target address only by using the navigation device, without a lot of manpower and material resources, no network, and no need to lay out a lot of hardware equipment, and the cost is low; and the whole process does not require user's cumbersome operation and does not rely on The user's active memory, map data of the site environment, network signals, etc. are convenient and feasible, which greatly improves the user's navigation experience and effects.

In a possible implementation manner, the device further includes a measurement module coupled to the first processor: the measurement module is configured to measure the navigation device at any two of the multiple route reference points The movement direction and the number of steps between adjacent route reference points; the first processor is further configured to generate navigation assistance information of the navigation route according to the direction and the number of steps.

In the embodiment of the present invention, the measurement module in the navigation device measures the direction and the number of steps between any two adjacent route reference lines on the moving section of the navigation device, so that the first processor can use the direction and the number of steps Determine the navigation assistance information of the navigation route to reduce the cumulative error of the route and improve the accuracy. For example, the navigation route pushed by the navigation device to the user includes multiple route reference points, as well as the direction and number of steps between the route reference points calculated based on the pedestrian dead reckoning PDR technology.

In a possible implementation, the multiple signs are M signs, the multiple route reference points are N signs in the M signs, N is less than or equal to M, and N and M are greater than An integer of 1, the navigation route includes the N signs. Optionally, the navigation route further includes a time stamp corresponding to at least one route reference point. For example, the neural network processor recognizes M signs from multiple images on the road section, where M is an integer greater than 0; the first processor is specifically configured to: determine from the M signs N signs, the N signs are determined as N route reference points on the road section; N is an integer greater than 0 and less than or equal to M; generated according to the N route reference points and the corresponding time stamp Navigation route.

In the embodiment of the present invention, the first processor in the navigation device selects N from the M signs recognized by the neural network processor as the route reference points on the road section, and according to the timestamps of the N route reference points Generate navigation routes. That is to say, the embodiment of the present invention can eliminate some useless or insignificant signs, and use signs with better and more significant indications as the route reference points on the final navigation route, thereby ensuring more accurate navigation and better results. .

In a possible implementation manner, the multiple signs are M signs, the multiple route reference points are L reference areas, L is less than or equal to M, and N and M are integers greater than 1, the The navigation route includes the L reference areas, and each reference area includes at least one sign. Optionally, the navigation route further includes a time stamp corresponding to at least one route reference point. For example, the neural network processor recognizes M signs from multiple images on the road section, where M is an integer greater than 0; the first processor is specifically configured to: determine according to the M signs L reference areas; where L is an integer greater than 0 and less than M, any one of the reference areas includes at least one of the signs; a route reference point is determined in each of the L reference areas , Obtain L route reference points; generate a navigation route according to the L route reference points and the corresponding time stamp.

In the embodiment of the present invention, the first processor in the navigation device determines L reference areas from the M signs recognized by the neural network processor, and finally determines a route reference point in each reference area. That is, the user's line of sight is taken as a reference area. In each reference area, a sign is selected as the route reference point, and some of the signs with repeated indications are removed, and redundant information is removed, which not only ensures that the user has the sight range Route reference points can also ensure better navigation effects.

In a possible implementation manner, the device further includes a second processor coupled to the first processor; the first processor is further configured to send the navigation route to the second processor The second processor is also used to push the navigation route to the user.

In the embodiment of the present invention, the first processor can be used as a coprocessor with lower power consumption, and the second processor can be used as a main processor with normal power consumption. After the coprocessor generates the navigation route required by the user, and When it is judged that the user reaches the return navigation starting address, the main processor is awakened and the judgment result is sent to the main processor; the main processor can push the navigation route to the user through multimedia methods such as text, image, and voice to ensure The user can easily find the navigation target address, and can ensure the low power consumption of the navigation device.

In a possible implementation manner, the road section further includes a navigation start position, and the navigation start position is located at the end position of the road section.

In the embodiment of the present invention, the neural network processor can be instructed by the first processor in the navigation device to recognize the end position of the road section according to the image collected by the camera module. For example, if the navigation target position is the starting position on the road section, it is the parking lot. For parking spaces, the navigation start position is the end position used to collect and recognize images on the road section, and is the position where the user exits the parking lot. Then the neural network processor can identify the user’s parking point and the exit point of the parking lot, and according to According to the recognition result, it is judged whether the user returns to the parking lot to find a parking space, so that the first processor can wake up the second processor at the right time to push the navigation route to the user, help the user find his car easily, and greatly improve the user search The experience and effects of car navigation.

In a possible implementation manner, the second processor is further configured to: receive instruction information sent by an in-vehicle system, where the instruction information is used to indicate that the in-vehicle system has reached the navigation target location; The target position is located at the starting position for collecting and recognizing images on the road section; the first processor is awakened according to the instruction information to acquire the multiple images on the road section collected by the camera module. Optionally, the navigation target location is a parking space of the vehicle in a parking lot.

The embodiment of the present invention can be applied to the scene of finding a car in a parking lot. Through the interaction between the navigation device and the on-board system, the on-board system can use the vehicle's own camera system to capture the parking location before the vehicle enters the parking lot to find a parking space. The parking space image, and the parking space is identified, and then sent to the navigation device in this application through communication methods such as Bluetooth or Wi-Fi, and then the second processor in the navigation device wakes up the first processor to control the camera module to start collecting road sections Multiple images on the. For example, when the navigation device is a smart phone, smart bracelet or smart glasses that the user carries, before the user drives the vehicle into the parking lot and arrives at the parking space, it is inconvenient to use the navigation device to image the parking lot environment and parking space. Acquisition and recognition. At this time, the image can be collected through the front or rear camera in the vehicle driven by the user, and recognized through the on-board system; and between the vehicle reaching the parking space of the parking lot and the user exiting the parking lot Because the user will carry a navigation device, it is more convenient to identify the signs on the road section. This ensures that the navigation device can start collecting images at the right time, thereby ensuring the completeness and accuracy of the sign recognition on the road section, helping users find their car easily, and greatly improving the user experience and effect of car-finding navigation .

In a second aspect, an embodiment of the present invention provides a navigation method applied to a navigation device, which may include: the method includes: acquiring a plurality of images on a road section, wherein the navigation device moves along the road section, and The road segment includes a navigation target location; the multiple images are identified to obtain multiple signs, each of which is used to mark a position on the road segment; and multiple route reference points in the road segment are determined according to the multiple signs , And generate a navigation route of the road segment based on the multiple route reference points, and the navigation route is used to guide the user to the navigation target location.

In a possible implementation, the method further includes: measuring the direction and number of steps the navigation device moves between any two adjacent route reference points among the multiple route reference points; And the number of steps to generate navigation assistance information of the navigation route.

In a possible implementation, the multiple signs are M signs, the multiple route reference points are N signs in the M signs, N is less than or equal to M, and N and M are greater than An integer of 1, the navigation route includes the N signs.

In a possible implementation manner, the multiple signs are M signs, the multiple route reference points are L reference areas, L is less than or equal to M, and N and M are integers greater than 1, the The navigation route includes the L reference areas, and each reference area includes at least one sign.

In a possible implementation manner, the navigation route further includes a time stamp corresponding to at least one route reference point.

In a possible implementation manner, the method further includes: pushing the navigation route to the user.

In a possible implementation manner, the method further includes: receiving instruction information sent by an in-vehicle system, where the instruction information is used to indicate that the in-vehicle system has reached the navigation target position; the navigation target position is located in the The starting position of the road section; triggering the execution of the acquiring of multiple images on the road section according to the indication information.

In a third aspect, the present application provides a navigation device, which has the function of implementing any of the above-mentioned navigation methods. This function can be realized by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned functions.

In a fourth aspect, the present application provides a terminal, the terminal includes a processor, and the processor is configured to support the terminal to perform a corresponding function in a navigation method provided in the second aspect. The terminal may also include a memory, which is used for coupling with the processor and stores necessary program instructions and data for the terminal. The terminal may also include a communication interface for the terminal to communicate with other devices or communication networks.

In a fifth aspect, the present application provides a computer storage medium that stores a computer program that, when executed by a processor, implements the navigation method process described in any one of the above-mentioned second aspects.

In a sixth aspect, an embodiment of the present invention provides a computer program, the computer program includes instructions, when the computer program is executed by a computer, the computer can execute the navigation method flow described in any one of the second aspect.

In a seventh aspect, the present application provides a chip system, which includes a processor, configured to implement the functions involved in the process of the navigation method described in any one of the second aspects. In a possible design, the chip system further includes a memory for storing program instructions and data necessary for the navigation method. The chip system can be composed of chips, or include chips and other discrete devices.

Description of the drawings

Figure 1 is a schematic structural diagram of a navigation device provided by an embodiment of the present invention;

FIG. 2 is a schematic diagram of identifying multiple images on a road section according to an embodiment of the present invention;

Fig. 3 is a schematic diagram of parking lot navigation provided by an embodiment of the present invention;

Figure 4 is a schematic diagram of another parking lot navigation provided by an embodiment of the present invention;

Figure 5 is a schematic structural diagram of another navigation device provided by an embodiment of the present invention;

FIG. 6A is a schematic structural diagram of another navigation device provided by an embodiment of the present invention;

6B is a schematic structural diagram of yet another navigation device provided by an embodiment of the present invention;

FIG. 7 is a distribution diagram of parking spaces in a parking lot provided by an embodiment of the present invention;

FIG. 8 is a schematic diagram of identifying multiple images on another road section according to an embodiment of the present invention;

9 is a schematic diagram of identifying directions and steps between reference points of adjacent routes according to an embodiment of the present invention;

FIG. 10 is another schematic diagram of multiple image recognition on a road section according to an embodiment of the present invention:

FIG. 11 is a schematic diagram of reference area division according to an embodiment of the present invention;

FIG. 12 is a schematic diagram of path push provided by an embodiment of the present invention;

FIG. 13 is a hardware structure diagram of a neural network processor provided by an embodiment of the present invention;

14 is a schematic flowchart of a navigation method provided by an embodiment of the present invention;

FIG. 15 is a schematic structural diagram of another navigation device provided by an embodiment of the present invention.

detailed description

The embodiments of the present invention will be described below in conjunction with the drawings in the embodiments of the present invention.

The terms "first", "second", "third" and "fourth" in the specification and claims of this application and the drawings are used to distinguish different objects, not to describe a specific order . In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes unlisted steps or units, or optionally also includes Other steps or units inherent to these processes, methods, products or equipment.

Reference to "embodiments" herein means that a specific feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art clearly and implicitly understand that the embodiments described herein can be combined with other embodiments.

The terms "component", "module", "system", etc. used in this specification are used to denote computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to, a process, a processor, an object, an executable file, an execution thread, a program, and/or a computer running on a processor. Through the illustration, both the application running on the computing device and the computing device can be components. One or more components may reside in processes and/or threads of execution, and components may be located on one computer and/or distributed among two or more computers. In addition, these components can be executed from various computer readable media having various data structures stored thereon. A component can be based on a signal having one or more data packets (for example, data from two components interacting with another component between a local system, a distributed system, and/or a network, such as the Internet that interacts with other systems through signals) Communicate through local and/or remote processes.

First, some terms in this application are explained to facilitate the understanding of those skilled in the art.

(1) The parking lot is a place for vehicles to park. There are simple parking lots that only draw parking grids without management and charge, and there are also paid parking lots equipped with entry and exit barriers, parking guards and hourly cashiers. Modern parking lots often have automated time-toll collection systems, closed-circuit television and video recorder systems. The flat parking lot (also known as the square type) has a certain area of land, divided into passages and parking spaces by traffic markings, and equipped with traffic facilities such as pointing arrows and signs. Its parking methods include vertical (at right angles to the passage), parallel (parallel to the passage), diagonal and staggered arrangements. The parking lot in this application may include an underground parking lot, a road parking lot, a three-dimensional parking lot, etc. The application does not limit the specific form of the parking lot.

(2) Artificial Intelligence (AI) is the use of digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. operating system. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making. Research in the field of artificial intelligence includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, and basic AI theories.

(3) Convolutional Neural Network (CNN) is a multi-layer neural network. Each layer is composed of multiple two-dimensional planes, and each plane is composed of multiple independent neurons. Each neuron shares weights, and the number of parameters in the neural network can be reduced through weight sharing. At present, in convolutional neural networks, the convolution operation performed by the processor usually converts the convolution of the input signal characteristics and the weight into a matrix multiplication operation between the signal matrix and the weight matrix. In the specific matrix multiplication operation, the signal matrix and the weight matrix are divided into blocks to obtain multiple fractal signal matrices and fractal weight matrices, and then the multiple fractal signal matrices and fractal weight matrices are subjected to matrix multiplication and accumulation operations.

(4) Inertial measurement unit (IMU) is a device that measures the three-axis attitude angle (or angular rate) and acceleration of an object. It generally includes three single-axis accelerometers and three single-axis gyroscopes. Among them, The accelerometer is used to detect the acceleration signal of the object in the independent three-axis of the carrier coordinate system, and the gyroscope is used to detect the angular velocity signal of the carrier relative to the navigation coordinate system, and measure the angular velocity and acceleration of the object in the three-dimensional space, and then calculate the object Gesture. IMUs are mostly used in equipment that requires motion control, such as automobiles and robots, and are generally installed on the center of gravity of the object to be measured.

(5) Pedestrian Dead Reckoning (PDR) mainly uses Inertial Measurement Unit (IMU) in a beacon-free environment to sense the acceleration, angular velocity, magnetic force, and pressure of a person in the process of traveling. And use these data to calculate the step length and direction of the marching personnel, so as to achieve the purpose of positioning and tracking the personnel. The main processes involved are gait detection, step length and direction calculation.

(6) Optical character recognition (Optical Character Recognition, OCR) refers to the process of analyzing and recognizing image files of text data to obtain text and layout information. Generally divided into two steps: text positioning, that is, finding the position of the text in the image; text recognition, that is, identifying the found text. Text positioning may also include some binarization and correction steps.

(7) Fast object detection (Faster-RCNN) can be used to solve complex computer vision problems and achieve good results. The idea is that using conv+pooling in the neural network does not change the relative position of the features in the image. Among them, RCNN (Regions with CNN) target detection can apply the strategy of region proposal to the convolutional neural network, and training from bottom to top can be used to locate the target and image division. R-CNN uses the Selective Search algorithm to extract (propose) possible regions of interest (regions of interest, RoIs), and then uses standard CNN to classify each extracted region.

(8) Navigation is an operation used to determine the location of the target object and guide the user to the location. The navigation technology of this embodiment can be used to guide the user to the location of his vehicle, that is, to guide the user to find a car. It should be noted that the vehicles or vehicles involved in this implementation generally refer to vehicles, including but not limited to motor vehicles or non-motor vehicles. Motor vehicles may include automobiles, electric vehicles, motorcycles, smart cars, battery cars or tractors, etc.; non-motor vehicles include but are not limited to bicycles, bicycles, or scooters.

In order to facilitate the understanding of the embodiments of the present invention, the following will further analyze the specific technical problems to be solved by the embodiments of the present invention in combination with the application scenarios of car finding in the parking lot and several common car finding methods. Common methods and application scenarios for finding cars include the following: traditional methods, car owners generally find cars through brain memory combined with some special landmarks in the parking lot; or, car owners use their mobile phones to take pictures and record the parking space and the surrounding environment. Find your car according to the parking space recorded by the photo and the instructions inside the parking lot. The shortcomings of the above-mentioned traditional methods: completely dependent on the user's memory, and usually due to inaccurate memory, unclear instructions, and complex underground garage environment, which consumes a lot of time, energy and physical strength, and the effect is not good.

Scenario 1: Global Positioning System (GPS) navigation and positioning to find a car: GPS technology is widely used for navigation and positioning, but the signal strength of GPS signals in the underground or indoors is weak, and the positioning effect is not ideal. Therefore, indoor positioning usually uses Wi-Fi positioning technology, Bluetooth positioning technology, and geomagnetic positioning technology. Among them, Wi-Fi positioning is the most widely used in indoor positioning. Wi-Fi positioning technology can usually be used to realize the function of finding a car in a parking lot: after the user stops parking, mark the location of the parking, when the user picks up the car and leaves, the current location is determined through Wi-Fi positioning, and then according to the internal map of the parking lot, Determine the navigation route for the pickup.

The disadvantage of the above-mentioned GPS navigation and positioning: Wi-Fi positioning technology requires the deployment of a large number of routers in the environment. If the Wi-Fi system deployed in the parking lot is only used for car search, it will bring about low efficiency and high cost. Similarly, when geomagnetic positioning and Bluetooth positioning are used to find a car in a parking lot, it will also cause high costs.

Scenario 2: IoV navigation and positioning to find a car: When the user parks, use the car's on-board system to obtain the sign image corresponding to the current parking space; identify the location information of the parking space according to the above-mentioned sign image, and send it to the mobile phone through the Internet of Vehicles; the user returns When looking for a car, the mobile phone scans the QR code of the pedestrian exit to obtain the current location (each QR code information corresponds to a preset location on the map of the parking lot); according to the location of the parking space and the current location of the user, and parking The map inside the field, planning the navigation path.

The disadvantages of the above-mentioned IoV navigation and positioning: all of the users' vehicles are required to use the IoV and rely on the internal map of the parking lot, which is costly and cumbersome to operate.

Scenario 3: Finding a car through a parking lot camera: Each parking space is equipped with one or more cameras. When the user parks, the camera near the parking space records the license plate and sends the corresponding information of the license plate and the parking space to the server; when the user picks up the car, use it The mobile phone queries the license plate information to get the corresponding parking location; finally use the parking lot map to search for the parking location, and get the car navigation route.

The above-mentioned disadvantages of finding a car through parking lot cameras: relying on a large number of cameras, relying on the map of the parking lot, and requiring users to use APP to input license plate information, which is costly and cumbersome. This application proposes a method, and the comparison with the above method is shown in Table 1 below.

Table 1

Therefore, in view of the above-mentioned traditional methods and three common application scenarios, the car-finding problem that the embodiment of the present invention mainly solves specifically includes the following: in view of the high cost of the prior art, the cumbersome operation, and the poor user experience, a low cost is provided. , Navigation device with easy operation and good user experience. That is, this application needs to provide a "low-cost, zero-operation, and micro-sensing" fast car-finding solution that does not rely on environmental deployment and does not require user operations.

It is understandable that the technical problems solved by this application include, but are not limited to, the navigation problem in the above-mentioned parking lot car-finding scene. The above-mentioned application scene is only an exemplary application scene in this application. The navigation device, method and related The application scenarios of the equipment can also include, for example, in large-scale shopping malls, office buildings, stations, parking lots, amusement parks, schools and other places, the navigation in the process of turning back and looking for a certain landmark, commodity, storage place, etc., The following scenes and example descriptions will not be listed and repeated.

Based on the foregoing, the following describes the application of the navigation device provided in the embodiment of the present invention in a car-searching scene in a parking lot as an example. Please refer to FIG. 1, which is a schematic structural diagram of a navigation device provided by an embodiment of the present invention. The navigation device 10 may include: a first processor 101 and a neural network processor coupled to the first processor 104; Optionally, the navigation device 10 may also include a camera module 102 and a memory 103; wherein the camera module 102 is used to collect multiple images on the road section and store them in the memory, and the navigation device along the The road segment is moving, and the road segment includes the navigation target position. For example, the road segment includes the road segment between the parking space where the vehicle reaches the parking lot and the user exiting the parking lot, and the navigation target position is the parking space where the vehicle reaches the parking lot. The camera module 101 may be a rear camera, a front camera, a side camera, a rotating camera or a reversible camera of the terminal device. The main function of the camera module 101 is to track and capture the environment image of the navigation device on the moving road section in real time during the navigation process, for example, the environment image from the parking space to the elevator. Optionally, the camera module 102 is a camera module with low power consumption and low-resolution frames, or the camera module 102 operates in a low-power, low-resolution frame mode when collecting multiple images on a road section. That is, as long as the multiple images on the road section collected by the camera module 102 satisfy the requirement that the neural network processor 104 can recognize the signs, the power consumption can be reduced.

The memory 103 is used to store multiple images on the road section collected by the camera module 102. The memory 103 can be used as a shared memory in the navigation device 10. That is, the first processor 101, the camera module 102, and the neural network processor 104 can all be connected to the memory 103, and the memory 103 can be used for the first processor 101 and the camera module. 102 and the neural network processor 104 provide memory space required for related image collection, recognition, and generation of navigation routes.

The first processor 101 is used to instruct the neural network processor 104 to recognize multiple images on the road section. Optionally, the first processor 101 can be used as a co-processor of the navigation device 10. When the navigation device 10 is in a standby or dormant state, it still supports the normal operation of the camera module 102 and the neural network processor 104, so as to ensure that there is more information on the road. Collection and recognition of images. For example, the first processor 101 in this application may be a smart sensor hub (Sensor hub), and the Sensor hub is based on a low-power microcontroller unit (MCU) and a lightweight real-time multitasking operating system (Real Time Operating System (RTOS) is a solution that combines software and hardware, which can connect and process data from various sensor devices. Due to its low power consumption and lightweight features, it can ensure that the navigation device 10 The power consumption is for the user to search and navigate the car.

A neural network processor (Neutral Processing Unit, NPU) 104 is used to identify the multiple images to obtain multiple signs, and each sign is used to mark a position on the road section. For example, the signs are various types in a parking lot. Logo images, such as numbers, letters, straight arrows, diagonal arrows, turning arrows, designated patterns, exits (EXIT), boxes, entrances (ENTRANCE), signs, etc. As shown in Figure 2, Figure 2 is a schematic diagram of the recognition of multiple images on a road section provided by an embodiment of the present invention. In the multiple images on multiple road sections, the marks are H050, H084, K108, K120, L005, L100, M008, M070, elevator 3, etc. It is understandable that the neural network processor 104 needs to collect a large number of parking lot signs that need to be learned in advance, and then perform image preprocessing on the parking lot signs that need to be learned, and then use the deep learning engine to train the processed images to obtain Deep learning model, and train the classifier, and finally use the trained model and classifier to recognize multiple images on the received road section.

The first processor 102 is further configured to determine a route reference point in the road section according to the sign recognized by the neural network processor 104, and generate a navigation route based on the route reference point. That is, the first processor 102 determines the route reference point of the user between the parking space and the exit parking place according to the sign recognized by the neural network processor 104, and the route reference point can be understood as an indication node on the navigation route. For example, when the recognized signs include: "A027, A029, A030, A032, B005, B006, B007, B008, arrows, ENTRANCE entrance", you can follow the preset rules (e.g., exclude signs that are closer, Eliminate signs with unclear indications, etc.), determine the route reference point from the signs as "A027, A030, A032, B005, B008, arrow, ENTRANCE exit", and finally generate it according to the route reference point according to the principle of reverse car search The navigation route is: "ENTRANCE→Arrow→Exit→B008→B005→A032→A030→A027". The embodiment of the present invention does not specifically limit how to determine the route reference point from the sign, nor does it specifically limit how to generate the navigation route according to the route reference point.

Optionally, the aforementioned neural network processor 104 can also be integrated in the first processor 101 as a part of the first processor 101; it can also be coupled to the aforementioned first processor 101 and can implement multiple Other functional chips or modules in the chip recognized by the logo in the image; similarly, the functions executed by the first processor 101 can also be located on one chip or distributed on multiple different functional chips, which are not specifically described in the embodiment of the present invention. limited.

In the embodiment of the present invention, through the camera module in the navigation device, multiple images on the road section between the vehicle reaching the parking space of the parking lot and the user exiting the parking lot are collected and stored in the memory. The first processor instructs the neural network to process The device recognizes multiple images on the road section. Finally, the first processor determines the route reference point in the road section according to the signs recognized by the neural network processor, and generates a navigation route based on the route reference point to help the user in When returning to find a car, you can easily find the navigation target location according to the navigation route. That is to say, the embodiment of the present invention realizes the navigation function of finding a car only by using the navigation device, without a lot of manpower and material resources, and no need to lay out a lot of hardware equipment, and the cost is low; and the whole process does not require the user's cumbersome operation and does not rely on the user's active memory The map data and network signal of the parking lot are convenient and feasible, which greatly improves the user experience and effect of car navigation.

In a possible implementation, the multiple signs are M signs, the multiple route reference points are N signs in the M signs, N is less than or equal to M, and N and M are greater than An integer of 1, the navigation route includes the N signs. For example, the neural network processor 104 recognizes M signs from multiple images on the road section; the first processor 101 determines N signs from the M signs, and determines the N signs as the N route reference points on the road segment, and a navigation route is generated according to the N route reference points and the corresponding time stamp. As shown in FIG. 3, FIG. 3 is a schematic diagram of parking lot navigation provided by an embodiment of the present invention, in which the neural network processor 104 recognizes M signs from multiple images on multiple road sections, for example, A014, B004, A015, B005, A016, B006, A017, B007, A018, B008, A019, B009, A020, B010, F008, E001, E003, E004, D001, D002, D003, D004, D005, D006; then M is equal to 24 . The first processor 101 determines N signs from the above M signs, for example, B004, B005, B006, B007, B008, B009, B010, E001, E003, D001, D002, D003, D004, D005, D006, then N is equal to 15, and its determination principle is to determine one of the signs with the same indicating function. Finally, based on the 15 route reference points and the corresponding time stamps, combined with the principle of reverse pickup, the navigation route is generated as: "D006→D005→D004→D003→D002→D001→E003→E001→B010→B009→B008→B007 →B006→B005→B004".

In a possible implementation manner, the multiple signs are M signs, the multiple route reference points are L reference areas, L is less than or equal to M, and N and M are integers greater than 1, the The navigation route includes the L reference areas, and each reference area includes at least one sign. For example, the neural network processor 104 recognizes M signs from a plurality of images on the road section; the first processor 103 determines L reference regions according to the M signs, and among the L reference regions A route reference point is determined in each reference area; a navigation route is generated according to the determined L route reference points and the corresponding time stamp. As shown in FIG. 4, FIG. 4 is a schematic diagram of another parking lot navigation provided by an embodiment of the present invention, in which the neural network processor 104 recognizes M signs from multiple images on multiple road sections, which are A014, B004 , A015, B005, A016, B006, A017, B007, A018, B008, A019, B009, A020, B010, F008, E001, E003, E004, D001, D002, D003, D004, D005, D006; then M is equal to 24. The first processor 101 determines L reference areas according to the above M signs, which are reference area 1, reference area 2, reference area 3, reference area 4, reference area 5, reference area 6, reference area 7, and reference area 8. Then L=8; and determine a route reference point in each of the above L reference areas, for example, the route reference point in reference area 1 is B005, the route reference point in reference area 2 is B007, and the reference area The route reference point in reference area 3 is B009, the route reference point in reference area 4 is B010, the route reference point in reference area 5 is E011, the route reference point in reference area 6 is D001, and the route reference point in reference area 7 It is D003, the route reference point in the reference area 8 is D005, and the determination principle is to determine a route reference point from a reference area within the visible range. Therefore, the first processor 101 generates the navigation route according to the determined L route reference points and the corresponding timestamps, combined with the reverse pick-up principle: "D005→D003→D002→D001→E011→B010→B009→B007→ B005".

In a possible implementation manner, the road section further includes a navigation start position, and the navigation start position is located at the end position of the road section. For example, the navigation starting position is the elevator where the user exits the parking lot, and the recognized signs are "EXIT", "Elevator B", etc. In the embodiment of the present invention, the first processor in the navigation device may be used to instruct the neural network processor to recognize the end position of the road section according to the image collected by the camera module. For example, if the navigation target position is the start of the collection and the start of the other image on the road section The location is the parking space of the parking lot, and the navigation start location is the end location of the end collection and different images on the road segment. It is the location where the user exits the parking lot. Then the neural network processor can identify the user’s parking spot And the point of exiting the parking lot, and based on the recognition results, determine whether the user returns to the parking lot to find a parking space, so that the first processor can wake up the second processor at the right time to push the navigation route to the user, helping the user find their own The car greatly improves the user experience and effect of car-finding navigation.

Please refer to FIG. 5, which is a schematic structural diagram of another navigation device according to an embodiment of the present invention. Based on the structure of the navigation device 10 in FIG. 1, the navigation device 10 may further include a measurement module 105 coupled with the first processor 101. Wherein, the measurement module 105 is used to measure the direction and the number of steps between any two adjacent route reference points of the navigation device in the road section, and store the direction and the number of steps in the memory 103. Optionally, the measurement module can be an inertial measurement unit (IMU), which is a combination of sensors (accelerometer, gyroscope, and magnetometer) to measure and report at least one of speed, direction, and gravity Electronic equipment. It can achieve 3 degrees of freedom (DoF, degrees of freedom) or 6DoF data measurement. Therefore, IMU can be used to calculate the user's direction and number of steps on the first road based on pedestrian dead reckoning PDR technology. It can be understood that the aforementioned IMU may be used as a part of the first processor 101, or may be coupled to the functional chip or an on-chip module of the first processor 101, which is not specifically limited in the embodiment of the present invention.

The memory 103 is also used to store the direction and the number of steps measured by the measurement module 105. The memory 103 can be used as a shared memory in the navigation device 10, for example, for storing images collected by the camera module 102 and storing the directions and steps measured by the measurement module. The first processor 101 is further configured to generate navigation assistance information of the navigation route according to the direction and the number of steps. For example, while providing navigation routes for car owners, they also provide navigation assistance information for them to indicate the direction and number of steps between any two adjacent route reference points, enhancing the indication effect of navigation routes, and helping car owners find navigation easily target location.

In the embodiment of the present invention, the measurement module in the navigation device measures the direction and the number of steps of the navigation device between any two adjacent route reference lines on the road section between the parking space of the parking lot and the user exiting the parking lot. , So that the first processor determines the navigation assistance information of the navigation route according to the direction and the number of steps, so as to reduce the accumulated error of the route and improve the accuracy. For example, the navigation route pushed by the navigation device to the user includes multiple route reference points, as well as the direction and number of steps between the route reference points calculated based on the pedestrian dead reckoning PDR technology.

Please refer to FIG. 6A, which is a schematic structural diagram of another navigation device provided by an embodiment of the present invention. Based on the structure of the navigation device 10 in FIG. 1 above, the navigation device 10 may also include a second processor 106 coupled to the first processor 101, and the power consumption of the first processor 101 is lower than that of the second processor 106. Power consumption; optionally, as shown in FIG. 6B, FIG. 6B is a schematic structural diagram of another navigation device provided by an embodiment of the present invention. Based on the structure of the navigation device 10 in FIG. 6A, the navigation device 10 may also include For the measurement module 105, refer to FIG. 5 for details. It is assumed that the location where the user exits the parking lot is the starting position of the navigation. Wherein, the first processor 101 is further configured to send the navigation route to the second processor. In the embodiment of the invention, since the first processor has lower power consumption than the second processor, the first processor 101 can be used as a co-processor to perform operations related to navigation route generation with lower power consumption. The second processor 106 can be used as the main processor to perform other related operations that require higher computing capabilities, and therefore can process other processes than generating the navigation route.

The second processor 106 is configured to push the navigation route to the user. For example, when the first processor 101 determines according to the recognition result that the user is currently returning to the navigation starting position (that is, the end position of the road section) where the user exited the parking lot, then it is determined that the user currently needs to return to pick up the car, so the recognition result is It is sent to the second processor 106, and the second processor 106 pushes the navigation route to the user. There is no need for the user to independently obtain the navigation route, which is more convenient and faster, and improves the user experience. In a possible implementation manner, the road section further includes a navigation start position.

In a possible implementation, the second processor 106 is further configured to receive instruction information sent by the on-board system, where the instruction information is used to indicate that the on-board system has reached the navigation target position; according to the instruction information Wake up the first processor to control the camera module to collect multiple images on the road section. That is, the interaction between the vehicle-mounted system and the navigation device 10 triggers the navigation device 10 to enter the navigation mode, that is, starts to collect and identify multiple images on the road section. Among them, the vehicle-mounted system may be a vehicle-mounted system on a vehicle, which can control a camera device inside or outside the vehicle to collect and identify surrounding environment images to identify whether the vehicle currently reaches a parking space in a parking lot.

It can be understood that the second processor 106 is also used to run general operating system software and control the operation of the first processor 101 under the action of the general operating system software. Further, the second processor 106 is also used for processing and completing other related calculation processing and control in the navigation process. It is understandable that the navigation device in Figure 1, Figure 6A or Figure 6B can be located in a terminal (such as a smart phone, tablet, smart wearable device, etc.), a smart camera device (smart camera, smart camera, smart tracking device) , Intelligent monitoring equipment, aerial drones, etc., this application will not list them all.

In the embodiment of the present invention, through the interaction between the navigation device and the on-board system, the on-board system can use the vehicle's own camera system to capture the parking space image of the parking place and identify the parking space before the vehicle enters the parking lot to find the parking space. , And then send it to the navigation device in this application through communication methods such as Bluetooth or Wi-Fi, and then wake up the first processor through the second processor in the navigation device to control the camera module to start collecting multiple images on the road section. For example, when the navigation device is a smart phone, smart bracelet or smart glasses that the user carries, before the user drives the vehicle into the parking lot and arrives at the parking space, it is inconvenient to use the navigation device to image the parking lot environment and parking space. Acquisition and recognition. At this time, the image can be collected through the front or rear camera in the vehicle driven by the user, and recognized through the on-board system; and between the vehicle reaching the parking space of the parking lot and the user exiting the parking lot Because the user will carry a navigation device, it is more convenient to identify the signs on the road section. This ensures that the navigation device can start collecting images at the right time, thereby ensuring the completeness and accuracy of the sign recognition on the road section, helping users find their car easily, and greatly improving the user experience and effect of car-finding navigation .

The following takes the navigation device as a smart phone or a part of a smart phone as an example, combined with the structure of the navigation device provided in Figure 6A, based on the application scenario of the user parking in the parking lot and car-finding navigation, how the smart phone is implemented in the application Route navigation. In the following embodiment 1, the first processor 101 is the Sensor hub, the camera module 102 is the rear camera of the mobile phone, the memory 103 is shared memory, and the second processor is the main processor, such as a central processing unit (CPU), as an example. According to the functions performed by the functional modules in the smart phone in time sequence, the following steps can be included.

Step 1: Scene recognition (camera collects the image and puts it in the shared memory, the Sensor hub calls the NPU to recognize the image, and the NPU returns the identification information to the Sensor hub). When the user gets off the car, when the camera captures the image of the underground garage environment (such as a large number of parked cars), it can be considered that the user enters the underground garage parking route recording mode; when the image captured by the camera is no longer an underground garage Image of the environment, or when it recognizes that the user enters the elevator/escalator, it can be determined that the user exits the underground garage parking route recording mode; when the camera acquires the image of the underground garage environment again, it determines that the user enters the return to the car-finding mode.

Step 2: Image collection and recognition (the camera collects the image and stores it in the shared memory, calls the NPU to recognize the image, and the NPU returns the recognition information to the Sensorhub). When entering the underground garage recognition mode, the camera starts to collect images of the surrounding environment (ie multiple images on the road section). When exiting the underground garage mode, the camera stops collecting images of the surrounding environment. Assume that Figure 2 is on the road section collected by the camera. A collection of multiple images. FIG. 7 is a distribution diagram of parking spaces in a parking lot provided by an embodiment of the present invention. The location information usually marked on the ground, pillars and other objects of the parking lot is the sign in this application, such as area A, Area B, parking space A01, etc. Among them, the NPU can recognize the signs in the image through recognition algorithms such as Single Shot MultiBox Detector (SSD)/fast target detection (fast RCNN) and optical character recognition (OCR), including those entering the parking lot. Signs (navigation target location), parking lot signs (route reference points) in the road section, and elevator number (navigation starting position) the user takes.

Step 3: Record the departure route (Sensor hub records the information returned by the NPU, as shown in Table 2 below).

Table 2

时间戳 Timestamp	信息information

时间戳1Timestamp 1	H050 H050

时间戳2Timestamp 2	H084H084
……	……
时间戳nTimestamp n	M044M044

Record the route according to the timestamp to determine the route reference point in the road segment. You can simply give the area code such as HKLM. For details, you can also give: H050→H080→H118→K010→K055→K112→L120→L077→L020→M005→ M030→M044.

Step 4: Push the pick-up path (implemented by the second processor). When parking is completed, it recognizes the user from underground garage mode→other modes (elevator, elevator, pedestrian exit), or starts to pick up the car, namely recognizes the user’s other modes→underground garage mode, the first processor wakes up the second processor and pushes User’s parking spot: Push the number corresponding to the parking position (generally considered that the first number identified is the parking space); push the elevator the user takes when entering the mall (label 3); it is recommended that the user take the third elevator to pick up the car, and Push the pick-up path (the reverse of HKLM); the way to notify users can be: text notification, voice notification, multimedia notification, etc. Specifically, the second processor may run software, including but not limited to operating system and application software, to generate a user interface, and use text or image to make the notification on the user interface, which is not limited in the embodiment of the present invention.

The embodiment of the present invention uses low-power devices to collect information, such as a low-power camera, to obtain path information from the user after parking the car to leaving the parking lot. When the user finds a car, the mobile phone automatically pushes the path information.

The following takes the navigation device as a smart phone or a part of a smart phone as an example, combined with the structure of the navigation device provided in Figure 6B, based on the application scenarios of the user parking in the parking lot and car-finding navigation, how the smart phone is implemented in the application Route navigation. In the second embodiment below, the first processor 101 is the Sensor hub, the camera module 102 is the rear camera of the mobile phone, the memory 103 is the shared memory, the measurement module is the MCU, and the second processor is the main processor. The functions performed by the functional modules in the mobile phone in time sequence may include the following steps.

Step 1: Scene recognition, which is the same as the first embodiment. Step 2: Image acquisition and recognition are the same as the first embodiment above. Step 3: Record the departure route (Sensor hub gets the information returned by the NPU, as shown in Table 3 below).

table 3

时间戳Timestamp	位置position	步数和角度Steps and angle

时间戳1Timestamp 1	A100A100	00
时间戳2 Timestamp 2	A012A012	30度，100步30 degrees, 100 steps
……	……	……
时间戳nTimestamp n	D032D032	20度，200步20 degrees, 200 steps

As shown in FIGS. 8 and 9, FIG. 8 is a schematic diagram of identifying multiple images on another road section provided by an embodiment of the present invention, and FIG. 9 is a schematic diagram of an adjacent route reference point provided by an embodiment of the present invention. Schematic diagram of direction and step identification. After identifying the starting point A100 (route reference point), start counting steps, use PDR (pedestrian dead reckoning) for trajectory and direction, and record the number of steps from A100 to A102 when A012 (route reference point) is identified Direction; similarly, when a route reference point is identified, the number of steps and directions from the previous route reference point to the next route reference point are recorded; optionally, the count is cleared every time a route reference point is identified, you can Reduce cumulative error.

Step 4: Push the pick-up path. The process of picking up the car is the same as in the first embodiment. In addition to text and voice, the way of informing the user can also display a graphical trajectory, which is not specifically limited in the embodiment of the present invention.

The following takes the navigation device as a smart phone or a part of a smart phone as an example, combined with the structure of the navigation device provided in Figure 6B, based on the application scenarios of the user parking in the parking lot and car-finding navigation, how the smart phone implements this application In the route navigation. In the third embodiment below, the first processor 101 is the Sensor hub, the camera module 102 is the rear camera of the mobile phone, the memory 103 is the shared memory, the measurement module is the MCU, and the second processor is the main processor. The functions performed by the functional modules in the mobile phone in time sequence may include the following steps.

The first step: scene recognition, the same as the first embodiment. The second step: image acquisition and recognition, the same as the first embodiment. The third step: record the signs in each image (multiple images on the road section), as shown in FIG. 10, which is another schematic diagram of multiple image recognition on the road section provided by the embodiment of the present invention: Set the route reference point in the range, as shown in FIG. 11, which is a schematic diagram of the reference area division provided by the embodiment of the present invention. For example, every 15 parking spaces is a reference area. If there are 120 parking spaces in the H area, the H area is shared 8 reference areas. According to the parking position identified in the second step, such as H050, record the reference area H4 where H050 is located.

The fourth step: record the route to leave the underground garage, the same as the first or second embodiment. Step 5: Push the pick-up path. Start to pick up the car, identify the user's other modes→basement mode, that is, if the first processor recognizes that the elevator to pick up the car is the same as the elevator entering the mall, the second processor will wake up and push the user's parking space to the user (recognized parking on the road section) Location H050) and pick up route. The way to notify the user can be: text notification, voice notification, etc.

If it is recognized that the elevator to take the car is different from the elevator entering the mall, then according to the reference area H4 obtained when multiple images on the road section are collected, and the elevator 4 entering the basement, the server obtains the best path from elevator 4 to H4: OJIH, As shown in Fig. 12, Fig. 12 is a schematic diagram of pushing the pick-up route (for example, including navigation route and navigation assistance information) provided by an embodiment of the present invention; pushing the user's parking space H050, pushing the best route OJIH; if the elevator is not recognized, then Push the user's parking space, push the elevator number when entering, push the pick-up path.

The way to obtain the pick-up path can use the data collected from the local database to obtain the optimal path from each route reference point to each elevator entrance. Among them, the pedometer is used to determine the path with the least number of steps as the best path, or the path with the least number of key nodes is the best path. The user enters the basement and automatically downloads the route library of the optimal path.

Based on the structure of the navigation device 10 in FIG. 1, FIG. 6A and FIG. 6B, FIG. 13 is a hardware structure diagram of a neural network processor provided by an embodiment of the present invention, in which the core part of the NPU is the arithmetic circuit 1403, and the controller 1404 controls the arithmetic circuit 1403 to extract matrix data in the memory 103 and perform multiplication operations. In some implementations, the arithmetic circuit 1403 includes multiple processing units (Process Engine, PE). In some implementations, the arithmetic circuit 1403 is a two-dimensional systolic array. The arithmetic circuit 1403 may also be a one-dimensional systolic array or other electronic circuit capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 1403 is a general-purpose matrix processor.

For example, suppose there is an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit 1403 fetches the weight data corresponding to matrix B from the weight memory 1402 and caches it on each PE in the arithmetic circuit 1403. The arithmetic circuit 1403 fetches the matrix A data and matrix B from the input memory 1401 to perform matrix operations, and the partial result or final result of the obtained matrix is stored in the accumulator 1408.

The unified memory 1406 is used to store input data and output data. The weight data is directly transferred to the weight memory 1402 through the direct memory access controller (DMAC) 1405. The input data is also transferred to the unified memory 1406 through the DMAC1405. That is, the Bus Interface Unit (BIU) 1410 is used to realize the interaction between the Advanced Extensible Interface (AXI) bus and the DMAC 1405 or the instruction fetch buffer (Instruction Fetch Buffer) 1409. The bus interface unit 1410 is used for the instruction fetch memory 1409 to obtain instructions from the external memory 103, and is also used for the storage unit access controller 1405 to obtain the original data of the input matrix A or the weight matrix B from the external memory 103. DMAC1405 is mainly used to transfer input data from external memory 103, such as double-rate synchronous dynamic random access memory (DDR) to unified memory 1406 or to transfer weight data to weight memory 1402 or to transfer input data to input memory In 1401.

The vector calculation unit 1407 includes a plurality of arithmetic processing units, and if necessary, further processes the output of the arithmetic circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison and so on. Mainly used for non-convolutional/fully connected layer (FC) network calculations in neural networks, such as Pooling, Batch Normalization, Local Response Normalization, etc. . In some implementations, the vector calculation unit 1407 can store the processed output vector to the unified buffer 1406. For example, the vector calculation unit 1407 may apply a nonlinear function to the output of the arithmetic circuit 1403, such as a vector of accumulated values, to generate an activation value. In some implementations, the vector calculation unit 1407 generates a normalized value, a combined value, or both. In some implementations, the processed output vector can be used as an activation input to the arithmetic circuit 1403, for example for use in subsequent layers in a neural network.

The instruction fetch buffer 1409 connected to the controller 1404 is used to store instructions used by the storage controller 1404; the unified memory 1406, the input memory 1401, the weight memory 1402, and the instruction fetch memory 1409 are all On-Chip memories. It is understandable that the relevant functions such as the recognition of multiple images on the road section described in this application are all implemented by the relevant functional units in the above-mentioned NPU, and will not be repeated here.

Please refer to FIG. 14, which is a schematic flowchart of a navigation method provided by an embodiment of the present invention. The navigation method is applicable to any one of the navigation devices in FIGS. 1 and 6A and 6B and includes the navigation device. device of. The method may include the following steps S201-S203, and optionally, may also include step S204-step S205. Wherein, step S201: acquiring multiple images on a road section, wherein the navigation device moves along the road section, and the road section includes a navigation target position; step S202: identifying the multiple images to obtain multiple signs, each A mark is used to mark a position on the road section; step S203: according to the multiple signs, determine multiple route reference points in the road section, and generate a navigation route for the road section based on the multiple route reference points , The navigation route is used to guide the user to the navigation target location.

In a possible implementation, the multiple signs are M signs, the multiple route reference points are N signs in the M signs, N is less than or equal to M, and N and M are greater than An integer of 1, the navigation route includes the N signs. In a possible implementation manner, the multiple signs are M signs, the multiple route reference points are L reference areas, L is less than or equal to M, and N and M are integers greater than 1, the The navigation route includes the L reference areas, and each reference area includes at least one sign.

Step S204: Measure the direction and the number of steps the navigation device moves between any two adjacent route reference points among the multiple route reference points; Step S205: Generate the navigation route according to the direction and the number of steps Navigation assistance information.

In a possible implementation manner, the navigation route further includes a time stamp corresponding to at least one route reference point. In a possible implementation manner, the method further includes: pushing the navigation route to the user. In a possible implementation manner, receiving instruction information sent by an in-vehicle system, where the instruction information is used to indicate that the in-vehicle system has reached the navigation target position; the navigation target position is located at the starting position of the road section, That is, when the device is along the cloud on the road section, it starts to collect images and record the starting position; according to the instruction information, trigger execution of the acquisition of multiple images on the road section. In a possible implementation manner, the road section further includes a navigation start position. It should be noted that, for the specific process in the navigation method described in the embodiment of the present invention, please refer to the relevant description in the embodiment of the invention described in Figs. 1 to 13, and will not be repeated here.

Please refer to FIG. 15. FIG. 15 is a schematic structural diagram of another navigation device provided by an embodiment of the present invention. The navigation device 30 may include an acquisition unit 301, an identification unit 302, and a navigation unit 303. Optionally, it may also include a measuring unit 304, an auxiliary unit 305, a pushing unit 306, an indicating unit 307, and a triggering unit 308. The acquiring unit 301 is used to acquire multiple images on the road section; and the identifying unit 302 is used to identify The multiple images are used to obtain multiple signs, and each sign is used to mark a position on the road section; the navigation unit 303 is configured to determine multiple route reference points in the road section according to the multiple signs, and A navigation route of the road segment is generated based on the multiple route reference points, and the navigation route is used to guide the user to the navigation target location.

In a possible implementation manner, the device 30 further includes: a measuring unit 304, configured to measure the movement direction and the number of steps of the navigation device between any two adjacent route reference points among the multiple route reference points ; Auxiliary unit 305 for generating navigation assistance information of the navigation route according to the direction and the number of steps.

In a possible implementation, the multiple signs are M signs, the multiple route reference points are N signs in the M signs, N is less than or equal to M, and N and M are greater than An integer of 1, the navigation route includes the N signs. In another possible implementation manner, the multiple signs are M signs, the multiple route reference points are L reference areas, L is less than or equal to M, and the N and M bits are integers greater than 1, so The navigation route includes the L reference areas, and each reference area includes at least one sign.

In a possible implementation manner, the navigation route further includes a time stamp corresponding to at least one route reference point. In a possible implementation manner, the device further includes: a pushing unit 306, configured to push the navigation route to the user. In a possible implementation manner, the device 30 further includes: an indication unit 307, configured to receive indication information sent by an on-board system, the indication information being used to indicate that the on-board system has reached the navigation target location; The target position is located at the start position of the road section; the trigger unit 308 is configured to trigger the execution of the acquisition of multiple images on the road section according to the indication information. In a possible implementation manner, the road section further includes a navigation start position.

It should be noted that the functions of the relevant units in the navigation device 30 described in the embodiments of the present invention can be referred to the relevant device embodiments described in Figures 1 to 13 and the method embodiments described in Figure 14 Related descriptions will not be repeated here. Each unit in FIG. 15 can be implemented by software, hardware, or a combination thereof. The hardware-implemented units can include circuits and electric furnaces, algorithm circuits or analog circuits, etc. A unit implemented in software may include program instructions, which is regarded as a software product, is stored in a memory, and can be run by a processor to implement related functions. For details, refer to the previous introduction.

An embodiment of the present invention also provides a computer storage medium, wherein the computer storage medium may store a program, and the program includes part or all of the steps of any one of the above method embodiments when executed.

The embodiment of the present invention further provides a computer program, which includes instructions, when the computer program is executed by a computer, the computer can execute part or all of the steps of any navigation method.

In the above-mentioned embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in an embodiment, reference may be made to related descriptions of other embodiments.

It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that this application is not limited by the described sequence of actions. Because according to this application, some steps may be performed in other order or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

In the several embodiments provided in this application, it should be understood that the disclosed device may be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the above-mentioned units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components can be combined or integrated. To another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical or other forms.

The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be realized in the form of hardware or software functional unit.

If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of the present application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , Including several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc., specifically a processor in a computer device) execute all or part of the steps of the above methods in the various embodiments of the present application. Among them, the aforementioned storage media may include: U disk, mobile hard disk, magnetic disk, optical disk, read-only memory (Read-Only Memory, abbreviation: ROM) or Random Access Memory (Random Access Memory, abbreviation: RAM), etc. A medium that can store program codes.

As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, a person of ordinary skill in the art should understand that: The technical solutions recorded in the embodiments are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

A navigation device, characterized by comprising: a first processor, and a neural network processor coupled to the first processor, wherein

The first processor is configured to acquire multiple images on the road section collected by the camera module, and instruct the neural network processor to recognize the multiple images, wherein the navigation device moves along the road section, And the road section includes a navigation target position;

The neural network processor is configured to recognize the multiple images to obtain multiple signs, and each sign is used to mark a position on the road section;

The first processor is further configured to determine multiple route reference points in the road section according to the multiple signs, and generate a navigation route of the road section based on the multiple route reference points, the navigation route Used to guide the user to the navigation target location.
The device according to claim 1, wherein the device further comprises: the camera module, configured to collect the multiple images on the road section.
The device according to claim 1 or 2, wherein the device further comprises a memory coupled to the first processor and the camera module;

The memory is used to store the multiple images collected by the camera module.
The device according to any one of claims 1 to 3, wherein the device further comprises a measurement module coupled to the first processor:

The measurement module is used to measure the direction and the number of steps the navigation device moves between any two adjacent route reference points among the multiple route reference points;

The first processor is further configured to generate navigation assistance information of the navigation route according to the direction and the number of steps.
The device according to any one of claims 1 to 4, wherein the multiple signs are M signs, the multiple route reference points are N signs in the M signs, and N is less than or Equal to M, and N and M are integers greater than 1, and the navigation route includes the N signs.
The device according to any one of claims 1-4, wherein the multiple signs are M signs, the multiple route reference points are L reference areas, L is less than or equal to M, and N and M bits are an integer greater than 1, the navigation route includes the L reference areas, and each reference area includes at least one sign.
The device according to claim 5 or 6, wherein the navigation route further includes a time stamp corresponding to at least one route reference point.
The device according to any one of claims 1-7, wherein the device further comprises a second processor coupled to the first processor;

The first processor is further configured to send the navigation route to the second processor;

The second processor is further configured to push the navigation route to the user.
The device according to claim 8, wherein the second processor is further configured to:

Receiving instruction information sent by an in-vehicle system, where the instruction information is used to indicate that the in-vehicle system has reached the navigation target position; the navigation target position is located at the start position of the road section;

Wake up the first processor according to the instruction information to acquire the multiple images on the road section collected by the camera module.
A navigation method, characterized by being applied to a navigation device, the method comprising:

Acquiring multiple images on a road section, wherein the navigation device moves along the road section, and the road section includes a navigation target position;

Identifying the multiple images to obtain multiple signs, each sign being used to mark a position on the road section;

According to the multiple signs, determine multiple route reference points in the road segment, and generate a navigation route of the road segment based on the multiple route reference points, the navigation route is used to guide the user to the navigation target position.
The method of claim 10, wherein the method further comprises:

Measuring the moving direction and the number of steps of the navigation device between any two adjacent route reference points among the multiple route reference points;

The navigation assistance information of the navigation route is generated according to the direction and the number of steps.
The method according to claim 10 or 11, wherein the multiple signs are M signs, the multiple route reference points are N signs in the M signs, and N is less than or equal to M, And N and M are integers greater than 1, and the navigation route includes the N signs.
The method according to claim 10 or 11, wherein the multiple signs are M signs, the multiple route reference points are L reference areas, L is less than or equal to M, and N and M bits are greater than An integer of 1, the navigation route includes the L reference areas, and each reference area includes at least one sign.
The method according to claim 12 or 13, wherein the navigation route further includes a time stamp corresponding to at least one route reference point.
The method according to any one of claims 10-14, wherein the method further comprises:

Push the navigation route to the user.
The method according to any one of claims 10-15, wherein the method further comprises:

Receiving instruction information sent by an in-vehicle system, where the instruction information is used to indicate that the in-vehicle system has reached the navigation target position; the navigation target position is located at the start position of the road section;

Triggering the execution of the acquiring of multiple images on the road section according to the instruction information.
A computer storage medium, characterized in that the computer storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 10-16 is implemented.
A computer program, characterized in that the computer program includes instructions, which when the computer program is executed by a computer, cause the computer to execute the method according to any one of claims 10-16.