FR3011923A1 - METHOD FOR GUIDING A USER OF A MOTOR VEHICLE, CORRESPONDING SYSTEM AND MOTOR VEHICLE - Google Patents

Abstract

The invention relates to a method of guiding a user of a vehicle along a route, said vehicle being equipped with a camera for capturing images of its environment. According to the invention, the guidance information at a given point of the route is generated by performing the following steps: - capture of at least one image of the vehicle environment, - extraction of at least one visual primitive in said at least one image, and - comparing said at least one visual primitive with a plurality of predetermined visual primitives, said guidance information being generated from the results of the comparison.

Description

TECHNICAL FIELD OF THE INVENTION The invention relates to the field of guiding a user of a motor vehicle. The invention more particularly relates to a method for guiding a user of a motor vehicle equipped with at least one camera for capturing images of the vehicle environment. It also relates to an onboard guidance system on board a motor vehicle, as well as a motor vehicle. State of the art Guidance systems have become standard equipment in today's vehicles. These systems typically include a position sensor, for example a GPS sensor, a map or geographical database and a guidance calculation unit for generating guidance information. The position sensor provides the current position of the vehicle. When the user wishes to be guided to a given destination, he enters the destination address into the system. The computing unit then determines, from the current position of the vehicle, the destination address and map or geographic data, the route to follow.

The guidance system then generates, as and when moving the vehicle, guiding information to follow this route. However, in some situations it may be impossible or difficult to enter a destination address in the guidance system. This is for example the case when the system does not recognize the address because it is not listed in its database, or when the destination does not have an address or when the destination is accessible by paths not listed in system maps. It may also happen that the accuracy at the end of the route is not sufficient to allow precise final guidance. This may be the case when the destination address is a campaign location comprising a plurality of dwellings that do not have street numbers. Current systems do not guide the user on the last meters of the route to arrive in front of the desired dwelling. An object of the invention is to propose a method or guidance system in which a destination can be defined otherwise than by means of a geographical address.

Another object of the invention is to provide a method or guidance system for guiding a user in areas not listed on cards.

Another object of the invention is to provide a method or guidance system for guiding a user accurately to a predetermined arrival point.

OBJECT OF THE INVENTION According to the invention, there is provided a guidance method based on an acquisition of visual primitives of the route to follow. For guidance, the vehicle generates guidance information by taking images of its environment independently of geographic data, then extracting visual primitives from the captured images and comparing them to visual primitives obtained in a previous trip over the route. The invention relates to a method for guiding a user of a vehicle along a route, said vehicle being equipped with at least one camera for capturing images of the environment of said vehicle, said method comprising a guidance phase during which guidance information is generated. According to the invention, to generate said guidance information at a given point of the route, the following steps are carried out: - capture of at least one image of the vehicle environment, - extraction of at least one visual primitive in said at least one image, and comparing said at least one visual primitive with a plurality of predetermined visual primitives, said guidance information being generated from the results of the comparison. The guidance is based on the extraction of visual primitives in images of the vehicle environment. This is called visual guidance. The predetermined visual primitives are acquired during a previous journey along said route. The method therefore comprises, prior to said guidance phase, a prior phase of acquisition of visual primitives to acquire said predetermined visual primitives during a first path along said route, said prior acquisition phase comprising the following steps: a plurality of images of the vehicle environment along the route, and - extracting visual primitives in said images. This method can be very useful for guiding the user towards his final destination, for example to travel the last kilometer or the last few hundred meters when the final destination is away from the mapped road network. This method is for example used to guide the user when the final destination is accessible only by unmapped paths. It can then be used to guide the user on the path separating the last mapped route and the final destination.

According to a particular embodiment, the preliminary phase of acquisition of the visual primitives is carried out by a first vehicle and the guiding phase is carried out by a second vehicle, the visual primitives acquired during the preliminary phase by the first vehicle being transmitted to the first vehicle. second vehicle before the guidance phase. For example, Mr. A travels with his vehicle and then transmits the acquired primitives to Mr. B so that he can be guided to redo the same route. According to a particular embodiment, the visual primitives acquired during the prior phase of acquiring primitives are stored in an electronic file. In the previous example, Mr. A then transmits this electronic file to Mr. B. According to a particular embodiment, the route being defined between a starting point and an arrival point, the electronic file further comprises the GPS position. starting point and / or point of arrival. According to a particular embodiment, the entire path is not performed by visual guidance. The guidance may comprise a first step of guidance by geographical location to bring the vehicle to the starting point of the route to be performed in visual guidance and then a visual guidance step by comparison of visual primitives as defined above.

Thus, according to a particular embodiment, the guidance phase comprises a prior step of guidance by geographical location to bring the vehicle to the starting point of the route.

In this embodiment, the user is for example guided in a traditional way by geographical location of his vehicle to a given geographical point on approaching the final destination, for example to a geographical point from which the The vehicle must leave the mapped road network to reach the final destination, and then guided by visual guidance to the final destination. Geographic location guidance is provided from position information from a position sensor or from motion information (eg, speed, steering angle, or acceleration) from dedicated sensors of the vehicle. The invention also relates to an on-board guidance system on board a motor vehicle for guiding the user of said vehicle, said vehicle being equipped with at least one camera for capturing images of the environment of the vehicle, which system comprises a guiding unit adapted to implement the method as defined above, the generated guidance information being transmitted to the user on a human / machine interface of the system or the vehicle.

The invention also relates to a motor vehicle equipped with at least one camera for capturing images of its environment, comprising said system. Other advantages may also appear to those skilled in the art upon reading the examples below, illustrated by the appended figures, given for illustrative purposes. BRIEF DESCRIPTION OF THE FIGURES FIG. 1 represents a flow chart in accordance with the method of the invention; FIG. 2 represents a schematic view of a first embodiment of a guidance system according to the invention; and FIG. 3 is a schematic view of a second embodiment of a guidance system according to the invention. Detailed Description of the Invention The invention provides guidance by acquiring visual primitives from the vehicle environment along a route. This method is preferably used on short routes, near the final destination. An application of the method of the invention is as follows: Mr. A wishes to invite Mr. B to his country house. This house is located at the end of a non-mapped road in Mr. B's navigation system. To guide Mr. B, Mr. A starts an acquisition of visual primitives on the final part of the journey, for example between the point where the vehicle is to leave the mapped road network and his country house. This point is for example identified by its GPS position. He launches this acquisition where he goes to his country house.

During this acquisition, the vehicle's camera captures images of the vehicle environment as it approaches the country house. An onboard system of the vehicle extracts from these images visual primitives that identify the position of the vehicle relative to its environment, particularly with respect to landscapes. At the destination, the embedded system stores the extracted visual primitives in a dedicated file. This file also advantageously contains the GPS position of the point from which vision guidance has been launched. This point is the starting point of the route to be covered by visual guidance. Mr. A transmits the file to Mr. B by any type of means, for example by electronic mail or by sending a USB key containing the file. On the agreed day of his visit, Mr. B loads this file into the vehicle's on-board system. According to the invention, the vehicle of Mr. B includes image capture equipment and an on-board navigation system substantially equivalent to those of the vehicle of Mr. A to be able to correctly use the file of Mr. A.

The onboard system of the vehicle of Mr. B reads at first the GPS position of the starting point of the route which will be to go by visual guidance. The on-board navigation system of Mr. B's vehicle then determines the route to be followed to reach the GPS position of the departure point of the route to be traveled by visual guidance. If the GPS position of the departure point of the route to be traveled by visual guidance is not contained in the file, Mr B enters it into his on-board navigation system, Mr A. having previously provided it. This position may possibly be in the form of an address listed on the cards. Mr. A can then simply communicate this address orally to Mr. B so that he can enter it in the on-board system of his vehicle. The onboard navigation system of the vehicle of Mr. B then generates by conventional means of navigation a route and the guidance information 25 to reach this point of departure. Once at this point of departure, the navigation system of Mr. B's vehicle switches to visual guidance.

The camera of Mr. B's vehicle is then turned on and captures images of the vehicle environment. Captured images are processed to generate visual primitives. These visual primitives are compared to the visual primitives contained in the file provided by Mr. A and guidance information is generated based on the results of the comparison. Thus, Mr. B is guided throughout his journey to the arrival in front of the country house of Mr. A. Moreover, it is guided without breaking in the transmission of guidance information at the time of switching between the mode of "classic" guidance and visual guidance mode.

A flow diagram of the "visual" guidance method according to the invention is illustrated in FIG. 1. This method comprises a phase P1 for acquiring visual primitives specific to the route to be carried out in visual guidance. This acquisition phase is performed by a vehicle equipped with at least one camera to capture images of the vehicle environment and processing means captured images. This acquisition phase is performed during a first movement of the vehicle along the route to be performed later in visual guidance. According to a first El, the vehicle's camera captures images of the vehicle environment from a starting point, preferably located by a GPS position. This GPS position is provided by the vehicle navigation system and stored in a memory thereof. This starting point is for example the point on the map from which the route no longer corresponds to a mapped route. This is for example the GPS position from the vehicle must leave a departmental road to engage on a path not mapped. The images are captured at regular intervals at a predetermined frequency.

According to a step E2, the captured images are processed so as to extract at regular intervals visual primitives suitable for characterizing the route followed by the vehicle. The method used to extract visual primitives is for example those described in the document entitled "Outdoor autonomous navigation using monocular vision" of E.Royer, J.Bom, M.Dhome, B.Thuilot, M.Lhuillier, F.Marmoiton; Intelligent Robots and Systems, 2005 (IROS 2005); International Conference on Issue Date: 2-6 Aug. 2005 in page (s): 1253 - 1258 print ISBN: 0-7803-8912-3 INSPEC Accession Number: 8750308 Digital Object Identifier: 10.1109 / IROS.2005.1545495 Date of Current Version: December 05, 2005 These visual primitives are, for example, sets points of interest detected in the captured images. The extraction of visual primitives is then similar to a detection of points of interest in the captured images. The visual primitives generally used in this type of situation are points of interest: a list of points of interest is for example described in the publication "International Journal of Computer Vision" June 2000, Volume 37, Issue 2, pp 151- In this way, a list of visual primitives is generated. The primitives of the list are preferably ordered so as to be able to quickly match the primitives of the list with those observed at the moment by the vehicle's camera. The classification aims to allow a pairing as fast as possible. According to a step E3, these visual primitives are then stored in a file stored in an internal or external memory of the navigation system of the vehicle. The GPS position of the starting point of the route is advantageously stored in this file. Advantageously, other parameters are recorded during the acquisition phase P1. For example, the GPS coordinates of the vehicle are also regularly recorded as well as parameters relating to the movement of the vehicle, for example the speed of the vehicle or its acceleration. These parameters are recorded to complement visual guidance when the captured images have few visual primitives. These parameters are provided by proprioceptive sensors of the vehicle.

Advantageously, the file also stores the GPS coordinates of the arrival point of the route. After the phase of acquisition of the visual primitives, the method of the invention comprises a guidance phase P2 in which the visual primitives acquired during the phase P1 are used to generate guidance information to redo the route. Between these two phases, the method of the invention comprises a step E4 for transmitting the visual primitive file when the vehicle responsible for following the route (vehicle to be guided) is not the vehicle having made the acquisition phase P1 visual primitives. This is the case in the example described above where Mr. A transmits to Mr. B the primitive file for it to reach his country house. If it is the same vehicle that travels the route a second time, this transmission step is absent from the process. In the rest of the explanation, the term "second vehicle" is used to designate the vehicle that must be guided to reproduce the route corresponding to the visual primitives of the file. The "first vehicle" refers to the vehicle that has completed the acquisition phase Pl.

To follow the same route, the second vehicle captures with its camera images of its visual environment at a step E5. The second vehicle preferably captures these images at the same frequency as the first vehicle.

According to a step E6, the navigation system of the second vehicle processes the images captured in the previous step so as to extract visual primitives in the captured images.

The processing applied to the images for the extraction of visual primitives is identical to that used in step E2 of the acquisition phase. According to a step E7, the navigation system of the second vehicle compares the visual primitives extracted in step E2 and those extracted in step E6. As they are extracted, the visual primitives are compared to the stored visual primitives. According to a step E8, the navigation system generates guidance information based on the results of the comparison. During this step, the navigation system calculates the difference between a current visual primitive and a stored visual primitive. It determines the difference between the two primitives and derives a guidance information. For example, if we consider that the visual primitives generated for a captured image or group of images are the points of interest in these images, if the points of interest of the current image coincide with the points of interest. interest of the stored image, the guidance information may be to continue in the same direction. Any deviation of trajectory of the vehicle with the trajectory initially recorded during the recording of primitives results in a shift of the position of points of interest. Depending on this offset, a correction transmitted to the driver or to the vehicle guidance system enables the vehicle to be repositioned in the correct trajectory. When guiding information is generated, it is then emitted, during a step E9, to the attention of the user of the vehicle. It is for example displayed on a screen of the vehicle or issued as a voice message. Steps E5 to E9 are repeated until the vehicle reaches the GPS position corresponding to the arrival point of the route.

As a variant, the steps E5 to E9 are repeated until the last extracted visual primitive corresponds to the last visual primitive of the file. A guidance system capable of implementing the acquisition phase P1 and the guidance phase P2 of the method of the invention is shown diagrammatically in FIG. 2. The guidance system is embedded in a vehicle equipped with minus a camera 10 and a GPS sensor 11.

The camera 10 has the function of capturing images of the environment of the vehicle. It is for example positioned in the front bumper, the grille of the vehicle, at the top of the windshield or anywhere to ensure observation of the scene surrounding the vehicle with wide field of view. The GPS sensor 11 measures the position of the vehicle from satellite signals and delivers the GPS position 5 of the vehicle. It can also deliver a vehicle speed value. The guidance system comprises a module 12 for extracting visual primitives. This module receives the images captured by the camera 10 and processes them according to the extraction method mentioned in step E2 to extract visual primitives. It is used both for the extraction of visual primitives from the learning phase and the extraction of visual primitives from the guidance phase. The extracted visual primitives are recorded as an electronic file in an internal or external memory 13 of the navigation system. Advantageously, the GPS positions of the departure point and the arrival point of the route are also stored in the file. These positions are provided by a GPS navigator of the system. The recording of the visual primitives and positions in the memory 13 is performed under the control of this control circuit not shown. The file F stored in the memory 13 will serve as a reference for the visual guidance phase. This file can be extracted from the navigation system via an appropriate interface and transmitted to the navigation system of the second vehicle. For the guidance phase, this file is loaded into the memory of the navigation system of the second vehicle.

The navigation system further comprises a module 16 for comparing the visual primitives stored in the memory 13 and visual primitives from the current images captured by the camera 10 of the second vehicle. This module 16 is used during the guiding phase. The module 16 generates guidance information 16 according to the results of the comparison. The navigation system further comprises a conventional GPS navigator 14 connected to the GPS sensor 11 and provided with a memory 15 storing cards for positioning the vehicle on the road network. The GPS navigator 14 typically generates guidance information for the user of the vehicle.

The guidance information delivered by the module 16 and those delivered by the GPS navigator are provided to a module 17 for selecting the guidance information to be displayed. According to a preferred embodiment, as long as the vehicle has not reached the starting point of the route to be carried out in visual guidance, the module 17 selects the guidance information from the GPS navigator 14. When the point of departure is reached, the module 17 selects the guidance information from the module 16. The selected guidance information is displayed on a screen 18 or transmitted by a voice message. All of the elements 12 to 17 form a so-called guiding unit 20 able to implement the steps of the method of the invention. According to an improved embodiment illustrated by FIG. 3, the vehicle is equipped with proprioceptive sensors 19 able to measure the displacement of the vehicle. These sensors measure, for example, the speed, acceleration or rotation of the wheels of the vehicle. Data from proprioceptive sensors is intended to supplement visual guidance when the captured environment has few visual primitives. These data are recorded in the file F, with the visual primitives and possibly the GPS positions of the starting point and the arrival point, during the acquisition phase P1. During the guidance phase, the data from the sensors proprioceptives 19 are transmitted to the module 16. These data are compared to those contained in the file F when the primitive visual guidance is not sufficient because of the low number of visual primitives on the path. Guiding information is then generated from this comparison to guide the vehicle this method of guidance is temporary and allows to wait for the appearance of new points of interest. The guidance method of the invention can therefore be implemented by the conventional navigation system of a vehicle by adding the modules 12, 13, 16 and 17. These modules can also be implemented in a "smart phone" or a multimedia terminal. The latter is then connected to the camera, to the GPS navigation system and, where appropriate, to the proprioceptive sensors of the vehicle. The guidance information is then transmitted by the navigation system of the vehicle or the terminal. If the vehicle does not have a navigation system, the guidance method of the invention can be fully implemented in the terminal or smart phone. In addition to the example given above to guide Mr. B to Mr. A's cottage, there are many applications to the guidance method of the invention. This method can be used in the context of tourist visits to bring the user to a specific point on a tourist site to make him discover a panorama or a particular view. This process can also be used by two friends who follow each other by car a few minutes apart. The first vehicle performs the learning phase and transmits the data to the second vehicle. The second vehicle is thus guided to follow the route taken by the first vehicle. The second vehicle no longer has to "stick" the first vehicle to follow it.

As indicated above, this guidance method is rather intended to complement conventional guidance methods based on GPS navigation. It is therefore rather used at the end of the route or on particular small areas (near a tourist site for example). This method makes it possible to achieve guidance accuracy of the order of a few centimeters. Although the invention has been described in connection with various particular embodiments, it is obvious that it is not limited thereto and that it comprises all the technical equivalents of the means described and their combinations if they fall within the scope of the invention.

Claims (9)

REVENDICATIONS1. A method of guiding a user of a vehicle along a route, said vehicle being equipped with at least one camera for capturing images of the environment of said vehicle, said method comprising a guiding phase (P2) during which guidance information is generated, characterized in that, for generating (E8) said guidance information at a given point of the route, the following steps are performed: - capturing (E5) at least one image of the vehicle environment, - extracting (E6) at least one visual primitive in said at least one image, and comparing (E7) said at least one visual primitive with a plurality of predetermined visual primitives, said information of guidance being generated from the results of the comparison. 20 2. Method according to claim 1, characterized in that it comprises, before said guiding phase (P2), a preliminary phase (P1) of acquisition of visual primitives to acquire said predetermined visual primitives during a first path along said route, said prior phase comprising the following steps - capturing (El) a plurality of images of the vehicle environment along the route, and 30 - extracting (E2) visual primitives in said images . 3. Method according to claim 2, characterized in that the preliminary phase (P1) of acquisition of visual primitives is performed by a first vehicle and the guide phase (P2) is performed by a second vehicle, the visual primitives acquired during of the preliminary phase by the first vehicle being transmitted to the second vehicle before the guiding phase. 4. Method according to claim 2 or 3, characterized in that the visual primitives acquired during the prior phase of acquiring primitives are stored in an electronic file (F). 5. Method according to claim 4, characterized in that, the route being defined between a starting point and an end point, the electronic file (F) further comprises the GPS position of said starting point and / or said arrival point. 6. Method according to any one of the preceding claims, characterized in that, the route being defined between a starting point and an arrival point, the guiding phase (P2) comprises a preliminary step of guiding by geographical location. to bring the vehicle to the starting point of the route. 7. Method according to claim 6, characterized in that the guidance by geographical location is carried out from position information from a position sensor mounted on the vehicle. 8. On-board guidance system on a motor vehicle for guiding the user of said vehicle, said vehicle being equipped with at least one camera (10) for capturing images of the environment of the vehicle, characterized in that it comprises a guiding unit (20) adapted to implement the method according to any one of claims 1 to 8, the generated guiding information being transmitted to the user on a man / machine interface of the system or the vehicle . 15 9. Motor vehicle equipped with at least one camera for capturing images of its environment, characterized in that it further comprises a system according to claim 8. 20