WO2020045345A1

WO2020045345A1 - Sign recognition system and sign recognition method

Info

Publication number: WO2020045345A1
Application number: PCT/JP2019/033315
Authority: WO
Inventors: 臼井　美雅; 朋夫野村
Original assignee: 株式会社デンソー
Priority date: 2018-08-31
Filing date: 2019-08-26
Publication date: 2020-03-05

Abstract

This sign recognition system is a system (1, 21, 31) for photographing a prescribed sign by an on-vehicle camera (4) mounted on a vehicle and recognizing the sign on the basis of the photographed image data thereof, and comprises: an extraction device (11, 25, 34) that recognizes and extracts only a character of a specific type in the sign from the photographed image data; and a sign database (17, 28, 33) that sets information of a character string extracted by the extraction device (11, 25, 34) as attribute information for identifying the sign, and stores sign data including installation position information and attribute information of the sign.

Description

Sign object recognition system and sign object recognition method

Cross-reference of related applications

This application is based on Japanese Patent Application No. 2018-163075 filed on Aug. 31, 2018 and Japanese Application No. 2019-136947 filed on Jul. 25, 2019, the contents of which are described herein. Invite.

The present disclosure relates to a sign object recognition system and a sign object that photograph a predetermined sign object including a guide sign and a signboard for road guidance by an on-vehicle camera mounted on a vehicle and recognize the sign object based on the photographed image data. It relates to an object recognition method.

車両 In vehicles such as automobiles, for example, it is considered that an in-vehicle camera that captures an image of the front in the traveling direction is mounted, and an image captured by the in-vehicle camera is used for driving assistance or the like. For example, Patent Document 1 proposes the following. That is, the presence of a guide sign in front of the road is detected from the image of the vehicle-mounted camera, and the image data of the guide sign portion is processed. Then, characters such as a place name, a direction, a distance, and an intersection display on the destination local plane are recognized, and simplified display is performed on a monitor in the vehicle.

JP 2010-266383 A

By the way, in recent years, there has been an increasing trend toward realization of autonomous driving technology for automobiles, and there is a demand for preparing high-precision road map data for that purpose. In this case, a system has been considered in which an in-vehicle camera mounted on a vehicle captures an image of the front of the vehicle while traveling on a road, and generates a road map based on the captured image, that is, probe data. Alternatively, a system that detects a vehicle position by comparing it with road map data from an image captured by a vehicle-mounted camera has been considered. Here, since the guide signs provided on the roads have different display contents and are installed at appropriate intervals on the main road, the guide signs may be included as landmarks in the road map data. Becomes effective.

However, using the entire photographed image data of the guide sign to specify or identify each of the guide signs provided in a large number of guide signs requires a large amount of data to be handled, and requires processing required for communication and image recognition. Time will be long. Therefore, how to identify the guide sign photographed by the on-vehicle camera becomes a problem, and it is demanded to easily specify the sign.
Therefore, the present disclosure provides a sign object recognition system and a sign object recognition method that can easily execute a process of specifying which sign object in the database is a sign object photographed by the vehicle-mounted camera. The purpose is to: Note that the sign here includes, for example, guide signs and signboards for road guidance.

In the first aspect of the present disclosure, a sign object recognition system that picks up a sign object with an on-board camera mounted on a vehicle and recognizes the sign object based on the shot image data. From the data, an extraction device that recognizes and extracts only a specific type of character in the sign, and information on the character string extracted by the extraction device is used as attribute information for identifying the sign, as the attribute information of the sign. A sign object database for storing sign object data including the installation position information and the attribute information.

According to this, when a predetermined sign is photographed by the on-board camera mounted on the vehicle, only a specific type of character in the sign is extracted from the photographed image data by the extraction device. The sign object data is stored in the sign object database, and the information of the character string extracted by the extraction device is used as the attribute information, and the sign object data includes the installation position information of the sign object and its information. Contains attribute information.

At this time, as the attribute information, information of a character string extracted by recognizing only a specific type of character in the sign can be used to specify or identify each sign. Therefore, the amount of data to be handled in order to specify the sign is significantly reduced, and the communication time for communication and the time required for data processing such as image recognition can be shortened. In this case, since the recognition target is limited to a specific type of character, the dictionary used for recognition can be small, and the processing can be speeded up. As a result, there is an excellent effect that it is possible to easily execute the process of specifying which of the sign objects in the sign object database the sign object photographed by the vehicle-mounted camera has.

The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings. The drawing is
FIG. 1 is a block diagram schematically showing an overall configuration of a system according to the first embodiment, FIG. 2 is a flowchart schematically illustrating a procedure of a process of registering guide sign data performed by the control unit and the processing control device according to the first embodiment; FIG. 3A is a diagram (part 1) illustrating a specific example of a guide sign according to the first embodiment; FIG. 3B is a diagram (part 2) illustrating a specific example of the guide sign according to the first embodiment; FIG. 3C is a diagram (part 3) illustrating a specific example of the guide sign according to the first embodiment; FIG. 3D is a diagram (part 4) illustrating a specific example of the guidance sign according to the first embodiment; FIG. 3E is a diagram (part 5) illustrating a specific example of the guidance sign according to the first embodiment; FIG. 4 is a diagram for explaining a method of processing for extracting only numbers from captured image data according to the first embodiment, FIG. 5 is a diagram for explaining a case where the installation position information of the guide sign data according to the first embodiment is updated and registered, FIG. 6 is a block diagram schematically illustrating an entire configuration of a system according to the second embodiment. FIG. 7 is a flowchart schematically illustrating a procedure of a process of collating guide sign data executed by the control unit and the processing control device according to the second embodiment; FIG. 8 is a diagram for explaining a method of performing collation according to the second embodiment. FIG. 9 is a block diagram schematically illustrating a configuration of an in-vehicle device according to the third embodiment. FIG. 10 is a flowchart schematically illustrating a procedure of a process of collating guide sign data executed by the control unit according to the third embodiment; FIG. 11 is a flowchart schematically illustrating a procedure of a process of collating guide sign data executed by the control unit according to the fourth embodiment; FIG. 12 is a diagram illustrating a specific example of the guide sign according to the fifth embodiment, FIG. 13 is a diagram for explaining a method of processing for extracting a number together with position information from captured image data according to the sixth embodiment.

Hereinafter, some concrete embodiments will be described with reference to the drawings. Note that portions common to a plurality of embodiments are denoted by the same reference numerals, and new illustration and repeated description are omitted. In the following description, the "guidance sign" is a sign that the road manager installs at a required position on the road and performs route guidance, point guidance, attached facility guidance, etc., and the installation position, shape, color, It is installed according to a prescribed format such as the size of characters. In addition, "signboards" are installed in stores or along roads, etc., and are made mainly for commercial purposes for advertising purposes and to be shown to passersby. , A position including a direction and a distance, and the like. The "signs" include information signs and signs for road guidance.

(1) First Embodiment A first embodiment will be described with reference to FIGS. FIG. 1 schematically shows an entire configuration of a guide sign recognition system 1 as a sign recognition system according to the present embodiment. Here, the guide sign recognition system 1 includes a data center 2 and a vehicle-mounted device 3. The data center 2 collects and analyzes data, and generates a guide sign database as a highly accurate sign object database. The in-vehicle device 3 is provided in each of a plurality of vehicles such as a passenger car and a truck traveling on a road, and only one is shown for convenience.

The on-vehicle device 3 mounted on each vehicle includes an on-vehicle camera 4, a position detection unit 5, various on-vehicle sensors 6, a map data storage unit 7, a communication unit 8, a detection data storage unit 9, an operation display unit 10, and a control unit 11. It has. The in-vehicle camera 4 is provided, for example, at the front of a vehicle and configured to photograph at least a road condition ahead in the traveling direction. The position detector 5 has a well-known configuration for detecting the position of the vehicle based on data received by a GPS receiver and the like. The various in-vehicle sensors 6 include sensors for detecting speed information and traveling direction of the own vehicle, that is, information on the direction of the vehicle body.

The map data storage unit 7 stores, for example, road map information for the whole country. The communication unit 8 performs communication with the data center 2 via a mobile communication network or using road-to-vehicle communication or the like. The communication unit 8 functions as a transmission unit 8a as a transmission device and a reception unit 8b as a reception device. As will be described later, the detection data storage unit 9 stores detection data including shooting position information at which a guide sign is estimated, attribute information obtained, and the like. The operation display unit 10 has a switch (not shown), for example, a touch panel or a display, and is operated by a user of the vehicle, for example, a driver, and performs necessary display such as a navigation screen to the user.

The control unit 11 includes a computer and controls the entire on-vehicle device 3. At this time, the control unit 11 captures an image of a road condition ahead by the on-vehicle camera 4 while the vehicle is running. Then, when a guide sign as a sign is detected in the photographed image data, a specific type of characters in the guide sign is obtained from the photographed image data of the guide sign using, for example, a well-known OCR technique. Recognize and extract characters belonging to characters one by one. Further, the extracted character string is used as attribute information for specifying the guide sign. Therefore, the control unit 11 has a function as an extraction device. Details of the attribute information in the present embodiment will be described later.

At the same time, the control unit 11 estimates the position where the guide sign is installed from the own vehicle position, the traveling speed, the traveling direction, and the like detected by the position detection unit 5 at the time of photographing the guide sign as the sign object. The position is taken as the shooting position information. Then, the control unit 11 causes the detection data storage unit 9 to store the detection data including the photographing position information of the guide sign and the obtained attribute information together with data such as the photographing date and time. After that, the communication unit 8 transmits the detection data stored in the detection data storage unit 9 to the data center 2.

On the other hand, the data center 2 includes a communication unit 12, an input operation unit 13, a processing control unit 14, a detection data storage unit 15, a road map database 16, and a guide sign database 17 as a sign object database. The communication unit 12 receives the detection data by communication with the communication unit 8 of each vehicle, and functions as a receiving unit 12a as a receiving device and a transmitting unit 12b as a transmitting device. The input operation unit 13 is for an operator to perform necessary input operations.

The processing control device 14 is mainly composed of a computer, and controls the entire data center 2. At the same time, as described later in detail, the processing control device 14 performs a process of generating road map data and the like, and also performs a process of generating and updating guide sign data (see FIG. 5) as the sign object data. . At this time, the detection data transmitted from each vehicle is collected and temporarily stored in the detection data storage unit 15. At this time, for example, a huge amount of detection data is collected from a large number of ordinary vehicles traveling all over Japan.

道路 The road map database 16 stores high-precision road map data generated by the processing control device 14. The guide sign database 17 stores guide sign data as sign data used for landmark information and the like. As shown in FIG. 5, the guide sign data includes installation position information of a sign including a guide sign installed on each major road in the whole country and a commercial signboard installed near the road, that is, longitude and latitude. , And attribute information for specifying the guide sign. The road map database 16 may include a guide sign database 17 as a sign object database. It is also possible to include sign object data as landmarks in the road map data and to include attribute information in each sign object data.

In the present embodiment, the control unit 11 of the in-vehicle apparatus 3 uses the captured image data of the in-vehicle camera 4 to convert a specific type of character in a guide sign as a sign, as will be described later in the description of the operation, that is, the description of the flowchart. An extraction step of recognizing and extracting only numbers from 0 to 9 is executed. The information of the extracted character string is used as attribute information. In the present embodiment, a specific example is described in which a guide sign is photographed as a sign. At this time, in the process of extracting the number in the guide sign as the sign, the control unit 11 determines that the search for the number from the left to the right of the guide sign from the photographed image data of the guide sign is performed in the up-down direction. Repeat in order. A character string in which numbers are arranged in the extracted order is used as attribute information.

Then, the processing control device 14 of the data center 2 receives and collects the detection data from the vehicle-mounted device 3 of each vehicle, and stores the data in the detection data storage unit 15. At the same time, a sign object data storing step of registering and updating guide sign data in the guide sign database 17 based on the collected detection data is executed. Therefore, the processing control device 14 also has functions as a collection unit 14a as a collection device and a registration unit 14b as a registration device. At this time, as described in the following description of operation, the processing control device 14 registers the same attribute information from the plurality of pieces of detection data received and stored in the detection data storage unit 15 when registering the guide sign data. Collect the detection data that you have. Then, the installation position information is determined based on statistical processing of the shooting position information of the collected detection data, and is configured to be used as guide sign data.

FIGS. 3A to 3E show examples of images of guide signs A1 to A5 installed on a highway, for example, as specific examples of guide signs as sign objects. These guide signs A1 to A5 are for guiding a direction and an exit on an expressway, etc., and are formed by writing white letters mainly on a green background on a square signboard. Specifically, the guide sign A1 shown in FIG. 3A indicates that the distance to the exit of "Yatomi" of the inter number "26" is 2 km.

案内 The guide sign A2 shown in FIG. 3B indicates that the exit to the “Yatomi, Tsushima” area is 1 km, and the exit is connected to National Route 155. The guide sign A3 shown in FIG. 3C indicates that the distance to the exit in the direction of "Yatomi, Tsushima" is 550 m. The guidance sign A4 shown in FIG. 3D indicates that the exit is in the direction of "Yatomi, Tsushima". The guide sign A5 shown in FIG. 3E indicates that the highway radio can be heard from this point at a frequency of 1620 kHz.

Next, the operation of the guide sign recognition system 1 having the above configuration will be described with reference to FIGS. The flowchart of FIG. 2 shows a procedure of processing up to registration of guide sign data, which is executed by the control unit 11 on the in-vehicle apparatus 3 and the processing control apparatus 14 on the data center 2 side, ie, each of the display object recognition methods in the present embodiment. The steps are shown. In FIG. 2, steps S1 to S3 are processes executed by the control unit 11 of the vehicle-mounted device 3 while the vehicle is running. First, in step S1, the front of the vehicle is photographed by the in-vehicle camera 4, and the presence or absence of a guide sign as a sign in the photographed image is constantly monitored. When the guidance sign is photographed by the vehicle-mounted camera 4, in step S2, a process of recognizing and extracting a specific type of character, in this case, a numeral, is performed from the photographed image data, in this case, a still image (extraction step).

As described above, this extraction processing is performed by repeating the search for numbers from the photographed image data from the left to the right of the guide sign in the vertical direction, and arranging the numbers in the extracted order. Will be FIG. 4 shows a processing method when the control unit 11 extracts only numbers from the captured image data of the guide sign. That is, taking the guide sign A2 shown in FIG. 3B as a specific example, first, when tracing from left to right in the upper stage, the number “155” is recognized and extracted. No numbers are recognized for the second and third rows from the top. In the lower part, the number “26” is recognized on the left side, and the number “1” is recognized and extracted on the right side.

Accordingly, the attribute information of the guide sign A2 is a character string composed of six numbers “155261”. Similarly, in the example of FIG. 3, the attribute information of the guide sign A1 shown in FIG. 3A is a character string of “262”. The attribute information of the guide sign A3 illustrated in FIG. 3C is a character string of “15552650”. The attribute information of the guide sign A4 illustrated in FIG. 3D is a character string of “15526”. The attribute information of the guide sign A5 illustrated in FIG. 3E is a character string of “1620”.

Returning to FIG. 2, in the next step S3, the position of the photographed guide sign, that is, photographing position information is specified, and the photographing position information and the character string information, ie, attribute information, obtained in step S2 are detected data. Is transmitted to the data center 2 by the communication unit 8. In this case, regarding the photographing position information, the own vehicle position detected by the position detection unit 5 at the time of photographing the guide sign, and the guide sign obtained from the position and size of the guide sign in the photographed image data. Is estimated based on the distance to

The following steps S4 and S5 are processing executed by the processing control device 14 of the data center 2. In step S <b> 4, in the data center 2, the detection data transmitted from the in-vehicle device 3 is received by the communication unit 12 and written into the detection data storage unit 15. In step S5, statistical processing is performed on the received large number of pieces of detected data, the location where the guide sign is present is specified, and the guide sign data including the information on the position coordinates of the installation position of the guide sign and the attribute information is set as A process of registering in the sign database 17 is performed (a sign object data storing step). In this case, the registration includes not only new registration but also update registration.

When registering the guide sign data, the same attribute information is collected from a large number of detection data collected in the detection data storage unit 15, data having a number in this case, and the position coordinates, which are the photographing position information of the detection data, are statistically processed. The obtained position coordinates are used as true installation position information. The statistical processing at that time can be performed by, for example, excluding data having an outlier, that is, an abnormal value, and then calculating the average, median, mode, and the like of the installation position information. FIG. 5 shows an example in the case where update registration has been performed. The installation position coordinates of the guide sign whose first attribute information is “155261” are updated from (X1, Y1) to (X11, Y11). It shows how it was.

According to the guide sign recognition system 1 and the recognition method of the present embodiment, the following effects can be obtained. That is, in the in-vehicle apparatus 3, the guide sign is photographed by the in-vehicle camera 4 while the vehicle is traveling, and the control unit 11 extracts only a specific type of character in the guide sign from the photographed image data to obtain attribute information. (Extraction step). Then, the communication unit 8 transmits detection data including the photographing position information of the guide sign and the obtained attribute information to the data center 2. In the data center 2, the processing control device 14 receives and collects detection data from the vehicle-mounted devices 3 of a plurality of vehicles. Then, based on the collected detection data, guide sign data including the installation position information and attribute information of the guide sign is generated and registered in the guide sign database 17 (sign object data storing step).

At this time, since the attribute information for specifying the guide sign is composed of character string data in which only a specific type of character in the guide sign is extracted, the amount of data transmitted from the vehicle-mounted device 3 to the data center 2 side Can be greatly reduced. Thereby, the communication time for communication and the time required for data processing can be shortened. The data amount of the guide sign data in the road map database 16 is also reduced, the storage capacity is reduced, and the handling of the data is facilitated.

As described above, the guide sign recognition system 1 and the recognition method of the present embodiment use the in-vehicle camera 4 mounted on the vehicle to photograph a guide sign for road guidance as a sign, and based on the photographed image data, A system 1 and a method for recognizing guide signs. According to the guide sign recognition system 1 and the recognition method, it is possible to easily execute a process of specifying which guide sign in the guide sign database 17 is a guide sign photographed by the vehicle-mounted camera 4. It has excellent effects.

In the present embodiment, the processing control device 14 of the data center 2 collects detection data having the same attribute information from the received plurality of detection data, and performs statistical processing on the shooting position information of the collected detection data. To determine the installation location information and use it as guide sign data. As a result, data including more accurate installation position information can be generated, and a highly accurate guide sign database 17 can be constructed.

In the present embodiment, numbers are used as specific types of characters constituting attribute information. As a result, while character type identification processing can be performed with sufficient certainty and in a short time, only ten characters from 0 to 9 need to be extracted and recognized, so that character recognition is extremely simple. And data processing becomes easier. In addition, in extracting numbers, a search for numbers from the photographed image data of the guide sign from left to right of the guide sign is repeated in the vertical direction, and a character string in which the numbers are arranged in the extracted order is repeated. A rule of attribute information was adopted. Thereby, the process of extracting the attribute information can be easily performed.

Although not described in the above embodiment, the in-vehicle camera 4 repeatedly executes image capture, for example, every 100 msec, and sets at which time point the captured image data is to be used for processing. There is a need. From the viewpoint of improving the recognition accuracy, basically, it is preferable to use photographed image data in which the sign appears most immediately before the sign deviates from the screen. However, when it is used for localization from a distance, recognition based on photographed image data photographed relatively early becomes more useful. For example, it is also possible to make settings such that processing of photographed image data is performed at a timing 50 m away from the sign. Here, the localization refers to the relative position of the sign recognized by analyzing the captured image data with respect to the own vehicle, and the position coordinates of the sign registered in the map data, based on the position of the sign. Refers to the process of specifying the position coordinates.

(2) Second Embodiment Next, a second embodiment will be described with reference to FIGS. As shown in FIG. 6, the guide sign recognition system 21 as the sign object recognition system according to the second embodiment can communicate between the data center 22 and the in-vehicle devices 23 mounted on a plurality of vehicles. Connected to. The in-vehicle device 23 includes an in-vehicle camera 4, a position detection unit 5, various on-vehicle sensors 6, a map data storage unit 7, a communication unit 24, an operation display unit 10, and a control unit 25 having a function as an extraction device. The communication unit 24 has functions of a transmission unit 24a as a transmission device and a reception unit 24b as a reception device.

(4) The control unit 25 uses the vehicle-mounted camera 4 to photograph a road condition ahead while the vehicle is running. When a guide sign as a sign is detected in the photographed image data, a specific type of character in the guide sign, in this case, only a numeral, is recognized and extracted from the photographed image data of the guide sign, and extracted. The character string is used as attribute information for specifying the guide sign. The control unit 25 transmits detection data including the photographing position information of the guide sign and the obtained attribute information to the data center 22 by the communication unit 24. In addition, as described later, in the present embodiment, the communication unit 24 receives the vehicle position data transmitted from the communication unit 26 of the data center 22.

On the other hand, the data center 22 includes a communication unit 26, an input operation unit 13, a processing control device 27, a road map database 16, and a guide sign database 28 as a sign object database. The communication unit 26 receives the detection data transmitted from the communication unit 24 of the vehicle-mounted device 23 and transmits vehicle position data to the vehicle-mounted device 23 of the vehicle. Therefore, the communication unit 26 has the functions of a receiving unit 26a as a receiving device and a transmitting unit 26b as a transmitting device. The guide sign database 28 stores guide sign data including installation position information and attribute information of a guide sign as a sign.

When the processing control device 27 receives the detection data from the in-vehicle device 23 of the vehicle through the communication unit 26, the processing control device 27 checks the attribute information of the detection data with the guide sign data in the guide sign database 28. When there is matching attribute information in the guide sign data of the guide sign database 28, the processing control device 27 refers to the road map database 16 based on the installation position information of the guide sign to determine the position of the vehicle. to decide. Therefore, the processing control device 27 has a function of a collating unit 27a as a collating device and a vehicle position determining unit 27b as a vehicle position determining device.

At this time, in the present embodiment, the processing control device 27 performs the following processing when collating the attribute information of the detected data with the guide sign data in the guide sign database 28. That is, based on the shooting position information, the attribute information matches from the guide sign data in which the installation position information exists within a predetermined range around the shooting position, for example, within a circle having a radius of 100 m centered on the coordinates indicated by the shooting position information. Collation is performed by searching for a guide sign. Further, in the present embodiment, the processing control device 27 causes the communication unit 26 to transmit data of the determined vehicle position to the vehicle-mounted device 23 of the vehicle. The in-vehicle device 23 of the vehicle can recognize the own vehicle position or update the own vehicle position on the navigation based on receiving the information of the vehicle position.

Now, the flowchart of FIG. 7 shows a procedure of processing from the photographing of the guide sign by the vehicle-mounted camera 4 to the determination of the vehicle position, which is executed by the control unit 25 of the vehicle-mounted device 23 and the processing control device 27 of the data center 22. ing. In FIG. 7, steps S11 to S13 are processes executed by the control unit 25 of the vehicle-mounted device 23 while the vehicle is running. As in the first embodiment, in step S11, a guidance sign is photographed by the vehicle-mounted camera 4, and in step S12, a specific type of character, in this case, a number, is recognized and extracted from the photographed image data, and the attribute information is extracted. Is required. In step S13, the detection data including the shooting position information and the attribute information is transmitted to the data center 22 by the communication unit 24.

Steps S14 to S16 are processing executed by the processing control device 27 of the data center 22. In step S14, in the data center 22, the detection data transmitted from the in-vehicle device 23 is received by the communication unit 26. Then, in step S15, the attribute information in the received detection data is compared with the attribute information in the guide sign data of the guide sign database 28, and the guide sign as the photographed sign is specified. In step S16, the position of the vehicle is determined from the installation position information of the guide sign specified in step S15, the vehicle position information is transmitted to the vehicle, and the process ends.

FIG. 8 shows an example of a method when the processing control device 27 performs the collation in step S15. Now, it is assumed that the shooting position information of the detection data is, for example, (X0, Y0) and the attribute information is, for example, “155261”. The processing control device 27 first draws a circle R having a radius of 100 m centered on the photographing position information (X0, Y0) as a predetermined range, and extracts guide sign data located within the circle R. In this example, three pieces of guidance sign data of

numbers

1, 2, and 3 are extracted.

{Circle around (4)} If any of the attribute information matches, it is determined that the guide sign is a photographed sign. In this case, since the attribute information of the guide sign data of No. 1 matches “155261”, it is recognized that the guide sign of No. 1 is the photographed guide sign. It should be noted that if there is no attribute information that matches the guide sign data within the predetermined range, and if there are two or more corresponding guide sign data, it is determined that no identification was possible.

In the guide sign recognition system 1 according to the second embodiment, the detection data is transmitted from the vehicle-mounted device 23 to the data center 22. In the processing control device 27 of the data center 22, the attribute information in the received detection data is compared with the attribute information in the guide sign data of the guide sign database 28. Thereby, the guide sign can be specified, and the position of the vehicle can be determined from the installation position information of the guide sign. Further, by transmitting the information on the determined vehicle position to the on-vehicle device 23, the on-vehicle device 23 can accurately recognize the own vehicle position.

In the guide sign recognition system 21 of the second embodiment, the attribute information for specifying the guide sign also includes data of a character string in which only a specific type of character in the guide sign, in this case, only a numeral is extracted. Therefore, the amount of data transmitted from the vehicle-mounted device 23 to the data center 22 can be reduced, and the communication time for communication and the time required for data processing can be reduced. Since the collation processing in the processing control device 27 also requires a small amount of data, it can be easily performed in a short time.

Further, in the present embodiment, the matching process in the processing control device 27 matches the attribute information from the guide sign database 28 from the guide sign data within a predetermined range around the shooting position based on the shooting position in the detection data. This is performed by searching for a guide sign. Thereby, the process of collating the photographed guide sign with the guide sign data in the guide sign database 28 can be easily performed with sufficient certainty.

(3) Third Embodiment FIGS. 9 and 10 show a third embodiment. As shown in FIG. 9, a guide sign recognition system 31 as a sign object recognition system according to the present embodiment includes an in-vehicle device 32 mounted on a vehicle. The in-vehicle device 32 includes an in-vehicle camera 4, a position detection unit 5, various on-vehicle sensors 6, a map data storage unit 7, a communication unit 8, a guide sign database 33 as a sign object database, an operation display unit 10, and a control unit 34. I have. The guide sign database 33 stores guide sign data as the latest high-precision sign object data. For example, those data are generated and updated with high accuracy in the data center 2 or the like described in the first embodiment, and are distributed to the in-vehicle devices 31.

(4) The control unit 34 uses the on-board camera 4 to photograph the road condition ahead while the vehicle is running. When a guide sign as a sign is detected in the photographed image data, a specific type of character in the guide sign, in this case, only a numeral, is recognized and extracted from the photographed image data of the guide sign, and extracted. The character string is used as attribute information for specifying the guide sign. Further, the control unit 34 checks the detection data including the photographing position information of the photographed guide sign and the extracted attribute information with the guide sign data of the guide sign database 33. When there is matching attribute information in the guide sign data of the guide sign database 33, the location of the own vehicle is determined from the installation position information of the guide sign, so-called localization is performed. Therefore, the control unit 34 has the function of the extracting unit 34a as the extracting device, and also has the function of the checking unit 34b as the checking device and the own vehicle position determining unit 34c as the own vehicle position determining device.

フローチャート The flowchart of FIG. 10 shows a procedure of a process executed by the control unit 34 of the in-vehicle device 32 from the photographing of the guide sign by the in-vehicle camera 4 to the determination of the position of the own vehicle. In FIG. 10, at step S21, a guidance sign as a sign is photographed by the vehicle-mounted camera 4. In step S22, a specific type of character, in this case, a number, is recognized and extracted from the captured image data, and attribute information is obtained.

Then, in step S23, the obtained attribute information is compared with the attribute information in the guide sign data of the guide sign database 33, and the photographed guide sign is specified. At the same time, the position of the own vehicle is determined from the specified installation position information of the guide sign, and the process ends. In this case, the same processing as in the second embodiment is performed as a collation method. That is, the guide sign data located within a predetermined range around the photographing position, for example, within a circle having a radius of 100 m, is extracted from the guide sign database 33. If there is one whose attribute information matches, it can be determined that the guide sign has been taken.

In such a third embodiment, the in-vehicle device 32 captures a guide sign as a sign by the in-vehicle camera 4 while the vehicle is running, and only specific types of characters in the guide sign are captured from the captured image data. The extracted attribute information is obtained. Then, the detection data including the photographing position information of the guide sign and the obtained attribute information is collated with the guide sign data of the road map database 33, and if there is a matching attribute information, the information is obtained from the installation position information of the guide sign. The own vehicle position is determined. In this case, it is possible to provide a system capable of detecting the position of the host vehicle with high accuracy, that is, localizing only the in-vehicle device 32 without requiring communication with the data center 2. Thus, in an environment where similar sign objects, that is, guide signs and signboards appear continuously, for example, on an expressway or a general road in an urban area, localization can be performed without confusing the sign objects.

According to the guide sign recognition system 31 of the third embodiment, the attribute information for specifying the guide sign is composed of character string data from which only a specific type of character in the guide sign, in this case, only a numeral is extracted. Therefore, the amount of data to be handled in the collation processing and the like is significantly reduced, and data processing can be performed easily and in a short time. At this time, since the data amount of the guide sign data in the road map database 33 is also small, it is possible to obtain a sufficiently accurate position of the own vehicle even though the storage capacity is small.

Some of the in-vehicle devices incorporated in the vehicle have a function of collecting probe data including positional information of the vehicle during traveling and image information of the in-vehicle camera at that time. The probe data is transmitted to the center of the map data generation system, and the center collects a large number of probe data and integrates them, based on which high-precision map data that can be applied to automatic driving is also generated. Generated and updated. In this case, it is possible to use the position data of a sign object such as a guide sign as a landmark for the alignment between the probe data and the map data and the alignment between the probe data. As a result, it becomes possible to perform high-accuracy positioning associated with the association between landmarks, and thereby to generate high-accuracy map data.

(4) Fourth Embodiment FIG. 11 shows a fourth embodiment. Hereinafter, points different from the third embodiment will be described. That is, the vehicle-mounted device mounted on the vehicle includes a vehicle-mounted camera, a position detection unit, various vehicle-mounted sensors, a map data storage unit, a communication unit, a guide sign database as a sign object database, an operation display unit, and a control unit. The control unit functions as an extraction device, a collation device, a vehicle position determination device, and the like. The control unit recognizes only a specific type of character from photographed image data of a guide sign as a sign taken by a vehicle-mounted camera and extracts the character as attribute information.

In this embodiment, Chinese characters are used in addition to numbers as specific types of characters. Then, the number of Chinese characters in the character group to be recognized by the control unit is limited to a predetermined number, and the character group to be recognized is dynamically changed according to the position of the own vehicle detected by the position detection unit. Is done. Here, for example, as a sign present at an intersection or the like, there is a guide sign indicating an intersection name, a point name, a facility name, and the like. Often described in. However, it is difficult to target all kanji for character recognition. In the present embodiment, the kanji expected to be used for the guidance sign is narrowed down to, for example, about ten to at most several tens of characters from the current position and the traveling direction of the vehicle as a recognition target character group. Has become.

フローチャート The flowchart of FIG. 11 shows a processing procedure executed by the control unit for recognizing a character string of a guide sign as a sign from photographed image data. That is, first, in step S31, rough position information of the current vehicle position is acquired based on the detection of the position detection unit. In step S32, a character group to be recognized is set based on the current position and the traveling direction of the vehicle. In the above example, characters such as "Kari", "Valley", "Station", and "West", that is, Chinese characters, are added to the dictionary for recognition. In this case, the character group to be recognized may be distributed from the data center or may be extracted in the vehicle-mounted device.

In the next step S33, a guidance sign as a sign is photographed by the vehicle-mounted camera, and photographed image data is acquired. In step S34, a process of recognizing and extracting characters, that is, kanji and numerals, in the captured image data is executed. In this case, even in the case of recognizing a kanji, since the character group to be recognized is limited to a very small number, the recognition process can be easily performed in a short time. When the vehicle passes the guide sign portion, the processing from step S31 is repeated toward the next guide sign.

According to the fourth embodiment, similarly to the first to third embodiments, the process of specifying which guide sign in the database is the guide sign as the sign taken by the vehicle-mounted camera. Can be easily executed. Then, in the present embodiment, it becomes possible to recognize kanji as a specific type of character, so that the applicable range can be further expanded.

In the above embodiment, the mode in which the kanji to be recognized is dynamically changed according to the position of the vehicle is disclosed, but the present invention is not limited to this. A plurality of types of characters may be mixed as the types of characters to be recognized according to the own vehicle position. For example, the character group to be recognized may be a combination of hiragana, katakana, and kanji. The character type to be recognized may be dynamically changed according to the position of the host vehicle. For example, the configuration may be such that only numbers are recognized when traveling on a motorway such as an expressway, while numbers and alphabets are recognized when traveling on a general road. The processing load on the CPU is reduced as in the other embodiments by limiting the characters to be recognized not to all character groups used in the area where the vehicle is used, but to some of them. it can.

(5) Fifth and sixth embodiments and other embodiments FIG. 12 shows a fifth embodiment. Here, guide signs A6 and A7 as two right and left signs are provided side by side, but they are very similar, and when recognizing a number, both "26 1/2" are extracted and the difference is detected. No dots are seen. At this time, a sign A8 of “95” is provided above the left guide sign A6, and a sign A9 of “20” is provided above the right guide sign A7. In this case, the guide sign A6 and the signboard A8 are originally handled as different things, and similarly, the guide sign A7 and the signboard A9 are handled as different things.

However, in such a case, the guide sign A6 and the sign A8 are handled as an integral sign, and the guide sign A7 and the sign A9 are treated as an integral sign. As a result, the extracted character strings, that is, the attribute information, become “95 26 1/2” and “20 26 1/2”, and can be easily distinguished from each other. That is, by treating two guide signs or signboards arranged side by side in the vertical direction, that is, the Z-axis direction, as an integral object, an advantage that attribute information can be more easily distinguished can be obtained. Furthermore, of the character strings, it is also possible to adopt a configuration in which information giving higher priority than other numbers is added to “95” and “20”.

FIG. 13 shows a sixth embodiment. In the sixth embodiment, only a specific type of character, in this case, a numeral, is extracted from captured image data of a sign, for example, a guide sign A2. At the same time, information on the position of the character in the sign, in this case, coordinate information with the horizontal axis being the X axis and the vertical axis being the Y axis, is also included in the attribute information. According to this, attribute information can be more easily distinguished from a sign having a similar specific type of character, and recognition processing can be performed more accurately in a shorter time. When the position information is included in the attribute information, the position information may be rough position information such as the upper left, the center, and the right end instead of the coordinate information.

にも In addition, when a plurality of character strings are used as attribute information for one sign, not only the character strings are uniformly used as attribute information but also the following changes are possible. That is, it is possible to extract only the largest character from the plurality of character strings and use the extracted character string as attribute information. When one sign has a plurality of character sets, a delimiter such as a space, a colon, or a slash can be inserted for each character set. In addition, not only a number but also a unit attached to the number as a character string, for example, a unit such as “minute”, “min”, “km”, “m” can be included in the attribute information as a specific type of character. If the position information of the character in the sign and the font size information of the character are included in the map data, the localization is effective.

In each of the embodiments described above, numbers or kanji are adopted as specific types of characters. Hiragana or the like may be adopted. A plurality of types may be used, such as using both kanji and numbers, or using both uppercase letters and numbers in the alphabet. In addition, a combination of only a specific number of the numbers, for example, only 1 to 9, and only a specific one of the alphabets, for example, only A to N, may be used as a specific type of character. The specific type character may be changed depending on the type of landmark or sign. For example, a sign for a direction may be a number, a sign for a large commercial facility may be the letter of the store name, and a sign for an intersection may be a number + kanji or a number + alphabet.

では In the above embodiment, the guide sign on the expressway is mainly used as an example, but it is needless to say that the guide sign can be implemented on a general road. In the second embodiment, the process may be stopped until the vehicle position is determined or grasped in step S15, that is, step S16 may not be executed. In each of the above embodiments, a guide sign is used as an example of a sign, but various signs may be recognized as the sign. In this case, for example, a signboard indicating the name of the large shopping center and the distance to it, a signboard indicating the name of the building or facility, a gas station, a restaurant, a store name or logo of a fast food store having a drive-through, etc. are displayed. There are signboards. Signs installed mainly for commercial purposes can also be covered.

In addition, various changes can be made to the hardware configuration and software configuration of the in-vehicle device, the vehicle, and the data center. Although the present disclosure has been described with reference to the embodiments, it is understood that the present disclosure is not limited to the embodiments and the structures. The present disclosure also encompasses various modifications and variations within an equivalent range. In addition, various combinations and forms, and other combinations and forms including only one element, more or less, are also included in the scope and spirit of the present disclosure.

The control unit and the technique according to the present disclosure are realized by a dedicated computer provided by configuring a processor and a memory programmed to execute one or a plurality of functions embodied by a computer program. May be. Alternatively, the control unit and the technique described in the present disclosure may be implemented by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the control unit and the method according to the present disclosure may be implemented by a combination of a processor and a memory programmed to perform one or more functions and a processor configured with one or more hardware logic circuits. It may be realized by one or more dedicated computers configured. Further, the computer program may be stored in a computer-readable non-transitional tangible recording medium as instructions to be executed by a computer.

Claims

A system (1, 21, 31) for photographing a predetermined sign by a vehicle-mounted camera (4) mounted on a vehicle and recognizing the sign based on the photographed image data,
An extracting device (11, 25, 34) for recognizing and extracting only a specific type of character in the sign from the photographed image data;
The information of the character string extracted by the extracting device (11, 25, 34) is stored as attribute information for specifying the sign object, and the sign object data including the installation position information of the sign object and the attribute information is stored. A sign object recognition system comprising a sign object database (17, 28, 33) for performing the operation.
An in-vehicle device mounted on a plurality of vehicles and a data center are communicably connected and configured,
The in-vehicle device, the in-vehicle camera, the extraction device, and detection data including shooting position information of a sign taken by the in-vehicle camera while the vehicle is traveling and attribute information extracted by the extraction device are sent to the data center. And a transmitting device for transmitting.
The data center includes the sign object database, a collection device that receives and collects detection data from the on-board devices of the vehicles, and the sign object database based on the detection data collected by the collection device. 2. The sign recognition system according to claim 1, further comprising a registration device for registering the information.
The registration device collects detection data having the same attribute information from the received plurality of detection data, determines the installation position information based on statistical processing of the shooting position information of the collected detection data, and performs the labeling. The sign recognition system according to claim 2, wherein the sign data is object data.
An in-vehicle device mounted on a plurality of vehicles and a data center are communicably connected and configured,
The in-vehicle device, the in-vehicle camera, the extraction device, and detection data including shooting position information of a sign taken by the in-vehicle camera while the vehicle is traveling and attribute information extracted by the extraction device are sent to the data center. And a transmitting device for transmitting.
The data center includes the sign object database, a receiving device that receives detection data from the on-vehicle device, a matching device that compares attribute information of the detection data with sign object data of the sign object database, and the sign object database. 4. A vehicle position determining device for determining the position of the vehicle from the installation position information of the sign object when matching attribute information exists in the sign object data of any one of claims 1 to 3. Sign recognition system.
The vehicle is equipped with an in-vehicle device,
The on-vehicle device includes the on-vehicle camera, the extraction device, the sign object database, and detection information including photographing position information of a sign object photographed by the on-vehicle camera while the vehicle is traveling and attribute information extracted by the extraction device. A collation device for collating data with the sign object data of the sign object database, and, when there is matching attribute information in the sign object data of the sign object database, determining the own vehicle position from the installation position information of the sign object The marking object recognition system according to any one of claims 1 to 3, further comprising:
6. The collation device according to claim 4, wherein the collation device performs collation by searching for sign objects having the same attribute information from sign object data within a predetermined range around the photographing position based on the photographing position information. 7. Sign recognition system.
The sign recognition system according to any one of claims 1 to 6, wherein the specific type of character is a numeral.
The extraction device repeats a search for numbers from the photographed image data of the sign in a direction from left to right of the sign in the vertical direction, and attribute a character string in which the numbers are arranged in the extracted order. The sign recognition system according to claim 7, wherein the sign is information.
The sign recognition system according to any one of claims 1 to 6, wherein the character to be recognized by the extraction device is dynamically changed according to the position of the host vehicle.
The specific type of character is a kanji,
10. The sign recognition system according to claim 9, wherein a character group to be recognized by the extraction device is limited to a predetermined number, and the character group to be recognized is dynamically changed according to a position of the host vehicle.
A method in which a predetermined sign is photographed by a vehicle-mounted camera (4) mounted on a vehicle, and the sign is recognized based on the photographed image data,
From the photographed image data, an extraction step of recognizing and extracting only a specific type of character in the sign,
A sign object data storage step of storing information on the character string extracted in the extraction step as attribute information for specifying the sign object, and storing sign object data including the installation position information of the sign object and the attribute information; Sign recognition method including.