WO2019193762A1 - Guide sign identification device and guide sign identification method - Google Patents
Guide sign identification device and guide sign identification method Download PDFInfo
- Publication number
- WO2019193762A1 WO2019193762A1 PCT/JP2018/014795 JP2018014795W WO2019193762A1 WO 2019193762 A1 WO2019193762 A1 WO 2019193762A1 JP 2018014795 W JP2018014795 W JP 2018014795W WO 2019193762 A1 WO2019193762 A1 WO 2019193762A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- guide sign
- feature
- sign
- guide
- detection unit
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
Definitions
- the present invention relates to a guide sign recognition device and a guide sign recognition method for recognizing a guide sign from an image.
- Patent Document 1 discloses a result of determining a geometric figure having a road sign characteristic from an object in an image and collating a template image corresponding to the determined geometric figure with a road sign candidate image. An image processing method for recognizing road signs based on the above is described.
- Patent Document 1 The method described in Patent Document 1 is effective for recognizing a uniform sign such as a speed sign among road signs, but when recognizing a guide sign having various designs, It is necessary to prepare a large number of template images corresponding to various designs. Accordingly, since it is necessary to collate a large number of template images with candidate images, it takes an enormous amount of calculation time to complete recognition.
- the present invention solves the above-described problems, and an object of the present invention is to provide a guide sign recognition device that can reduce the amount of data required for recognition of guide signs.
- the guidance sign recognition apparatus includes a guidance sign detection unit, a feature detection unit, and a guidance sign recognition unit.
- the guide sign detection unit detects a region where the guide sign exists in the captured image.
- the feature detection unit detects the feature of the guide sign from the area detected by the guide sign detection unit.
- the guide sign recognition unit calculates the similarity between the guide sign in the captured image and the known guide sign using the feature of the guide sign detected by the feature detection unit and the feature of the known guide sign, and the calculated similarity Based on the degree, the guide sign in the captured image is recognized from the known guide sign.
- the guide sign recognition device calculates the similarity between the guide sign in the captured image and the known guide sign using the feature of the guide sign in the captured image and the feature of the known guide sign, Based on the similarity, the guide sign in the captured image is recognized from the known guide sign. Since the recognition is performed based on the similarity calculated from the feature of the guide sign, it is not necessary to prepare the entire guide sign image for recognition. Thereby, the data amount required for recognition of a guidance sign can be reduced.
- FIG. 3A is a diagram showing a guide sign G1.
- FIG. 3B shows the guide sign G2.
- FIG. 3C shows the guide sign G3.
- FIG. 3D is a diagram showing a guide sign G4.
- FIG. 4A is a diagram showing a guide sign G1 and its features.
- FIG. 4B is a diagram showing the guide sign G2 and its features.
- FIG. 4C is a diagram showing the guide sign G3 and its features.
- FIG. 4D is a diagram showing a guide sign G4 and its features. It is a figure which shows the characteristic of a guidance sign.
- FIG. 8A is a block diagram showing a hardware configuration for realizing the function of the guidance sign recognition apparatus according to Embodiment 1.
- FIG. 8B is a block diagram showing a hardware configuration for executing software for realizing the function of the guidance sign recognition apparatus according to Embodiment 1.
- FIG. 1 is a block diagram showing a configuration of a guide sign recognition apparatus 1 according to Embodiment 1 of the present invention.
- the guide sign recognition device 1 inputs the captured image stored in the first storage device 2, recognizes the guide sign from the input captured image, and outputs the recognition result to the second storage device 3.
- the first storage device 2 for example, a captured image outside the vehicle captured by an imaging device mounted on the vehicle is temporarily stored.
- the second storage device 3 stores the result of automatic recognition of guide signs by the guide sign recognition device 1.
- the guidance sign recognition device 1 is connected to the third storage device 4, the fourth storage device 5, and the fifth storage device 6.
- the third storage device 4 stores learning data for detecting an image area of a guide sign from an image obtained by capturing a known guide sign.
- the fourth storage device 5 stores learning data for detecting a feature of a known guide sign from the guide sign in the captured image.
- the fifth storage device 6 stores the sign contents of a known guide sign and the feature data of the guide sign.
- the guidance sign recognition device 1 includes a guidance sign detection unit 10, a feature detection unit 11, and a guidance sign recognition unit 12.
- the guide sign detection unit 10 is learned in advance so as to detect a region where the guide sign exists from the image using the learning data input from the third storage device 4.
- the guide sign detection unit 10 receives a captured image from the first storage device 2 and detects a region where the guide sign is present in the input captured image. Data indicating the image area of the guide sign detected by the guide sign detection unit 10 is output to the feature detection unit 11.
- the feature detection unit 11 is learned in advance so as to detect the feature of the guide sign from the image area detected by the guide sign detection unit 10 using the learning data input from the fourth storage device 5.
- Guide sign features are various known guide sign features.
- a plurality of features may be detected by one detector, or each of the plurality of features may be detected by an individual detector.
- the feature detector 11 receives data indicating the image area of the guide sign from the guide sign detector 10 and detects the feature data of the guide sign from the image area indicated by the input data.
- the feature data detected by the feature detection unit 11 is output to the guide sign recognition unit 12.
- the guide sign recognition unit 12 uses the guide sign feature detected by the feature detection unit 11 and the known guide sign feature input from the fifth storage device 6 to use the guide sign in the captured image and the known guide sign.
- the similarity is calculated.
- the guide sign recognition unit 12 recognizes the guide sign in the captured image from the known guide sign based on the calculated similarity. For example, the guide sign recognizing unit 12 recognizes that the guide sign having the maximum similarity among the known guide signs is the guide sign in the captured image.
- FIG. 2 is a flowchart showing a guide sign recognition method according to the first embodiment.
- the guide sign detection unit 10 detects an area where the guide sign exists in the captured image input from the first storage device 2 (step ST1). For example, the guide sign detection unit 10 detects an image area where the guide sign exists from the captured image using a neural network. The neural network is learned in advance so as to detect a region where a guide sign is present from an image using learning data input from the third storage device 4.
- the guidance sign detection part 10 may extract the edge part in a captured image, specify a guidance sign, and may detect the rectangular area
- the feature detection unit 11 detects the feature of the guide sign from the region detected by the guide sign detection unit 10 (step ST2). For example, the feature detection unit 11 detects the feature of the guide sign from the image area of the guide sign using a neural network.
- the neural network is learned in advance so as to detect the feature of the guide sign from the image area using the learning data input from the fourth storage device 5.
- the characteristics of the guide signs include characteristics regarding the color, characters, and symbols of the guide signs. As a characteristic regarding the color of the guide sign, there are background colors of various existing guide signs.
- the characteristic regarding the character of a guidance sign is the character described in the guidance sign. When the character string “ABCD” is described in the guide sign, the characteristics regarding the character of the guide sign are “A”, “B”, “C”, and “D”.
- the characteristic regarding the sign of a guide sign is the sign described in the guide sign.
- the feature detection unit 11 compares the features related to the color, characters, and symbols of the guide sign detected from the area detected by the guide sign detection unit 10 with the features related to the learned colors, characters, and symbols of the known guide sign. .
- the feature detection unit 11 determines the presence / absence of a known guide sign feature in the guide sign in the captured image based on the comparison result between the two, and indicates the feature of the guide sign in the captured image using the result of this determination. Generate a feature vector. For example, the features related to the characters of known guide signs are “A”, “B”, “C”, “D”, and only “A” and “B” are described in the guide signs in the captured image.
- the element corresponding to the characters “A” and “B” in the feature vector indicating the feature of the guide sign in the captured image is set to a value of 1, and the elements corresponding to the characters “C” and “D” The value 0 is set.
- the guide sign recognition unit 12 uses the feature of the guide sign detected by the feature detection unit 11 and the feature of the known guide sign input from the fifth storage device 6, and the guide sign in the captured image and the known guide sign And the guide sign in the captured image is recognized from the known guide sign based on the calculated similarity (step ST3).
- the fifth storage device 6 stores a known guidance sign content and a feature vector indicating the feature of the guidance sign.
- the guide sign recognition unit 12 calculates the similarity between the feature vector indicating the feature of the guide sign detected by the feature detection unit 11 and the feature vector indicating the feature of the known guide sign, and calculates the similarity. It is recognized that the known guide sign determined to have the highest degree is the guide sign in the captured image.
- FIG. 3A is a diagram showing a guide sign G1.
- the guide sign G1 is the number of lanes of a toll booth and ETC (Electronic Toll Collection System; registered trademark, hereinafter abbreviated to be a registered trademark) dedicated lane. It is a guide sign indicating the position.
- FIG. 3B is a diagram showing a guide sign G2, and the guide sign G2 is a guide sign indicating that it is a general lane that does not use ETC.
- FIG. 3C is a diagram showing a guide sign G3.
- the guide sign G3 is a guide sign indicating that the lane is not generally compatible and dedicated to ETC.
- FIG. 3D is a diagram showing a guide sign G4, which is a guide sign to be recognized by the guide sign recognition apparatus 1.
- FIG. 4A is a diagram showing a guide sign G1 and its features.
- FIG. 4B is a diagram showing the guide sign G2 and its features.
- FIG. 4C is a diagram showing the guide sign G3 and its features.
- FIG. 4D is a diagram showing a guide sign G4 and its features.
- the feature C11 shown in FIG. 4A is the color (green) of the guide sign G1
- the feature C21 shown in FIG. 4B is also the color of the guide sign G2.
- the color indicated by the feature C11 and the feature C21 is the same.
- a feature C31 illustrated in FIG. 4C is the color of the guide sign G3, and the color (purple) indicated by the feature C31 is a color different from the colors indicated by the feature C11 and the feature C21.
- the feature C12 shown in FIG. 4A is a character in the guide sign G1, and this character (E) is the same character as the feature C32 shown in FIG. 4C.
- a feature C13 shown in FIG. 4A is a character in the guide sign G1, and this character (T) is the same character as the feature C33 shown in FIG. 4C.
- a feature C14 shown in FIG. 4A is a character in the guide sign G1, and this character (C) is the same character as the feature C34 shown in FIG. 4C.
- a feature C22 shown in FIG. 4B is a character in the guide sign G2, and this character is different from any of the features C12, C13, C14, and C15 shown in FIG. 4A.
- a feature C23 illustrated in FIG. 4B is a character in the guide sign G2, and this character is different from any of the feature C12, the feature C13, the feature C14, the feature C15, and the feature C22.
- a feature C35 shown in FIG. 4C is a character in the guide sign G3, and this character is different from any of the feature C12, the feature C13, the feature C14, the feature C15, the feature C22, and the feature C23.
- a feature C36 illustrated in FIG. 4C is a character in the guide sign G3, and this character is different from any of the feature C12, the feature C13, the feature C14, the feature C15, the feature C22, the feature C23, and the feature C35. .
- a feature C16 and a feature C17 shown in FIG. 4A are symbols (arrows) in the guide sign G1.
- the feature detection unit 11 is learned to detect the above-described features related to colors, characters, and symbols from the images using the captured images of the guide sign G1, the guide sign G2, and the guide sign G3.
- FIG. 5 is a diagram showing the characteristics of the guide sign.
- the feature detection unit 11 is set with correspondence data between guide signs and features as shown in FIG. 5 by learning.
- the columns indicating the color of the guide signs include a color (green) column a1 indicated by the feature C11 and the feature C21 and a color (purple) column a2 indicated by the feature C31. Since both the guide sign G1 and the guide sign G2 have a color corresponding to the column a1, “1” is set in the column a1 and “0” is set in the column a2. Since the guide sign G3 has a color corresponding to the column a2, “1” is set in the column a2, and “0” is set in the column a1.
- the column indicating the sign of the guide sign includes a column c1 of a symbol (arrow) indicated by the feature C16 and the feature C17. Since the symbols (arrows) indicated by the feature C16 and the feature C17 are not included in the guide sign G2 and the guide sign G3, “1” is set in the column c1 corresponding to the guide sign G1, and the guide sign G2 and the guide sign G3 are displayed. “0” is set in the corresponding column c1.
- FIG. 6 is a flowchart showing the feature detection process, and shows details of the process in step ST2 of FIG.
- the feature detection unit 11 the features of the guide signs G1, G2, and G3 shown in FIG. 5 are learned in advance, and the guide sign G4 shown in FIGS. 3D, 4D, and 5D is a recognition target.
- the feature detection unit 11 detects features relating to the color, characters, and symbols of the guide sign G4 from the region where the guide sign G4 exists in the captured image detected by the guide sign detection unit 10.
- the feature detection unit 11 may use a convolutional neural network (hereinafter referred to as CNN) as a method for detecting features.
- CNN convolutional neural network
- a CNN that learns a combination of an image of a guide sign collected in advance and the contents of the indication of the guide sign is used.
- the feature detection unit 11 according to the first embodiment uses a CNN that learns a combination of a feature in an image of a guide sign and information (color, character, symbol, etc.) indicated by the feature.
- the feature detection method is not limited to CNN as long as the feature of the guide sign can be detected from the captured image of the guide sign.
- step ST1a-1 If the color green information sign G4 (step ST1a-1; Yes), the feature detection unit 11 sets "1" to the feature x 1 (step ST2a-1). On the other hand, if the color of the signposts G4 is familiar in green (step ST1a-1; No), the feature detection unit 11 sets "0" to the feature x 1 (step ST2a-2). Subsequently, although not shown, the feature detection unit 11 determines whether the color of the guide sign G4 is purple. Here, since the color indicated by the feature C41 shown in FIG. 4D is green, as shown in FIG. 5, “1” is set in the column a1 of the guide sign G4, and “0” is set in the column a2. Is done.
- the feature detection unit 11 sets “1” to the feature xI + 1 (step ST2a-3). If the guide sign G4 does not include the letter “E” (step ST1a-2; No), the feature detection unit 11 sets “0” for the feature xI + 1 (step ST2a-4). Subsequently, although not shown, the feature detection unit 11 determines whether or not the guide sign G4 includes the letter “T”.
- the feature C42 shown in FIG. 4D is the character “E” in the guide sign G4, and the character indicated by the feature C42 of the guide sign G4 and the character indicated by the feature C12 of the guide sign G1 are the same characters. “1” is set in the corresponding column b1.
- the feature C43 shown in FIG. 4D is the character “T” in the guide sign G4, and the character indicated by the feature C43 of the guide sign G4 and the character indicated by the feature C13 of the guide sign G1 are the same character. “1” is set in the field b2.
- 4D is the character “C” in the guide sign G4, and the character indicated by the feature C44 of the guide sign G4 and the character indicated by the feature C14 of the guide sign G1 are the same characters. “1” is set in the field b3.
- the feature C45 shown in FIG. 4D is a character in the guide sign G4, and the character indicated by the feature C45 of the guide sign G4 and the character indicated by the feature C15 of the guide sign G1 are the same character. Is set to “1”.
- the guide sign G4 does not include the character indicated by the feature C22 shown in FIG. 4B, the character indicated by the feature C23, the character indicated by the feature C35 shown in FIG. 4C, and the character indicated by the feature C36, as shown in FIG. “0” is set in the columns b5 to b8 of the guide sign G4.
- the feature detection unit 11 sets “1” to the feature xI + J + 1 (step ST2a-5). If the guide sign G4 does not include the symbol “arrow” (step ST1a-3; No), the feature detection unit 11 sets “0” to the feature xI + J + 1 (step ST2a-6).
- the symbol indicated by the feature C46 illustrated in FIG. 4D is an arrow, “1” is set in the column c1 of the guide sign G4 as illustrated in FIG.
- the feature detection unit 11 When the presence or absence of the feature of the learning data in the guidance sign G4 is determined, the feature detection unit 11 generates a feature vector x having the digital value set according to the presence or absence of the feature as an element and outputs the feature vector x to the guidance sign recognition unit 12 ( Step ST3a).
- FIG. 7 is a flowchart showing guidance sign recognition processing, and shows details of step ST3 in FIG.
- the guide sign recognizing unit 12 inputs the feature vector x of the recognition target guide sign generated by the feature detecting unit 11 (step ST1b). Thereafter, the guide sign recognition unit 12 proceeds to a repetition loop from 1 to D. D is the number of known guide signs stored in the fifth storage device 6.
- x is a feature vector of the guide sign a
- the guide sign recognition unit 12 performs the similarity c 1 between the first guide sign and the recognition target guide sign to the similarity c D between the Dth guide sign and the recognition target guide sign. After calculating the among these similarities c d, signs corresponding to the maximum similarity, and outputs the recognized as the signs of the recognition target (step ST4b).
- the cosine similarity between the feature vector of the guide sign G1 and the feature vector of the guide sign G4 to be recognized is 1.0. It becomes.
- the cosine similarity between the feature vector of the guide sign G2 and the feature vector of the guide sign G4 is 0.24
- the cosine similarity between the feature vector of the guide sign G3 and the feature vector of the guide sign G4 is 0.5. It becomes.
- the guide sign recognition unit 12 recognizes that the guide sign G4 is the guide sign G1 because the similarity of the guide sign G1 is the highest.
- the conventional recognition method could not recognize a guide sign with a different design from the learning data.
- the similarity between the guidance sign to be recognized and the guidance sign of the learning data is determined based on the feature of the guidance sign. For this reason, even if the guidance sign G4 to be recognized has a different design from the guidance sign G1 that is the learning data, it includes many of the same features, and is thus recognized as a guidance sign with the same marking content.
- the guidance sign recognition apparatus 1 includes a processing circuit for executing the processing from step ST1 to step ST3 in FIG.
- This processing circuit may be dedicated hardware, or may be a CPU (Central Processing Unit) that executes a program stored in a memory.
- CPU Central Processing Unit
- FIG. 8A is a block diagram showing a hardware configuration for realizing the function of the guide sign recognition apparatus 1.
- FIG. 8B is a block diagram showing a hardware configuration for executing software for realizing the function of the guidance sign recognition apparatus 1.
- the information input / output interface 100 includes the guide sign recognition device 1, the first storage device 2, the second storage device 3, the third storage device 4, the fourth storage device 5 and the like shown in FIG. This is an interface that relays information exchange with the fifth storage device 6.
- the guidance sign detection unit 10 inputs the image data of the guidance sign to be recognized from the first storage device 2 and the learning data from the third storage device 4 via the information input / output interface 100.
- the feature detection unit 11 inputs learning data from the fourth storage device 5 via the information input / output interface 100.
- the guide sign recognition unit 12 inputs characteristic data of known guide signs from the fifth storage device 6 via the information input / output interface 100 and outputs the recognition result to the second storage device 3.
- the first storage device 2, the second storage device 3, the third storage device 4, the fourth storage device 5, and the fifth storage device 6 may be storage devices included in the guidance sign recognition device 1.
- a storage device provided independently of the sign recognition device 1 may be used. These storage devices may be storage devices provided so as to be communicable from the guide sign recognition device 1, for example, storage devices existing on the cloud.
- the display interface 101 is an interface that relays information output from the guide sign recognition apparatus 1 to a display (not shown).
- the guide sign recognition unit 12 outputs the recognition result to the display device via the display device interface 101.
- the display unit displays the recognition result input from the guide sign recognition apparatus 1 via the display unit interface 101.
- the processing circuit 102 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated), or the like. Circuit), FPGA (Field-Programmable Gate Array), or a combination thereof.
- the functions of the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12 in the guide sign recognition device 1 may be realized by separate processing circuits, and these functions are collectively realized by one processing circuit. Also good.
- the functions of the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12 in the guide sign recognition device 1 are software, firmware, or a combination of software and firmware. It is realized by.
- the software or firmware is described as a program and stored in the memory 104.
- the processor 103 reads out and executes the program stored in the memory 104, thereby realizing the functions of the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12 in the guide sign recognition device 1. That is, the guidance sign recognition apparatus 1 includes a memory 104 for storing a program that, when executed by the processor 103, results in the processing from step ST1 to step ST3 shown in FIG. These programs cause the computer to execute the procedure or method of the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12.
- the memory 104 may be a computer-readable storage medium storing a program for causing a computer to function as the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12.
- the memory 104 includes, for example, a nonvolatile memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically-EPROM), or a volatile memory such as an EEPROM (Electrically-EPROM).
- a nonvolatile memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically-EPROM), or a volatile memory such as an EEPROM (Electrically-EPROM).
- a nonvolatile memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically-EPROM), or a volatile memory such as an EEPROM (Electrically-EPROM).
- EEPROM Electrically
- the functions of the guidance sign detection unit 10, the feature detection unit 11, and the guidance sign recognition unit 12 may be partially realized with dedicated hardware and partly realized with software or firmware.
- the guidance sign detection unit 10 realizes the function by a processing circuit that is dedicated hardware, and the feature detection unit 11 and the guidance sign recognition unit 12 read and execute a program stored in the memory 104 by the processor 103. To realize the function.
- the processing circuit can realize the above functions by hardware, software, firmware, or a combination thereof.
- the guide sign in the captured image and the known guide sign are used by using the feature of the guide sign in the captured image and the feature of the known guide sign.
- the similarity is calculated, and the guide sign in the captured image is recognized from the known guide sign based on the calculated similarity. Since the recognition is performed based on the similarity calculated from the feature of the guide sign, it is not necessary to prepare the entire guide sign image for recognition. Thereby, the data amount required for recognition of a guidance sign can be reduced.
- the guidance sign recognition apparatus can reduce the amount of data necessary for recognition of the guidance sign, it can be used, for example, in a driving assistance apparatus that automatically recognizes the guidance sign and supports driving.
- 1 guidance sign recognition device 2 first storage device, 3rd storage device, 4th 3rd storage device, 5th 4th storage device, 6th 5th storage device, 10 guidance sign detection unit, 11 feature detection unit, 12 guidance sign Recognition unit, 100 information input / output interface, 101 display interface, 102 processing circuit, 103 processor, 104 memory.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
This guide sign identification device (1) uses the features of a guide sign in a captured image and the features of an existing guide sign to calculate the similarity between the features of the guide sign in the captured image and the features of the existing guide sign, and identifies the guide sign in the captured image from the existing guide sign on the basis of the calculated similarity.
Description
この発明は、画像から案内標識を認識する案内標識認識装置および案内標識認識方法に関する。
The present invention relates to a guide sign recognition device and a guide sign recognition method for recognizing a guide sign from an image.
従来から、画像から道路標識を認識する技術が提案されている。
例えば、特許文献1には、画像中のオブジェクトから道路標識の特徴を有した幾何学的図形を判定し、判定した幾何学的図形に対応したテンプレート画像と道路標識の候補画像とを照合した結果に基づいて道路標識を認識する画像処理方法が記載されている。 Conventionally, techniques for recognizing road signs from images have been proposed.
For example,Patent Document 1 discloses a result of determining a geometric figure having a road sign characteristic from an object in an image and collating a template image corresponding to the determined geometric figure with a road sign candidate image. An image processing method for recognizing road signs based on the above is described.
例えば、特許文献1には、画像中のオブジェクトから道路標識の特徴を有した幾何学的図形を判定し、判定した幾何学的図形に対応したテンプレート画像と道路標識の候補画像とを照合した結果に基づいて道路標識を認識する画像処理方法が記載されている。 Conventionally, techniques for recognizing road signs from images have been proposed.
For example,
特許文献1に記載された方法は、道路標識のうち、速度標識のように画一的なデザインの標識の認識には有効であるが、多様なデザインが存在する案内標識を認識する場合は、多様なデザインに対応した多数のテンプレート画像を用意する必要がある。これにより、多数のテンプレート画像と候補画像とを照合する必要があるため、認識が完了するまでに膨大な計算時間がかかる。
The method described in Patent Document 1 is effective for recognizing a uniform sign such as a speed sign among road signs, but when recognizing a guide sign having various designs, It is necessary to prepare a large number of template images corresponding to various designs. Accordingly, since it is necessary to collate a large number of template images with candidate images, it takes an enormous amount of calculation time to complete recognition.
この発明は上記課題を解決するものであって、案内標識の認識に要するデータ量を削減することができる案内標識認識装置を得ることを目的とする。
The present invention solves the above-described problems, and an object of the present invention is to provide a guide sign recognition device that can reduce the amount of data required for recognition of guide signs.
この発明に係る案内標識認識装置は、案内標識検出部、特徴検出部および案内標識認識部を備える。案内標識検出部は、撮像画像中の案内標識が存在する領域を検出する。特徴検出部は、案内標識検出部によって検出された領域から案内標識の特徴を検出する。案内標識認識部が、特徴検出部によって検出された案内標識の特徴と既知の案内標識の特徴とを用いて撮像画像中の案内標識と既知の案内標識との類似度を算出し、算出した類似度に基づいて、既知の案内標識から撮像画像中の案内標識を認識する。
The guidance sign recognition apparatus according to the present invention includes a guidance sign detection unit, a feature detection unit, and a guidance sign recognition unit. The guide sign detection unit detects a region where the guide sign exists in the captured image. The feature detection unit detects the feature of the guide sign from the area detected by the guide sign detection unit. The guide sign recognition unit calculates the similarity between the guide sign in the captured image and the known guide sign using the feature of the guide sign detected by the feature detection unit and the feature of the known guide sign, and the calculated similarity Based on the degree, the guide sign in the captured image is recognized from the known guide sign.
この発明によれば、案内標識認識装置は、撮像画像中の案内標識の特徴と既知の案内標識の特徴とを用いて撮像画像中の案内標識と既知の案内標識との類似度を算出し、類似度に基づいて、既知の案内標識から撮像画像中の案内標識を認識する。案内標識の特徴から算出した類似度を基準として認識を行うので、認識のために案内標識の画像全体を用意する必要はない。これにより、案内標識の認識に必要なデータ量を削減することができる。
According to this invention, the guide sign recognition device calculates the similarity between the guide sign in the captured image and the known guide sign using the feature of the guide sign in the captured image and the feature of the known guide sign, Based on the similarity, the guide sign in the captured image is recognized from the known guide sign. Since the recognition is performed based on the similarity calculated from the feature of the guide sign, it is not necessary to prepare the entire guide sign image for recognition. Thereby, the data amount required for recognition of a guidance sign can be reduced.
以下、この発明をより詳細に説明するため、この発明を実施するための形態について、添付の図面に従って説明する。
実施の形態1.
図1は、この発明の実施の形態1に係る案内標識認識装置1の構成を示すブロック図である。案内標識認識装置1は、第1記憶装置2に記憶された撮像画像を入力し、入力した撮像画像から案内標識を認識し、認識結果を第2記憶装置3に出力する。第1記憶装置2には、例えば、車両に搭載された撮像装置によって撮像された車外の撮像画像が一時的に記憶される。第2記憶装置3には、案内標識認識装置1による案内標識の自動認識の結果が記憶される。 Hereinafter, in order to describe the present invention in more detail, modes for carrying out the present invention will be described with reference to the accompanying drawings.
Embodiment 1 FIG.
FIG. 1 is a block diagram showing a configuration of a guidesign recognition apparatus 1 according to Embodiment 1 of the present invention. The guide sign recognition device 1 inputs the captured image stored in the first storage device 2, recognizes the guide sign from the input captured image, and outputs the recognition result to the second storage device 3. In the first storage device 2, for example, a captured image outside the vehicle captured by an imaging device mounted on the vehicle is temporarily stored. The second storage device 3 stores the result of automatic recognition of guide signs by the guide sign recognition device 1.
実施の形態1.
図1は、この発明の実施の形態1に係る案内標識認識装置1の構成を示すブロック図である。案内標識認識装置1は、第1記憶装置2に記憶された撮像画像を入力し、入力した撮像画像から案内標識を認識し、認識結果を第2記憶装置3に出力する。第1記憶装置2には、例えば、車両に搭載された撮像装置によって撮像された車外の撮像画像が一時的に記憶される。第2記憶装置3には、案内標識認識装置1による案内標識の自動認識の結果が記憶される。 Hereinafter, in order to describe the present invention in more detail, modes for carrying out the present invention will be described with reference to the accompanying drawings.
FIG. 1 is a block diagram showing a configuration of a guide
案内標識認識装置1は、第3記憶装置4、第4記憶装置5および第5記憶装置6に接続されている。第3記憶装置4には、既知の案内標識が撮像された画像から案内標識の画像領域を検出するための学習データが記憶されている。第4記憶装置5には、撮像画像中の案内標識から既知の案内標識の特徴を検出するための学習データが記憶されている。第5記憶装置6には、既知の案内標識の標示内容とこの案内標識の特徴データとが記憶されている。
The guidance sign recognition device 1 is connected to the third storage device 4, the fourth storage device 5, and the fifth storage device 6. The third storage device 4 stores learning data for detecting an image area of a guide sign from an image obtained by capturing a known guide sign. The fourth storage device 5 stores learning data for detecting a feature of a known guide sign from the guide sign in the captured image. The fifth storage device 6 stores the sign contents of a known guide sign and the feature data of the guide sign.
案内標識認識装置1は、案内標識検出部10、特徴検出部11および案内標識認識部12を備える。案内標識検出部10は、第3記憶装置4から入力した学習データを用いて、画像から案内標識が存在する領域を検出するように事前に学習されている。案内標識検出部10は、第1記憶装置2から撮像画像を入力し、入力した撮像画像中の案内標識が存在する領域を検出する。案内標識検出部10によって検出された案内標識の画像領域を示すデータは、特徴検出部11に出力される。
The guidance sign recognition device 1 includes a guidance sign detection unit 10, a feature detection unit 11, and a guidance sign recognition unit 12. The guide sign detection unit 10 is learned in advance so as to detect a region where the guide sign exists from the image using the learning data input from the third storage device 4. The guide sign detection unit 10 receives a captured image from the first storage device 2 and detects a region where the guide sign is present in the input captured image. Data indicating the image area of the guide sign detected by the guide sign detection unit 10 is output to the feature detection unit 11.
特徴検出部11は、第4記憶装置5から入力した学習データを用いて、案内標識検出部10によって検出された画像領域から案内標識の特徴を検出するように事前に学習されている。案内標識の特徴は、既知の様々な案内標識の特徴である。特徴の検出方法は、複数の特徴を1つの検出器で検出してもよく、複数の特徴のそれぞれについて個別の検出器で検出してもよい。特徴検出部11は、案内標識検出部10から案内標識の画像領域を示すデータを入力し、入力したデータが示す画像領域から案内標識の特徴データを検出する。特徴検出部11によって検出された特徴データは、案内標識認識部12に出力される。
The feature detection unit 11 is learned in advance so as to detect the feature of the guide sign from the image area detected by the guide sign detection unit 10 using the learning data input from the fourth storage device 5. Guide sign features are various known guide sign features. In the feature detection method, a plurality of features may be detected by one detector, or each of the plurality of features may be detected by an individual detector. The feature detector 11 receives data indicating the image area of the guide sign from the guide sign detector 10 and detects the feature data of the guide sign from the image area indicated by the input data. The feature data detected by the feature detection unit 11 is output to the guide sign recognition unit 12.
案内標識認識部12は、特徴検出部11によって検出された案内標識の特徴と第5記憶装置6から入力した既知の案内標識の特徴とを用いて、撮像画像中の案内標識と既知の案内標識との類似度を算出する。案内標識認識部12は、算出した類似度に基づいて既知の案内標識から撮像画像中の案内標識を認識する。例えば、案内標識認識部12は、既知の案内標識のうち、類似度が最大の案内標識が撮像画像中の案内標識であると認識する。
The guide sign recognition unit 12 uses the guide sign feature detected by the feature detection unit 11 and the known guide sign feature input from the fifth storage device 6 to use the guide sign in the captured image and the known guide sign. The similarity is calculated. The guide sign recognition unit 12 recognizes the guide sign in the captured image from the known guide sign based on the calculated similarity. For example, the guide sign recognizing unit 12 recognizes that the guide sign having the maximum similarity among the known guide signs is the guide sign in the captured image.
次に動作について説明する。
図2は、実施の形態1に係る案内標識認識方法を示すフローチャートである。
案内標識検出部10は、第1記憶装置2から入力した撮像画像中の案内標識が存在する領域を検出する(ステップST1)。例えば、案内標識検出部10は、ニューラルネットワークを用いて撮像画像から案内標識が存在する画像領域を検出する。ニューラルネットワークは、第3記憶装置4から入力した学習データを用いて画像から案内標識が存在する領域を検出するように事前に学習されている。なお、案内標識検出部10は、撮像画像中のエッジ部分を抽出して案内標識を特定し、特定した案内標識が含まれる矩形領域を検出してもよい。 Next, the operation will be described.
FIG. 2 is a flowchart showing a guide sign recognition method according to the first embodiment.
The guidesign detection unit 10 detects an area where the guide sign exists in the captured image input from the first storage device 2 (step ST1). For example, the guide sign detection unit 10 detects an image area where the guide sign exists from the captured image using a neural network. The neural network is learned in advance so as to detect a region where a guide sign is present from an image using learning data input from the third storage device 4. In addition, the guidance sign detection part 10 may extract the edge part in a captured image, specify a guidance sign, and may detect the rectangular area | region where the specified guidance sign is included.
図2は、実施の形態1に係る案内標識認識方法を示すフローチャートである。
案内標識検出部10は、第1記憶装置2から入力した撮像画像中の案内標識が存在する領域を検出する(ステップST1)。例えば、案内標識検出部10は、ニューラルネットワークを用いて撮像画像から案内標識が存在する画像領域を検出する。ニューラルネットワークは、第3記憶装置4から入力した学習データを用いて画像から案内標識が存在する領域を検出するように事前に学習されている。なお、案内標識検出部10は、撮像画像中のエッジ部分を抽出して案内標識を特定し、特定した案内標識が含まれる矩形領域を検出してもよい。 Next, the operation will be described.
FIG. 2 is a flowchart showing a guide sign recognition method according to the first embodiment.
The guide
特徴検出部11は、案内標識検出部10によって検出された領域から、案内標識の特徴を検出する(ステップST2)。例えば、特徴検出部11は、ニューラルネットワークを用いて案内標識の画像領域から案内標識の特徴を検出する。ニューラルネットワークは、第4記憶装置5から入力した学習データを用いて画像領域から案内標識の特徴を検出するように事前に学習されている。案内標識の特徴には、案内標識の色、文字および記号に関する特徴がある。案内標識の色に関する特徴としては、既存の様々な案内標識の背景色が挙げられる。案内標識の文字に関する特徴は、案内標識に記載されている文字である。案内標識に“ABCD”という文字列が記載されていた場合、案内標識の文字に関する特徴は“A”、“B”、“C”、“D”である。案内標識の記号に関する特徴は、案内標識に記載されている記号である。
The feature detection unit 11 detects the feature of the guide sign from the region detected by the guide sign detection unit 10 (step ST2). For example, the feature detection unit 11 detects the feature of the guide sign from the image area of the guide sign using a neural network. The neural network is learned in advance so as to detect the feature of the guide sign from the image area using the learning data input from the fourth storage device 5. The characteristics of the guide signs include characteristics regarding the color, characters, and symbols of the guide signs. As a characteristic regarding the color of the guide sign, there are background colors of various existing guide signs. The characteristic regarding the character of a guidance sign is the character described in the guidance sign. When the character string “ABCD” is described in the guide sign, the characteristics regarding the character of the guide sign are “A”, “B”, “C”, and “D”. The characteristic regarding the sign of a guide sign is the sign described in the guide sign.
特徴検出部11は、案内標識検出部10によって検出された領域から検出した案内標識の色、文字および記号に関する特徴と、学習済みの既知の案内標識の色、文字および記号に関する特徴とを比較する。特徴検出部11は、両者の比較結果に基づいて、撮像画像中の案内標識における既知の案内標識の特徴の有無を判定し、この判定の結果を用いて撮像画像中の案内標識の特徴を示す特徴ベクトルを生成する。例えば、既知の案内標識の文字に関する特徴は、“A”、“B”、“C”、“D”であり、撮像画像中の案内標識には、“A”および“B”のみが記載されていた場合、撮像画像中の案内標識の特徴を示す特徴ベクトルにおける文字“A”および“B”に対応する要素に値1が設定され、文字“C”および“D”に対応する要素には値0が設定される。
The feature detection unit 11 compares the features related to the color, characters, and symbols of the guide sign detected from the area detected by the guide sign detection unit 10 with the features related to the learned colors, characters, and symbols of the known guide sign. . The feature detection unit 11 determines the presence / absence of a known guide sign feature in the guide sign in the captured image based on the comparison result between the two, and indicates the feature of the guide sign in the captured image using the result of this determination. Generate a feature vector. For example, the features related to the characters of known guide signs are “A”, “B”, “C”, “D”, and only “A” and “B” are described in the guide signs in the captured image. In this case, the element corresponding to the characters “A” and “B” in the feature vector indicating the feature of the guide sign in the captured image is set to a value of 1, and the elements corresponding to the characters “C” and “D” The value 0 is set.
案内標識認識部12は、特徴検出部11によって検出された案内標識の特徴と第5記憶装置6から入力した既知の案内標識の特徴とを用いて撮像画像中の案内標識と既知の案内標識との類似度を算出し、算出した類似度に基づいて、既知の案内標識から撮像画像中の案内標識を認識する(ステップST3)。例えば、第5記憶装置6には、既知の案内標識の標示内容とこの案内標識の特徴を示す特徴ベクトルが記憶されている。案内標識認識部12は、特徴検出部11によって検出された案内標識の特徴を示す特徴ベクトルと、既知の案内標識の特徴を示す特徴ベクトルとを用いて両者の類似度を算出し、算出した類似度が最も大きいと判定された既知の案内標識が撮像画像中の案内標識であると認識する。
The guide sign recognition unit 12 uses the feature of the guide sign detected by the feature detection unit 11 and the feature of the known guide sign input from the fifth storage device 6, and the guide sign in the captured image and the known guide sign And the guide sign in the captured image is recognized from the known guide sign based on the calculated similarity (step ST3). For example, the fifth storage device 6 stores a known guidance sign content and a feature vector indicating the feature of the guidance sign. The guide sign recognition unit 12 calculates the similarity between the feature vector indicating the feature of the guide sign detected by the feature detection unit 11 and the feature vector indicating the feature of the known guide sign, and calculates the similarity. It is recognized that the known guide sign determined to have the highest degree is the guide sign in the captured image.
次に、特徴検出処理について詳細に説明する。
図3Aは、案内標識G1を示す図であり、案内標識G1は、料金所のレーン数とETC(Electronic Toll Collection System;登録商標、以下、登録商標であることの記載を省略する)専用レーンの位置を示す案内標識である。図3Bは、案内標識G2を示す図であり、案内標識G2は、ETCを使用しない一般用のレーンであることを示す案内標識である。図3Cは、案内標識G3を示す図であり、案内標識G3は、一般に対応しておらず、ETC専用のレーンであることを示す案内標識である。図3Dは、案内標識G4を示す図であり、案内標識G4は、案内標識認識装置1の認識対象の案内標識である。 Next, the feature detection process will be described in detail.
FIG. 3A is a diagram showing a guide sign G1. The guide sign G1 is the number of lanes of a toll booth and ETC (Electronic Toll Collection System; registered trademark, hereinafter abbreviated to be a registered trademark) dedicated lane. It is a guide sign indicating the position. FIG. 3B is a diagram showing a guide sign G2, and the guide sign G2 is a guide sign indicating that it is a general lane that does not use ETC. FIG. 3C is a diagram showing a guide sign G3. The guide sign G3 is a guide sign indicating that the lane is not generally compatible and dedicated to ETC. FIG. 3D is a diagram showing a guide sign G4, which is a guide sign to be recognized by the guidesign recognition apparatus 1.
図3Aは、案内標識G1を示す図であり、案内標識G1は、料金所のレーン数とETC(Electronic Toll Collection System;登録商標、以下、登録商標であることの記載を省略する)専用レーンの位置を示す案内標識である。図3Bは、案内標識G2を示す図であり、案内標識G2は、ETCを使用しない一般用のレーンであることを示す案内標識である。図3Cは、案内標識G3を示す図であり、案内標識G3は、一般に対応しておらず、ETC専用のレーンであることを示す案内標識である。図3Dは、案内標識G4を示す図であり、案内標識G4は、案内標識認識装置1の認識対象の案内標識である。 Next, the feature detection process will be described in detail.
FIG. 3A is a diagram showing a guide sign G1. The guide sign G1 is the number of lanes of a toll booth and ETC (Electronic Toll Collection System; registered trademark, hereinafter abbreviated to be a registered trademark) dedicated lane. It is a guide sign indicating the position. FIG. 3B is a diagram showing a guide sign G2, and the guide sign G2 is a guide sign indicating that it is a general lane that does not use ETC. FIG. 3C is a diagram showing a guide sign G3. The guide sign G3 is a guide sign indicating that the lane is not generally compatible and dedicated to ETC. FIG. 3D is a diagram showing a guide sign G4, which is a guide sign to be recognized by the guide
特徴検出部11は、案内標識G1、案内標識G2および案内標識G3のそれぞれの撮像画像を用いて、画像から、色、文字および記号に関する特徴を検出するように学習されているものとする。図4Aは、案内標識G1およびその特徴を示す図である。図4Bは、案内標識G2およびその特徴を示す図である。図4Cは、案内標識G3およびその特徴を示す図である。図4Dは、案内標識G4およびその特徴を示す図である。
Suppose that the feature detection unit 11 has been learned to detect features related to colors, characters, and symbols from images using the respective captured images of the guide sign G1, the guide sign G2, and the guide sign G3. FIG. 4A is a diagram showing a guide sign G1 and its features. FIG. 4B is a diagram showing the guide sign G2 and its features. FIG. 4C is a diagram showing the guide sign G3 and its features. FIG. 4D is a diagram showing a guide sign G4 and its features.
図4Aに示す特徴C11は、案内標識G1の色(緑)であり、図4Bに示す特徴C21も案内標識G2の色である。特徴C11と特徴C21が示す色は同じである。図4Cに示す特徴C31は、案内標識G3の色であり、特徴C31が示す色(紫)は、特徴C11と特徴C21が示す色とは異なる色である。
The feature C11 shown in FIG. 4A is the color (green) of the guide sign G1, and the feature C21 shown in FIG. 4B is also the color of the guide sign G2. The color indicated by the feature C11 and the feature C21 is the same. A feature C31 illustrated in FIG. 4C is the color of the guide sign G3, and the color (purple) indicated by the feature C31 is a color different from the colors indicated by the feature C11 and the feature C21.
図4Aに示す特徴C12は、案内標識G1中の文字であり、この文字(E)は、図4Cに示す特徴C32と同じ文字である。図4Aに示す特徴C13は、案内標識G1中の文字であり、この文字(T)は、図4Cに示す特徴C33と同じ文字である。図4Aに示す特徴C14は、案内標識G1中の文字であり、この文字(C)は、図4Cに示す特徴C34と同じ文字である。
The feature C12 shown in FIG. 4A is a character in the guide sign G1, and this character (E) is the same character as the feature C32 shown in FIG. 4C. A feature C13 shown in FIG. 4A is a character in the guide sign G1, and this character (T) is the same character as the feature C33 shown in FIG. 4C. A feature C14 shown in FIG. 4A is a character in the guide sign G1, and this character (C) is the same character as the feature C34 shown in FIG. 4C.
図4Bに示す特徴C22は、案内標識G2中の文字であり、この文字は、図4Aに示す特徴C12、特徴C13、特徴C14および特徴C15のいずれとも異なる文字である。図4Bに示す特徴C23は、案内標識G2中の文字であり、この文字は、特徴C12、特徴C13、特徴C14、特徴C15および特徴C22のいずれとも異なる文字である。
A feature C22 shown in FIG. 4B is a character in the guide sign G2, and this character is different from any of the features C12, C13, C14, and C15 shown in FIG. 4A. A feature C23 illustrated in FIG. 4B is a character in the guide sign G2, and this character is different from any of the feature C12, the feature C13, the feature C14, the feature C15, and the feature C22.
図4Cに示す特徴C35は、案内標識G3中の文字であり、この文字は、特徴C12、特徴C13、特徴C14、特徴C15、特徴C22および特徴C23のいずれとも異なる文字である。また、図4Cに示す特徴C36は、案内標識G3中の文字であり、この文字は、特徴C12、特徴C13、特徴C14、特徴C15、特徴C22、特徴C23および特徴C35のいずれとも異なる文字である。
A feature C35 shown in FIG. 4C is a character in the guide sign G3, and this character is different from any of the feature C12, the feature C13, the feature C14, the feature C15, the feature C22, and the feature C23. A feature C36 illustrated in FIG. 4C is a character in the guide sign G3, and this character is different from any of the feature C12, the feature C13, the feature C14, the feature C15, the feature C22, the feature C23, and the feature C35. .
図4Aに示す特徴C16および特徴C17は、案内標識G1中の記号(矢印)である。
特徴検出部11は、案内標識G1、案内標識G2および案内標識G3のそれぞれの撮像画像を用いて、画像から、前述した、色、文字および記号に関する特徴を検出するように学習されている。図5は、案内標識の特徴を示す図である。特徴検出部11には、学習によって、図5に示すような案内標識と特徴との対応データが設定されている。 A feature C16 and a feature C17 shown in FIG. 4A are symbols (arrows) in the guide sign G1.
The feature detection unit 11 is learned to detect the above-described features related to colors, characters, and symbols from the images using the captured images of the guide sign G1, the guide sign G2, and the guide sign G3. FIG. 5 is a diagram showing the characteristics of the guide sign. The feature detection unit 11 is set with correspondence data between guide signs and features as shown in FIG. 5 by learning.
特徴検出部11は、案内標識G1、案内標識G2および案内標識G3のそれぞれの撮像画像を用いて、画像から、前述した、色、文字および記号に関する特徴を検出するように学習されている。図5は、案内標識の特徴を示す図である。特徴検出部11には、学習によって、図5に示すような案内標識と特徴との対応データが設定されている。 A feature C16 and a feature C17 shown in FIG. 4A are symbols (arrows) in the guide sign G1.
The feature detection unit 11 is learned to detect the above-described features related to colors, characters, and symbols from the images using the captured images of the guide sign G1, the guide sign G2, and the guide sign G3. FIG. 5 is a diagram showing the characteristics of the guide sign. The feature detection unit 11 is set with correspondence data between guide signs and features as shown in FIG. 5 by learning.
図5において、案内標識の色を示す欄には、特徴C11と特徴C21が示す色(緑)の欄a1と特徴C31が示す色(紫)の欄a2とがある。案内標識G1および案内標識G2は、両方とも欄a1に対応する色を有するので、欄a1に“1”が設定され、欄a2には“0”が設定されている。案内標識G3は、欄a2に対応する色を有するので、欄a2に“1”が設定され、欄a1には“0”が設定されている。
In FIG. 5, the columns indicating the color of the guide signs include a color (green) column a1 indicated by the feature C11 and the feature C21 and a color (purple) column a2 indicated by the feature C31. Since both the guide sign G1 and the guide sign G2 have a color corresponding to the column a1, “1” is set in the column a1 and “0” is set in the column a2. Since the guide sign G3 has a color corresponding to the column a2, “1” is set in the column a2, and “0” is set in the column a1.
図5において、案内標識の文字を示す欄には、特徴C12および特徴C32が示す文字の欄b1、特徴C13と特徴C33が示す文字の欄b2、特徴C14と特徴C34が示す文字の欄b3、特徴C15が示す文字の欄b4、特徴C22が示す文字の欄b5、特徴C23が示す文字の欄b6、特徴C35が示す文字の欄b7、および特徴C36が示す文字の欄b8がある。
In FIG. 5, in the column indicating the characters of the guide signs, the character column b1 indicated by the feature C12 and the feature C32, the character column b2 indicated by the feature C13 and the feature C33, the character column b3 indicated by the feature C14 and the feature C34, There are a character column b4 indicated by the feature C15, a character column b5 indicated by the feature C22, a character column b6 indicated by the feature C23, a character column b7 indicated by the feature C35, and a character column b8 indicated by the feature C36.
案内標識G1の特徴C12が示す文字と案内標識G3の特徴C32が示す文字とは同じ文字(E)であることから、案内標識G1と案内標識G3の欄b1に“1”が設定され、案内標識G2には、特徴C12が示す文字(E)はないので、欄b1には“0”が設定されている。
Since the character indicated by the feature C12 of the guide sign G1 and the character indicated by the feature C32 of the guide sign G3 are the same character (E), “1” is set in the column b1 of the guide sign G1 and the guide sign G3, and the guide Since the sign G2 does not have the character (E) indicated by the feature C12, “0” is set in the column b1.
案内標識G1の特徴C13が示す文字と案内標識G3の特徴C33が示す文字とは同じ文字であることから、案内標識G1と案内標識G3の欄b2に“1”が設定され、案内標識G2には、特徴C13が示す文字はないので、欄b2には“0”が設定されている。
案内標識G1の特徴C14が示す文字と案内標識G3の特徴C34が示す文字とは同じ文字であることから、案内標識G1と案内標識G3の欄b3に“1”が設定され、案内標識G2には、特徴C14が示す文字はないので、欄b3には“0”が設定されている。 Since the character indicated by the feature C13 of the guide sign G1 and the character indicated by the feature C33 of the guide sign G3 are the same character, “1” is set in the column b2 of the guide sign G1 and the guide sign G3. Since there is no character indicated by the feature C13, "0" is set in the field b2.
Since the character indicated by the feature C14 of the guide sign G1 and the character indicated by the feature C34 of the guide sign G3 are the same character, “1” is set in the column b3 of the guide sign G1 and the guide sign G3, and the guide sign G2 is displayed. Since there is no character indicated by the feature C14, "0" is set in the field b3.
案内標識G1の特徴C14が示す文字と案内標識G3の特徴C34が示す文字とは同じ文字であることから、案内標識G1と案内標識G3の欄b3に“1”が設定され、案内標識G2には、特徴C14が示す文字はないので、欄b3には“0”が設定されている。 Since the character indicated by the feature C13 of the guide sign G1 and the character indicated by the feature C33 of the guide sign G3 are the same character, “1” is set in the column b2 of the guide sign G1 and the guide sign G3. Since there is no character indicated by the feature C13, "0" is set in the field b2.
Since the character indicated by the feature C14 of the guide sign G1 and the character indicated by the feature C34 of the guide sign G3 are the same character, “1” is set in the column b3 of the guide sign G1 and the guide sign G3, and the guide sign G2 is displayed. Since there is no character indicated by the feature C14, "0" is set in the field b3.
案内標識G1の特徴C15が示す文字は、案内標識G2と案内標識G3にはないので、案内標識G1に対応する欄b4には“1”が設定され、案内標識G2および案内標識G3に対応する欄b4には“0”が設定されている。案内標識G2の特徴C22が示す文字は案内標識G1と案内標識G3にはないので、案内標識G2に対応する欄b5には、“1”が設定され、案内標識G1および案内標識G3に対応する欄b5には“0”が設定されている。
Since the character indicated by the feature C15 of the guide sign G1 is not in the guide sign G2 and the guide sign G3, “1” is set in the column b4 corresponding to the guide sign G1, and the characters corresponding to the guide sign G2 and the guide sign G3 are set. In the column b4, “0” is set. Since the character indicated by the feature C22 of the guide sign G2 is not in the guide sign G1 and the guide sign G3, “1” is set in the column b5 corresponding to the guide sign G2, and the characters corresponding to the guide sign G1 and the guide sign G3 are set. In the column b5, “0” is set.
案内標識G2の特徴C23が示す文字は、案内標識G1と案内標識G3にはないので、案内標識G2に対応する欄b6には“1”が設定され、案内標識G1および案内標識G3に対応する欄b6には“0”が設定されている。案内標識G3の特徴C35が示す文字は案内標識G1と案内標識G2にはないので、案内標識G3に対応する欄b7には、“1”が設定され、案内標識G1および案内標識G2に対応する欄b7には“0”が設定されている。
Since the character indicated by the feature C23 of the guide sign G2 is not in the guide sign G1 and the guide sign G3, “1” is set in the column b6 corresponding to the guide sign G2, and the characters corresponding to the guide sign G1 and the guide sign G3 are set. In the column b6, “0” is set. Since the character indicated by the feature C35 of the guide sign G3 is not in the guide sign G1 and the guide sign G2, “1” is set in the column b7 corresponding to the guide sign G3, and the characters corresponding to the guide sign G1 and the guide sign G2 are set. In the column b7, “0” is set.
案内標識G3の特徴C36が示す文字は、案内標識G1と案内標識G2にはないので、案内標識G3に対応する欄b8には、“1”が設定され、案内標識G1および案内標識G2に対応する欄b8には“0”が設定されている。
Since the character indicated by the feature C36 of the guide sign G3 does not exist in the guide sign G1 and the guide sign G2, “1” is set in the column b8 corresponding to the guide sign G3, and corresponds to the guide sign G1 and the guide sign G2. “0” is set in the column b8 to be performed.
図5において、案内標識の記号を示す欄には、特徴C16および特徴C17が示す記号(矢印)の欄c1がある。特徴C16および特徴C17が示す記号(矢印)は、案内標識G2および案内標識G3にはないので、案内標識G1に対応する欄c1には“1”が設定され、案内標識G2および案内標識G3に対応する欄c1には“0”が設定されている。
In FIG. 5, the column indicating the sign of the guide sign includes a column c1 of a symbol (arrow) indicated by the feature C16 and the feature C17. Since the symbols (arrows) indicated by the feature C16 and the feature C17 are not included in the guide sign G2 and the guide sign G3, “1” is set in the column c1 corresponding to the guide sign G1, and the guide sign G2 and the guide sign G3 are displayed. “0” is set in the corresponding column c1.
図6は、特徴検出処理を示すフローチャートであって、図2のステップST2の処理の詳細を示している。特徴検出部11には、図5に示した案内標識G1,G2,G3の特徴が事前に学習されており、図3D、図4Dおよび図5Dに示した案内標識G4が認識対象である。まず、特徴検出部11は、案内標識検出部10によって検出された撮像画像中の案内標識G4が存在する領域から、案内標識G4の色、文字および記号に関する特徴を検出する。
FIG. 6 is a flowchart showing the feature detection process, and shows details of the process in step ST2 of FIG. In the feature detection unit 11, the features of the guide signs G1, G2, and G3 shown in FIG. 5 are learned in advance, and the guide sign G4 shown in FIGS. 3D, 4D, and 5D is a recognition target. First, the feature detection unit 11 detects features relating to the color, characters, and symbols of the guide sign G4 from the region where the guide sign G4 exists in the captured image detected by the guide sign detection unit 10.
特徴検出部11では、特徴を検出する手法として、畳み込みニューラルネットワーク(以下、CNNと記載する)を用いてもよい。従来のCNNを用いた画像認識では、事前に収集した案内標識の画像と案内標識の標示内容との組み合わせを学習させたCNNが用いられる。これに対して、実施の形態1における特徴検出部11では、案内標識の画像中の特徴とその特徴が示す情報(色、文字、記号など)との組み合わせを学習させたCNNを用いる。これにより、学習データである既知の案内標識と認識対象の未知の案内標識とのデザインが異なっていても、両者が同じ特徴を有していれば、その特徴が検出される。
なお、特徴を検出する手法は、案内標識の撮像画像から案内標識の特徴を検出することができる手法であれば、CNNに限定されるものではない。 The feature detection unit 11 may use a convolutional neural network (hereinafter referred to as CNN) as a method for detecting features. In conventional image recognition using a CNN, a CNN that learns a combination of an image of a guide sign collected in advance and the contents of the indication of the guide sign is used. On the other hand, the feature detection unit 11 according to the first embodiment uses a CNN that learns a combination of a feature in an image of a guide sign and information (color, character, symbol, etc.) indicated by the feature. Thereby, even if the design of the known guide sign that is the learning data and the unknown guide sign to be recognized are different, the feature is detected if both have the same feature.
Note that the feature detection method is not limited to CNN as long as the feature of the guide sign can be detected from the captured image of the guide sign.
なお、特徴を検出する手法は、案内標識の撮像画像から案内標識の特徴を検出することができる手法であれば、CNNに限定されるものではない。 The feature detection unit 11 may use a convolutional neural network (hereinafter referred to as CNN) as a method for detecting features. In conventional image recognition using a CNN, a CNN that learns a combination of an image of a guide sign collected in advance and the contents of the indication of the guide sign is used. On the other hand, the feature detection unit 11 according to the first embodiment uses a CNN that learns a combination of a feature in an image of a guide sign and information (color, character, symbol, etc.) indicated by the feature. Thereby, even if the design of the known guide sign that is the learning data and the unknown guide sign to be recognized are different, the feature is detected if both have the same feature.
Note that the feature detection method is not limited to CNN as long as the feature of the guide sign can be detected from the captured image of the guide sign.
特徴検出部11は、案内標識G4の色に関するI(Iは、1以上の自然数)個の設問として、案内標識G4の色が緑か否かを判定する(ステップST1a-1)。前述した例では、学習データの案内標識の色として緑と紫が学習されており、I=1,2である。特徴検出部11は、案内標識G4の色が緑であるか否かの判定と、案内標識G4の色が紫であるか否かの判定とを行う。
The feature detection unit 11 determines whether the color of the guide sign G4 is green as I questions (I is a natural number of 1 or more) regarding the color of the guide sign G4 (step ST1a-1). In the example described above, green and purple are learned as the colors of the guide signs of the learning data, and I = 1, 2. The feature detection unit 11 determines whether the color of the guide sign G4 is green and determines whether the color of the guide sign G4 is purple.
案内標識G4の色が緑であれば(ステップST1a-1;Yes)、特徴検出部11は特徴x1に“1”を設定する(ステップST2a-1)。一方、案内標識G4の色が緑ではなれば(ステップST1a-1;No)、特徴検出部11は特徴x1に“0”を設定する(ステップST2a-2)。続いて、図示を省略したが、特徴検出部11は、案内標識G4の色が紫か否かを判定する。ここでは、図4Dに示した特徴C41が示す色は緑であるので、図5に示すように、案内標識G4の欄a1には“1”が設定され、欄a2には“0”が設定される。
If the color green information sign G4 (step ST1a-1; Yes), the feature detection unit 11 sets "1" to the feature x 1 (step ST2a-1). On the other hand, if the color of the signposts G4 is familiar in green (step ST1a-1; No), the feature detection unit 11 sets "0" to the feature x 1 (step ST2a-2). Subsequently, although not shown, the feature detection unit 11 determines whether the color of the guide sign G4 is purple. Here, since the color indicated by the feature C41 shown in FIG. 4D is green, as shown in FIG. 5, “1” is set in the column a1 of the guide sign G4, and “0” is set in the column a2. Is done.
特徴検出部11は、案内標識G4の文字に関するJ(Jは、1以上の自然数)個の設問として、案内標識G4が文字“E”を含むか否かを判定する(ステップST1a-2)。学習データの案内標識の文字として、図5に示した欄b1~b8に対応する8つの文字が学習されており、J=1,2,3,4,5,6,7,8である。特徴検出部11は、これらの欄に対応する文字を含むか否かの判定を順次行う。
The feature detection unit 11 determines whether or not the guide sign G4 includes the letter “E” as J questions (J is a natural number of 1 or more) regarding the character of the guide sign G4 (step ST1a-2). Eight characters corresponding to the columns b1 to b8 shown in FIG. 5 have been learned as the guidance sign characters of the learning data, and J = 1, 2, 3, 4, 5, 6, 7, 8. The feature detection unit 11 sequentially determines whether or not the characters corresponding to these fields are included.
案内標識G4が文字“E”を含む場合(ステップST1a-2;Yes)、特徴検出部11は特徴xI+1に“1”を設定する(ステップST2a-3)。案内標識G4が文字“E”を含まなければ(ステップST1a-2;No)、特徴検出部11は、特徴xI+1に“0”を設定する(ステップST2a-4)。続いて、図示を省略したが、特徴検出部11は、案内標識G4が文字“T”を含むか否かを判定する。
When the guide sign G4 includes the character “E” (step ST1a-2; Yes), the feature detection unit 11 sets “1” to the feature xI + 1 (step ST2a-3). If the guide sign G4 does not include the letter “E” (step ST1a-2; No), the feature detection unit 11 sets “0” for the feature xI + 1 (step ST2a-4). Subsequently, although not shown, the feature detection unit 11 determines whether or not the guide sign G4 includes the letter “T”.
図4Dに示した特徴C42は案内標識G4中の文字“E”であり、案内標識G4の特徴C42が示す文字と案内標識G1の特徴C12が示す文字は同じ文字であるので、案内標識G4に対応する欄b1に“1”が設定される。図4Dに示した特徴C43は案内標識G4中の文字“T”であり、案内標識G4の特徴C43が示す文字と案内標識G1の特徴C13が示す文字は同じ文字であるので、案内標識G4の欄b2に“1”が設定される。図4Dに示した特徴C44は案内標識G4中の文字“C”であり、案内標識G4の特徴C44が示す文字と案内標識G1の特徴C14が示す文字は同じ文字であるので、案内標識G4の欄b3に“1”が設定される。図4Dに示した特徴C45は案内標識G4中の文字であり、案内標識G4の特徴C45が示す文字と案内標識G1の特徴C15が示す文字は同じ文字であるので、案内標識G4の欄b4には“1”が設定される。
The feature C42 shown in FIG. 4D is the character “E” in the guide sign G4, and the character indicated by the feature C42 of the guide sign G4 and the character indicated by the feature C12 of the guide sign G1 are the same characters. “1” is set in the corresponding column b1. The feature C43 shown in FIG. 4D is the character “T” in the guide sign G4, and the character indicated by the feature C43 of the guide sign G4 and the character indicated by the feature C13 of the guide sign G1 are the same character. “1” is set in the field b2. The feature C44 shown in FIG. 4D is the character “C” in the guide sign G4, and the character indicated by the feature C44 of the guide sign G4 and the character indicated by the feature C14 of the guide sign G1 are the same characters. “1” is set in the field b3. The feature C45 shown in FIG. 4D is a character in the guide sign G4, and the character indicated by the feature C45 of the guide sign G4 and the character indicated by the feature C15 of the guide sign G1 are the same character. Is set to “1”.
案内標識G4は、図4Bに示した特徴C22が示す文字、特徴C23が示す文字、図4Cに示した特徴C35が示す文字、および特徴C36が示す文字を含まないので、図5に示すように、案内標識G4の欄b5~b8には“0”が設定される。
Since the guide sign G4 does not include the character indicated by the feature C22 shown in FIG. 4B, the character indicated by the feature C23, the character indicated by the feature C35 shown in FIG. 4C, and the character indicated by the feature C36, as shown in FIG. “0” is set in the columns b5 to b8 of the guide sign G4.
特徴検出部11は、案内標識G4の記号に関するK(Kは、1以上の自然数)個の設問として、案内標識G4が記号“矢印”を含むか否かを判定する(ステップST1a-3)。前述した例では、学習データの案内標識の記号として矢印が学習されており、K=1である。特徴検出部11は、案内標識G4に記号“矢印”が含まれるか否かを判定する。
The feature detection unit 11 determines whether or not the guide sign G4 includes the symbol “arrow” as K questions (K is a natural number of 1 or more) regarding the sign of the guide sign G4 (step ST1a-3). In the above-described example, an arrow is learned as a guide sign symbol of learning data, and K = 1. The feature detection unit 11 determines whether or not the symbol “arrow” is included in the guide sign G4.
案内標識G4が記号“矢印”を含む場合(ステップST1a-3;Yes)、特徴検出部11は特徴xI+J+1に“1”を設定する(ステップST2a-5)。案内標識G4が記号“矢印”を含まなければ(ステップST1a-3;No)、特徴検出部11は、特徴xI+J+1に“0”を設定する(ステップST2a-6)。ここでは、図4Dに示した特徴C46が示す記号は矢印であるので、図5に示すように、案内標識G4の欄c1には“1”が設定される。
When the guide sign G4 includes the symbol “arrow” (step ST1a-3; Yes), the feature detection unit 11 sets “1” to the feature xI + J + 1 (step ST2a-5). If the guide sign G4 does not include the symbol “arrow” (step ST1a-3; No), the feature detection unit 11 sets “0” to the feature xI + J + 1 (step ST2a-6). Here, since the symbol indicated by the feature C46 illustrated in FIG. 4D is an arrow, “1” is set in the column c1 of the guide sign G4 as illustrated in FIG.
案内標識G4における学習データの特徴の有無を判定すると、特徴検出部11は、特徴の有無に応じて設定したデジタル値を要素とした特徴ベクトルxを生成して案内標識認識部12に出力する(ステップST3a)。特徴ベクトルxは、下記式(1)で表すことができる。図5に示すように、図4Aに示した案内標識G1と図4Dに示した案内標識G4とでは、デザインが異なるものの、同じ特徴を有している。
x=(x1,・・・,xI,xI+1,・・・,xI+J,xI+J+1)T ・・・(1) When the presence or absence of the feature of the learning data in the guidance sign G4 is determined, the feature detection unit 11 generates a feature vector x having the digital value set according to the presence or absence of the feature as an element and outputs the feature vector x to the guidance sign recognition unit 12 ( Step ST3a). The feature vector x can be expressed by the following equation (1). As shown in FIG. 5, the guide sign G1 shown in FIG. 4A and the guide sign G4 shown in FIG.
x = (x 1 ,..., x I , x I + 1 ,..., x I + J , x I + J + 1 ) T (1)
x=(x1,・・・,xI,xI+1,・・・,xI+J,xI+J+1)T ・・・(1) When the presence or absence of the feature of the learning data in the guidance sign G4 is determined, the feature detection unit 11 generates a feature vector x having the digital value set according to the presence or absence of the feature as an element and outputs the feature vector x to the guidance sign recognition unit 12 ( Step ST3a). The feature vector x can be expressed by the following equation (1). As shown in FIG. 5, the guide sign G1 shown in FIG. 4A and the guide sign G4 shown in FIG.
x = (x 1 ,..., x I , x I + 1 ,..., x I + J , x I + J + 1 ) T (1)
次に、案内標識認識処理について詳細に説明する。
図7は、案内標識認識処理を示すフローチャートであり、図2のステップST3の詳細を示している。案内標識認識部12は、特徴検出部11によって生成された認識対象の案内標識の特徴ベクトルxを入力する(ステップST1b)。この後、案内標識認識部12は、1~Dまでの繰り返しループに移行する。Dは、第5記憶装置6に記憶された既知の案内標識の数である。 Next, the guidance sign recognition process will be described in detail.
FIG. 7 is a flowchart showing guidance sign recognition processing, and shows details of step ST3 in FIG. The guidesign recognizing unit 12 inputs the feature vector x of the recognition target guide sign generated by the feature detecting unit 11 (step ST1b). Thereafter, the guide sign recognition unit 12 proceeds to a repetition loop from 1 to D. D is the number of known guide signs stored in the fifth storage device 6.
図7は、案内標識認識処理を示すフローチャートであり、図2のステップST3の詳細を示している。案内標識認識部12は、特徴検出部11によって生成された認識対象の案内標識の特徴ベクトルxを入力する(ステップST1b)。この後、案内標識認識部12は、1~Dまでの繰り返しループに移行する。Dは、第5記憶装置6に記憶された既知の案内標識の数である。 Next, the guidance sign recognition process will be described in detail.
FIG. 7 is a flowchart showing guidance sign recognition processing, and shows details of step ST3 in FIG. The guide
案内標識認識部12は、第5記憶装置6から第d(d=1~D)番目の案内標識の特徴ベクトルydを入力する(ステップST2b)。次に、案内標識認識部12は、認識対象の案内標識の特徴ベクトルxと既知の案内標識の特徴ベクトルydとを用いて、認識対象の案内標識と既知の案内標識との類似度cdを算出する(ステップST3b)。例えば、案内標識aと案内標識bの類似度は、下記式(2)で求められるコサイン類似度を用いてもよい。下記式(2)において、xは案内標識aの特徴ベクトルであり、yは案内標識bの特徴ベクトルである。
cos(案内標識a,案内標識b)=x・y/(|x||y|) ・・・(2)Signpost recognition unit 12 from the fifth storage device 6 inputs the first d (d = 1 ~ D) th signs feature vector y d (step ST2b). Then, information sign recognition unit 12 uses the feature vector y d of the feature vector x and the known signs of signs to be recognized, the similarity c d between signs and known signs to be recognized Is calculated (step ST3b). For example, the cosine similarity obtained by the following formula (2) may be used as the similarity between the guide sign a and the guide sign b. In the following formula (2), x is a feature vector of the guide sign a, and y is a feature vector of the guide sign b.
cos (guide sign a, guide sign b) = x · y / (| x || y |) (2)
cos(案内標識a,案内標識b)=x・y/(|x||y|) ・・・(2)
cos (guide sign a, guide sign b) = x · y / (| x || y |) (2)
案内標識認識部12は、上記繰り返しループにおいて、第1番目の案内標識と認識対象の案内標識との類似度c1から第D番目の案内標識と認識対象の案内標識との類似度cDまでを算出すると、これらの類似度cdのうち、最大の類似度に対応する案内標識が認識対象の案内標識であると認識して出力する(ステップST4b)。
In the above iterative loop, the guide sign recognition unit 12 performs the similarity c 1 between the first guide sign and the recognition target guide sign to the similarity c D between the Dth guide sign and the recognition target guide sign. After calculating the among these similarities c d, signs corresponding to the maximum similarity, and outputs the recognized as the signs of the recognition target (step ST4b).
例えば、上記式(1)および上記式(2)と図5に示した値を用いると、案内標識G1の特徴ベクトルと認識対象の案内標識G4の特徴ベクトルとのコサイン類似度は、1.0となる。一方、案内標識G2の特徴ベクトルと案内標識G4の特徴ベクトルとのコサイン類似度は、0.24となり、案内標識G3の特徴ベクトルと案内標識G4の特徴ベクトルとのコサイン類似度は、0.5となる。案内標識認識部12は、案内標識G1の類似度が最も大きいので、案内標識G4が案内標識G1であると認識される。
For example, using the above formulas (1) and (2) and the values shown in FIG. 5, the cosine similarity between the feature vector of the guide sign G1 and the feature vector of the guide sign G4 to be recognized is 1.0. It becomes. On the other hand, the cosine similarity between the feature vector of the guide sign G2 and the feature vector of the guide sign G4 is 0.24, and the cosine similarity between the feature vector of the guide sign G3 and the feature vector of the guide sign G4 is 0.5. It becomes. The guide sign recognition unit 12 recognizes that the guide sign G4 is the guide sign G1 because the similarity of the guide sign G1 is the highest.
従来の認識手法では、学習データと異なるデザインの案内標識を認識することができなかった。これに対して、実施の形態1に係る案内標識認識装置1では、案内標識の特徴に基づいて認識対象の案内標識と学習データの案内標識との類似度を判断する。このため、認識対象の案内標識G4が、学習データである案内標識G1とは異なるデザインであっても、同じ特徴を多く含んでいるため、同じ標示内容の案内標識であると認識される。
The conventional recognition method could not recognize a guide sign with a different design from the learning data. On the other hand, in the guidance sign recognition apparatus 1 according to the first embodiment, the similarity between the guidance sign to be recognized and the guidance sign of the learning data is determined based on the feature of the guidance sign. For this reason, even if the guidance sign G4 to be recognized has a different design from the guidance sign G1 that is the learning data, it includes many of the same features, and is thus recognized as a guidance sign with the same marking content.
なお、案内標識の特徴として、色、文字および記号を例に挙げたが、特徴から類似度の算出が可能であれば、これら3つの特徴よりも少なくても多くてもよい。
Note that colors, characters, and symbols are given as examples of the features of the guide signs. However, if the similarity can be calculated from the features, the number may be less or more than these three features.
案内標識認識装置1における案内標識検出部10、特徴検出部11および案内標識認識部12の機能は、処理回路によって実現される。すなわち、案内標識認識装置1は、図2におけるステップST1からステップST3までの処理を実行するための処理回路を備えている。この処理回路は、専用のハードウェアであってもよいが、メモリに記憶されたプログラムを実行するCPU(Central Processing Unit)であってもよい。
The functions of the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12 in the guide sign recognition device 1 are realized by a processing circuit. That is, the guidance sign recognition apparatus 1 includes a processing circuit for executing the processing from step ST1 to step ST3 in FIG. This processing circuit may be dedicated hardware, or may be a CPU (Central Processing Unit) that executes a program stored in a memory.
図8Aは、案内標識認識装置1の機能を実現するハードウェア構成を示すブロック図である。図8Bは、案内標識認識装置1の機能を実現するソフトウェアを実行するハードウェア構成を示すブロック図である。図8Aおよび図8Bにおいて、情報入出力インタフェース100は、案内標識認識装置1と、図1に示した第1記憶装置2、第2記憶装置3、第3記憶装置4、第4記憶装置5および第5記憶装置6との間の情報のやり取りを中継するインタフェースである。
FIG. 8A is a block diagram showing a hardware configuration for realizing the function of the guide sign recognition apparatus 1. FIG. 8B is a block diagram showing a hardware configuration for executing software for realizing the function of the guidance sign recognition apparatus 1. 8A and 8B, the information input / output interface 100 includes the guide sign recognition device 1, the first storage device 2, the second storage device 3, the third storage device 4, the fourth storage device 5 and the like shown in FIG. This is an interface that relays information exchange with the fifth storage device 6.
例えば、案内標識検出部10は、情報入出力インタフェース100を介して、第1記憶装置2から認識対象の案内標識の画像データを入力し、第3記憶装置4から学習データを入力する。特徴検出部11は、情報入出力インタフェース100を介して、第4記憶装置5から学習データを入力する。案内標識認識部12は、情報入出力インタフェース100を介して、第5記憶装置6から既知の案内標識の特徴データを入力し、認識結果を第2記憶装置3に出力する。なお、第1記憶装置2、第2記憶装置3、第3記憶装置4、第4記憶装置5および第5記憶装置6は、案内標識認識装置1が備える記憶装置であってもよいが、案内標識認識装置1とは独立して設けられた記憶装置であってもよい。これらの記憶装置は、案内標識認識装置1から通信可能に設けられた記憶装置、例えば、クラウド上に存在する記憶装置であってもよい。
For example, the guidance sign detection unit 10 inputs the image data of the guidance sign to be recognized from the first storage device 2 and the learning data from the third storage device 4 via the information input / output interface 100. The feature detection unit 11 inputs learning data from the fourth storage device 5 via the information input / output interface 100. The guide sign recognition unit 12 inputs characteristic data of known guide signs from the fifth storage device 6 via the information input / output interface 100 and outputs the recognition result to the second storage device 3. The first storage device 2, the second storage device 3, the third storage device 4, the fourth storage device 5, and the fifth storage device 6 may be storage devices included in the guidance sign recognition device 1. A storage device provided independently of the sign recognition device 1 may be used. These storage devices may be storage devices provided so as to be communicable from the guide sign recognition device 1, for example, storage devices existing on the cloud.
表示器インタフェース101は、案内標識認識装置1から不図示の表示器へ出力される情報を中継するインタフェースである。案内標識認識部12は、表示器インタフェース101を介して認識結果を表示器に出力する。表示器は、表示器インタフェース101を介して案内標識認識装置1から入力した認識結果を表示する。
The display interface 101 is an interface that relays information output from the guide sign recognition apparatus 1 to a display (not shown). The guide sign recognition unit 12 outputs the recognition result to the display device via the display device interface 101. The display unit displays the recognition result input from the guide sign recognition apparatus 1 via the display unit interface 101.
上記処理回路が図8Aに示す専用のハードウェアの処理回路102である場合、処理回路102は、例えば、単一回路、複合回路、プログラム化したプロセッサ、並列プログラム化したプロセッサ、ASIC(Application Specific Integrated Circuit)、FPGA(Field-Programmable Gate Array)、または、これらを組み合わせたものが該当する。案内標識認識装置1における案内標識検出部10、特徴検出部11および案内標識認識部12の機能を別々の処理回路で実現してもよく、これらの機能をまとめて1つの処理回路で実現してもよい。
When the processing circuit is the dedicated hardware processing circuit 102 shown in FIG. 8A, the processing circuit 102 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated), or the like. Circuit), FPGA (Field-Programmable Gate Array), or a combination thereof. The functions of the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12 in the guide sign recognition device 1 may be realized by separate processing circuits, and these functions are collectively realized by one processing circuit. Also good.
上記処理回路が図8Bに示すプロセッサ103である場合、案内標識認識装置1における案内標識検出部10、特徴検出部11および案内標識認識部12の機能は、ソフトウェア、ファームウェアまたはソフトウェアとファームウェアとの組み合わせによって実現される。なお、ソフトウェアまたはファームウェアは、プログラムとして記述されてメモリ104に記憶される。
When the processing circuit is the processor 103 shown in FIG. 8B, the functions of the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12 in the guide sign recognition device 1 are software, firmware, or a combination of software and firmware. It is realized by. The software or firmware is described as a program and stored in the memory 104.
プロセッサ103は、メモリ104に記憶されたプログラムを読み出して実行することにより、案内標識認識装置1における案内標識検出部10、特徴検出部11および案内標識認識部12の機能を実現する。すなわち、案内標識認識装置1は、プロセッサ103によって実行されるときに、図2に示したステップST1からステップST3までの処理が結果的に実行されるプログラムを記憶するためのメモリ104を備える。これらのプログラムは、案内標識検出部10、特徴検出部11および案内標識認識部12の手順または方法をコンピュータに実行させる。メモリ104は、コンピュータを、案内標識検出部10、特徴検出部11および案内標識認識部12として機能させるためのプログラムが記憶されたコンピュータ可読記憶媒体であってもよい。
The processor 103 reads out and executes the program stored in the memory 104, thereby realizing the functions of the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12 in the guide sign recognition device 1. That is, the guidance sign recognition apparatus 1 includes a memory 104 for storing a program that, when executed by the processor 103, results in the processing from step ST1 to step ST3 shown in FIG. These programs cause the computer to execute the procedure or method of the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12. The memory 104 may be a computer-readable storage medium storing a program for causing a computer to function as the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12.
メモリ104には、例えば、RAM(Random Access Memory)、ROM(Read Only Memory)、フラッシュメモリ、EPROM(Erasable Programmable Read Only Memory)、EEPROM(Electrically-EPROM)などの不揮発性または揮発性の半導体メモリ、磁気ディスク、フレキシブルディスク、光ディスク、コンパクトディスク、ミニディスク、DVDなどが該当する。
The memory 104 includes, for example, a nonvolatile memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically-EPROM), or a volatile memory such as an EEPROM (Electrically-EPROM). Magnetic disks, flexible disks, optical disks, compact disks, mini disks, DVDs, and the like are applicable.
案内標識検出部10、特徴検出部11および案内標識認識部12の機能について一部を専用のハードウェアで実現し、一部をソフトウェアまたはファームウェアで実現してもよい。例えば、案内標識検出部10は、専用のハードウェアである処理回路で機能を実現し、特徴検出部11および案内標識認識部12は、プロセッサ103がメモリ104に記憶されたプログラムを読み出して実行することによって機能を実現する。このように、処理回路は、ハードウェア、ソフトウェア、ファームウェアまたはこれらの組み合わせにより上記機能を実現することができる。
The functions of the guidance sign detection unit 10, the feature detection unit 11, and the guidance sign recognition unit 12 may be partially realized with dedicated hardware and partly realized with software or firmware. For example, the guidance sign detection unit 10 realizes the function by a processing circuit that is dedicated hardware, and the feature detection unit 11 and the guidance sign recognition unit 12 read and execute a program stored in the memory 104 by the processor 103. To realize the function. As described above, the processing circuit can realize the above functions by hardware, software, firmware, or a combination thereof.
以上のように、実施の形態1に係る案内標識認識装置1では、撮像画像中の案内標識の特徴と既知の案内標識の特徴とを用いて撮像画像中の案内標識と既知の案内標識との類似度を算出し、算出した類似度に基づいて既知の案内標識から撮像画像中の案内標識を認識する。案内標識の特徴から算出した類似度を基準として認識を行うので、認識のために案内標識の画像全体を用意する必要はない。これにより、案内標識の認識に必要なデータ量を削減することができる。
As described above, in the guide sign recognition apparatus 1 according to Embodiment 1, the guide sign in the captured image and the known guide sign are used by using the feature of the guide sign in the captured image and the feature of the known guide sign. The similarity is calculated, and the guide sign in the captured image is recognized from the known guide sign based on the calculated similarity. Since the recognition is performed based on the similarity calculated from the feature of the guide sign, it is not necessary to prepare the entire guide sign image for recognition. Thereby, the data amount required for recognition of a guidance sign can be reduced.
なお、本発明は上記実施の形態に限定されるものではなく、本発明の範囲内において、実施の形態のそれぞれの自由な組み合わせまたは実施の形態のそれぞれの任意の構成要素の変形もしくは実施の形態のそれぞれにおいて任意の構成要素の省略が可能である。
It should be noted that the present invention is not limited to the above-described embodiment, and within the scope of the present invention, each free combination of the embodiments or any component modification or embodiment of the embodiments. It is possible to omit arbitrary components in each of the above.
この発明に係る案内標識認識装置は、案内標識の認識に必要なデータ量を削減することができるので、例えば、案内標識を自動認識して運転を支援する運転支援装置に利用可能である。
Since the guidance sign recognition apparatus according to the present invention can reduce the amount of data necessary for recognition of the guidance sign, it can be used, for example, in a driving assistance apparatus that automatically recognizes the guidance sign and supports driving.
1 案内標識認識装置、2 第1記憶装置、3 第2記憶装置、4 第3記憶装置、5 第4記憶装置、6 第5記憶装置、10 案内標識検出部、11 特徴検出部、12 案内標識認識部、100 情報入出力インタフェース、101 表示器インタフェース、102 処理回路、103 プロセッサ、104 メモリ。
1 guidance sign recognition device, 2 first storage device, 3rd storage device, 4th 3rd storage device, 5th 4th storage device, 6th 5th storage device, 10 guidance sign detection unit, 11 feature detection unit, 12 guidance sign Recognition unit, 100 information input / output interface, 101 display interface, 102 processing circuit, 103 processor, 104 memory.
Claims (4)
- 撮像画像中の案内標識が存在する領域を検出する案内標識検出部と、
前記案内標識検出部によって検出された領域から案内標識の特徴を検出する特徴検出部と、
前記特徴検出部によって検出された案内標識の特徴と既知の案内標識の特徴とを用いて前記撮像画像中の案内標識と前記既知の案内標識との類似度を算出し、算出した類似度に基づいて、前記既知の案内標識から前記撮像画像中の案内標識を認識する案内標識認識部とを備えたこと
を特徴とする案内標識認識装置。 A guide sign detection unit for detecting a region where a guide sign exists in the captured image;
A feature detection unit for detecting a feature of the guide sign from the area detected by the guide sign detection unit;
Based on the calculated similarity, the similarity between the guide sign in the captured image and the known guide sign is calculated using the feature of the guide sign detected by the feature detection unit and the feature of the known guide sign. And a guide sign recognition unit for recognizing a guide sign in the captured image from the known guide sign. - 前記特徴検出部は、前記案内標識検出部によって検出された領域から、案内標識の色、文字および記号に関する特徴を検出し、
前記案内標識認識部は、前記特徴検出部によって検出された案内標識の色、文字および記号に関する特徴と前記既知の案内標識の色、文字および記号に関する特徴とを用いて、前記撮像画像中の案内標識と前記既知の案内標識との類似度を算出し、算出した類似度に基づいて、前記既知の案内標識から前記撮像画像中の案内標識を認識すること
を特徴とする請求項1記載の案内標識認識装置。 The feature detection unit detects features related to the color, characters and symbols of the guide sign from the area detected by the guide sign detection unit,
The guide sign recognizing unit uses the features related to the color, characters, and symbols of the guide sign detected by the feature detecting unit and the features related to the color, characters, and symbols of the known guide sign, to guide in the captured image. The guide according to claim 1, wherein a similarity between a sign and the known guide sign is calculated, and a guide sign in the captured image is recognized from the known guide sign based on the calculated similarity. Sign recognition device. - 前記特徴検出部は、前記撮像画像中の案内標識の特徴と前記既知の案内標識の特徴とを比較して前記撮像画像中の案内標識における前記既知の案内標識の特徴の有無を判定し、前記既知の案内標識の特徴の有無を判定した結果を用いて、前記撮像画像中の案内標識の特徴を示す特徴ベクトルを生成し、
前記案内標識認識部は、前記既知の案内標識の特徴を示す特徴ベクトルと前記撮像画像中の案内標識の特徴を示す特徴ベクトルとを用いて、前記撮像画像中の案内標識と前記既知の案内標識との類似度を算出し、算出した類似度に基づいて、前記既知の案内標識から前記撮像画像中の案内標識を認識すること
を特徴とする請求項1記載の案内標識認識装置。 The feature detection unit compares the feature of the guide sign in the captured image with the feature of the known guide sign to determine the presence or absence of the feature of the known guide sign in the guide sign in the captured image, Using the result of determining the presence or absence of a feature of a known guide sign, generating a feature vector indicating the feature of the guide sign in the captured image,
The guide sign recognizing unit uses the feature vector indicating the feature of the known guide sign and the feature vector indicating the feature of the guide sign in the captured image, and the guide sign in the captured image and the known guide sign. The guide sign recognition apparatus according to claim 1, wherein a guide sign in the captured image is recognized from the known guide sign based on the calculated similarity. - 案内標識検出部が、撮像画像中の案内標識が存在する領域を検出するステップと、
特徴検出部が、前記案内標識検出部によって検出された領域から案内標識の特徴を検出するステップと、
案内標識認識部が、前記特徴検出部によって検出された案内標識の特徴と既知の案内標識の特徴とを用いて前記撮像画像中の案内標識と前記既知の案内標識との類似度を算出し、算出した類似度に基づいて、前記既知の案内標識から前記撮像画像中の案内標識を認識するステップとを備えたこと
を特徴とする案内標識認識方法。 A guide sign detection unit detecting a region where the guide sign exists in the captured image;
A feature detecting unit detecting a feature of the guide sign from the region detected by the guide sign detecting unit;
The guide sign recognition unit calculates the similarity between the guide sign in the captured image and the known guide sign using the feature of the guide sign detected by the feature detection unit and the feature of the known guide sign, And a step of recognizing a guide sign in the captured image from the known guide sign based on the calculated similarity.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2018/014795 WO2019193762A1 (en) | 2018-04-06 | 2018-04-06 | Guide sign identification device and guide sign identification method |
JP2020511583A JP6762451B2 (en) | 2018-04-06 | 2018-04-06 | Guide sign recognition device and guide sign recognition method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2018/014795 WO2019193762A1 (en) | 2018-04-06 | 2018-04-06 | Guide sign identification device and guide sign identification method |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2019193762A1 true WO2019193762A1 (en) | 2019-10-10 |
Family
ID=68100582
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2018/014795 WO2019193762A1 (en) | 2018-04-06 | 2018-04-06 | Guide sign identification device and guide sign identification method |
Country Status (2)
Country | Link |
---|---|
JP (1) | JP6762451B2 (en) |
WO (1) | WO2019193762A1 (en) |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2013054522A (en) * | 2011-09-02 | 2013-03-21 | Pasuko:Kk | Road appurtenances detecting device, road appurtenances detecting method and program |
JP2015158739A (en) * | 2014-02-21 | 2015-09-03 | 日本電信電話株式会社 | Image sorting device, image classification method, and image classification program |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4380838B2 (en) * | 1999-04-08 | 2009-12-09 | アジア航測株式会社 | Video image automatic road sign recognition method, road sign automatic recognition device, and road sign automatic recognition program |
JP6591257B2 (en) * | 2015-10-21 | 2019-10-16 | 株式会社パスコ | Image processing apparatus, image processing method, and program |
-
2018
- 2018-04-06 JP JP2020511583A patent/JP6762451B2/en not_active Expired - Fee Related
- 2018-04-06 WO PCT/JP2018/014795 patent/WO2019193762A1/en active Application Filing
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2013054522A (en) * | 2011-09-02 | 2013-03-21 | Pasuko:Kk | Road appurtenances detecting device, road appurtenances detecting method and program |
JP2015158739A (en) * | 2014-02-21 | 2015-09-03 | 日本電信電話株式会社 | Image sorting device, image classification method, and image classification program |
Also Published As
Publication number | Publication date |
---|---|
JP6762451B2 (en) | 2020-09-30 |
JPWO2019193762A1 (en) | 2020-09-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10769487B2 (en) | Method and device for extracting information from pie chart | |
JP7051267B2 (en) | Image detection methods, equipment, electronic equipment, storage media, and programs | |
CN109086668B (en) | Unmanned aerial vehicle remote sensing image road information extraction method based on multi-scale generation countermeasure network | |
CN111489403A (en) | Method and device for generating virtual feature map by utilizing GAN | |
WO2017079522A1 (en) | Subcategory-aware convolutional neural networks for object detection | |
CN109582880A (en) | Interest point information processing method, device, terminal and storage medium | |
US20180349716A1 (en) | Apparatus and method for recognizing traffic signs | |
CN112997190A (en) | License plate recognition method and device and electronic equipment | |
CN113420745B (en) | Image-based target identification method, system, storage medium and terminal equipment | |
CN112200193B (en) | Distributed license plate recognition method, system and device based on multi-attribute fusion | |
CN111382625A (en) | Road sign identification method and device and electronic equipment | |
CN112232368B (en) | Target recognition model training method, target recognition method and related devices thereof | |
WO2023138538A1 (en) | Vehicle-mounted video image stabilization method and apparatus, vehicle and storage medium | |
CN116107591A (en) | Deployment model construction method based on corn cases | |
CN112052907A (en) | Target detection method and device based on image edge information and storage medium | |
CN108992033B (en) | Grading device, equipment and storage medium for vision test | |
CN111523351A (en) | Neural network training method and device and electronic equipment | |
CN114120259A (en) | Empty parking space identification method and system, computer equipment and storage medium | |
CN114267076B (en) | Image identification method, device, equipment and storage medium | |
WO2019193762A1 (en) | Guide sign identification device and guide sign identification method | |
CN116580230A (en) | Target detection method and training method of classification model | |
JP2011081614A (en) | Recognition system, recognition method, and program | |
JP2023069083A (en) | Learning apparatus, learning method, learning program, object detection apparatus, object detection method, object detection method, learning support system, learning support method, and learning support program | |
CN109087351B (en) | Method and device for carrying out closed-loop detection on scene picture based on depth information | |
KR20120062168A (en) | Apparatus and method for recogniting sub-trajectory |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 18913981 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2020511583 Country of ref document: JP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 18913981 Country of ref document: EP Kind code of ref document: A1 |