WO2019193762A1 - Guide sign identification device and guide sign identification method - Google Patents

Guide sign identification device and guide sign identification method Download PDF

Info

Publication number
WO2019193762A1
WO2019193762A1 PCT/JP2018/014795 JP2018014795W WO2019193762A1 WO 2019193762 A1 WO2019193762 A1 WO 2019193762A1 JP 2018014795 W JP2018014795 W JP 2018014795W WO 2019193762 A1 WO2019193762 A1 WO 2019193762A1
Authority
WO
WIPO (PCT)
Prior art keywords
guide sign
feature
sign
guide
detection unit
Prior art date
Application number
PCT/JP2018/014795
Other languages
French (fr)
Japanese (ja)
Inventor
秀之 増井
孝之 瀬光
Original Assignee
三菱電機株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 三菱電機株式会社 filed Critical 三菱電機株式会社
Priority to PCT/JP2018/014795 priority Critical patent/WO2019193762A1/en
Priority to JP2020511583A priority patent/JP6762451B2/en
Publication of WO2019193762A1 publication Critical patent/WO2019193762A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present invention relates to a guide sign recognition device and a guide sign recognition method for recognizing a guide sign from an image.
  • Patent Document 1 discloses a result of determining a geometric figure having a road sign characteristic from an object in an image and collating a template image corresponding to the determined geometric figure with a road sign candidate image. An image processing method for recognizing road signs based on the above is described.
  • Patent Document 1 The method described in Patent Document 1 is effective for recognizing a uniform sign such as a speed sign among road signs, but when recognizing a guide sign having various designs, It is necessary to prepare a large number of template images corresponding to various designs. Accordingly, since it is necessary to collate a large number of template images with candidate images, it takes an enormous amount of calculation time to complete recognition.
  • the present invention solves the above-described problems, and an object of the present invention is to provide a guide sign recognition device that can reduce the amount of data required for recognition of guide signs.
  • the guidance sign recognition apparatus includes a guidance sign detection unit, a feature detection unit, and a guidance sign recognition unit.
  • the guide sign detection unit detects a region where the guide sign exists in the captured image.
  • the feature detection unit detects the feature of the guide sign from the area detected by the guide sign detection unit.
  • the guide sign recognition unit calculates the similarity between the guide sign in the captured image and the known guide sign using the feature of the guide sign detected by the feature detection unit and the feature of the known guide sign, and the calculated similarity Based on the degree, the guide sign in the captured image is recognized from the known guide sign.
  • the guide sign recognition device calculates the similarity between the guide sign in the captured image and the known guide sign using the feature of the guide sign in the captured image and the feature of the known guide sign, Based on the similarity, the guide sign in the captured image is recognized from the known guide sign. Since the recognition is performed based on the similarity calculated from the feature of the guide sign, it is not necessary to prepare the entire guide sign image for recognition. Thereby, the data amount required for recognition of a guidance sign can be reduced.
  • FIG. 3A is a diagram showing a guide sign G1.
  • FIG. 3B shows the guide sign G2.
  • FIG. 3C shows the guide sign G3.
  • FIG. 3D is a diagram showing a guide sign G4.
  • FIG. 4A is a diagram showing a guide sign G1 and its features.
  • FIG. 4B is a diagram showing the guide sign G2 and its features.
  • FIG. 4C is a diagram showing the guide sign G3 and its features.
  • FIG. 4D is a diagram showing a guide sign G4 and its features. It is a figure which shows the characteristic of a guidance sign.
  • FIG. 8A is a block diagram showing a hardware configuration for realizing the function of the guidance sign recognition apparatus according to Embodiment 1.
  • FIG. 8B is a block diagram showing a hardware configuration for executing software for realizing the function of the guidance sign recognition apparatus according to Embodiment 1.
  • FIG. 1 is a block diagram showing a configuration of a guide sign recognition apparatus 1 according to Embodiment 1 of the present invention.
  • the guide sign recognition device 1 inputs the captured image stored in the first storage device 2, recognizes the guide sign from the input captured image, and outputs the recognition result to the second storage device 3.
  • the first storage device 2 for example, a captured image outside the vehicle captured by an imaging device mounted on the vehicle is temporarily stored.
  • the second storage device 3 stores the result of automatic recognition of guide signs by the guide sign recognition device 1.
  • the guidance sign recognition device 1 is connected to the third storage device 4, the fourth storage device 5, and the fifth storage device 6.
  • the third storage device 4 stores learning data for detecting an image area of a guide sign from an image obtained by capturing a known guide sign.
  • the fourth storage device 5 stores learning data for detecting a feature of a known guide sign from the guide sign in the captured image.
  • the fifth storage device 6 stores the sign contents of a known guide sign and the feature data of the guide sign.
  • the guidance sign recognition device 1 includes a guidance sign detection unit 10, a feature detection unit 11, and a guidance sign recognition unit 12.
  • the guide sign detection unit 10 is learned in advance so as to detect a region where the guide sign exists from the image using the learning data input from the third storage device 4.
  • the guide sign detection unit 10 receives a captured image from the first storage device 2 and detects a region where the guide sign is present in the input captured image. Data indicating the image area of the guide sign detected by the guide sign detection unit 10 is output to the feature detection unit 11.
  • the feature detection unit 11 is learned in advance so as to detect the feature of the guide sign from the image area detected by the guide sign detection unit 10 using the learning data input from the fourth storage device 5.
  • Guide sign features are various known guide sign features.
  • a plurality of features may be detected by one detector, or each of the plurality of features may be detected by an individual detector.
  • the feature detector 11 receives data indicating the image area of the guide sign from the guide sign detector 10 and detects the feature data of the guide sign from the image area indicated by the input data.
  • the feature data detected by the feature detection unit 11 is output to the guide sign recognition unit 12.
  • the guide sign recognition unit 12 uses the guide sign feature detected by the feature detection unit 11 and the known guide sign feature input from the fifth storage device 6 to use the guide sign in the captured image and the known guide sign.
  • the similarity is calculated.
  • the guide sign recognition unit 12 recognizes the guide sign in the captured image from the known guide sign based on the calculated similarity. For example, the guide sign recognizing unit 12 recognizes that the guide sign having the maximum similarity among the known guide signs is the guide sign in the captured image.
  • FIG. 2 is a flowchart showing a guide sign recognition method according to the first embodiment.
  • the guide sign detection unit 10 detects an area where the guide sign exists in the captured image input from the first storage device 2 (step ST1). For example, the guide sign detection unit 10 detects an image area where the guide sign exists from the captured image using a neural network. The neural network is learned in advance so as to detect a region where a guide sign is present from an image using learning data input from the third storage device 4.
  • the guidance sign detection part 10 may extract the edge part in a captured image, specify a guidance sign, and may detect the rectangular area
  • the feature detection unit 11 detects the feature of the guide sign from the region detected by the guide sign detection unit 10 (step ST2). For example, the feature detection unit 11 detects the feature of the guide sign from the image area of the guide sign using a neural network.
  • the neural network is learned in advance so as to detect the feature of the guide sign from the image area using the learning data input from the fourth storage device 5.
  • the characteristics of the guide signs include characteristics regarding the color, characters, and symbols of the guide signs. As a characteristic regarding the color of the guide sign, there are background colors of various existing guide signs.
  • the characteristic regarding the character of a guidance sign is the character described in the guidance sign. When the character string “ABCD” is described in the guide sign, the characteristics regarding the character of the guide sign are “A”, “B”, “C”, and “D”.
  • the characteristic regarding the sign of a guide sign is the sign described in the guide sign.
  • the feature detection unit 11 compares the features related to the color, characters, and symbols of the guide sign detected from the area detected by the guide sign detection unit 10 with the features related to the learned colors, characters, and symbols of the known guide sign. .
  • the feature detection unit 11 determines the presence / absence of a known guide sign feature in the guide sign in the captured image based on the comparison result between the two, and indicates the feature of the guide sign in the captured image using the result of this determination. Generate a feature vector. For example, the features related to the characters of known guide signs are “A”, “B”, “C”, “D”, and only “A” and “B” are described in the guide signs in the captured image.
  • the element corresponding to the characters “A” and “B” in the feature vector indicating the feature of the guide sign in the captured image is set to a value of 1, and the elements corresponding to the characters “C” and “D” The value 0 is set.
  • the guide sign recognition unit 12 uses the feature of the guide sign detected by the feature detection unit 11 and the feature of the known guide sign input from the fifth storage device 6, and the guide sign in the captured image and the known guide sign And the guide sign in the captured image is recognized from the known guide sign based on the calculated similarity (step ST3).
  • the fifth storage device 6 stores a known guidance sign content and a feature vector indicating the feature of the guidance sign.
  • the guide sign recognition unit 12 calculates the similarity between the feature vector indicating the feature of the guide sign detected by the feature detection unit 11 and the feature vector indicating the feature of the known guide sign, and calculates the similarity. It is recognized that the known guide sign determined to have the highest degree is the guide sign in the captured image.
  • FIG. 3A is a diagram showing a guide sign G1.
  • the guide sign G1 is the number of lanes of a toll booth and ETC (Electronic Toll Collection System; registered trademark, hereinafter abbreviated to be a registered trademark) dedicated lane. It is a guide sign indicating the position.
  • FIG. 3B is a diagram showing a guide sign G2, and the guide sign G2 is a guide sign indicating that it is a general lane that does not use ETC.
  • FIG. 3C is a diagram showing a guide sign G3.
  • the guide sign G3 is a guide sign indicating that the lane is not generally compatible and dedicated to ETC.
  • FIG. 3D is a diagram showing a guide sign G4, which is a guide sign to be recognized by the guide sign recognition apparatus 1.
  • FIG. 4A is a diagram showing a guide sign G1 and its features.
  • FIG. 4B is a diagram showing the guide sign G2 and its features.
  • FIG. 4C is a diagram showing the guide sign G3 and its features.
  • FIG. 4D is a diagram showing a guide sign G4 and its features.
  • the feature C11 shown in FIG. 4A is the color (green) of the guide sign G1
  • the feature C21 shown in FIG. 4B is also the color of the guide sign G2.
  • the color indicated by the feature C11 and the feature C21 is the same.
  • a feature C31 illustrated in FIG. 4C is the color of the guide sign G3, and the color (purple) indicated by the feature C31 is a color different from the colors indicated by the feature C11 and the feature C21.
  • the feature C12 shown in FIG. 4A is a character in the guide sign G1, and this character (E) is the same character as the feature C32 shown in FIG. 4C.
  • a feature C13 shown in FIG. 4A is a character in the guide sign G1, and this character (T) is the same character as the feature C33 shown in FIG. 4C.
  • a feature C14 shown in FIG. 4A is a character in the guide sign G1, and this character (C) is the same character as the feature C34 shown in FIG. 4C.
  • a feature C22 shown in FIG. 4B is a character in the guide sign G2, and this character is different from any of the features C12, C13, C14, and C15 shown in FIG. 4A.
  • a feature C23 illustrated in FIG. 4B is a character in the guide sign G2, and this character is different from any of the feature C12, the feature C13, the feature C14, the feature C15, and the feature C22.
  • a feature C35 shown in FIG. 4C is a character in the guide sign G3, and this character is different from any of the feature C12, the feature C13, the feature C14, the feature C15, the feature C22, and the feature C23.
  • a feature C36 illustrated in FIG. 4C is a character in the guide sign G3, and this character is different from any of the feature C12, the feature C13, the feature C14, the feature C15, the feature C22, the feature C23, and the feature C35. .
  • a feature C16 and a feature C17 shown in FIG. 4A are symbols (arrows) in the guide sign G1.
  • the feature detection unit 11 is learned to detect the above-described features related to colors, characters, and symbols from the images using the captured images of the guide sign G1, the guide sign G2, and the guide sign G3.
  • FIG. 5 is a diagram showing the characteristics of the guide sign.
  • the feature detection unit 11 is set with correspondence data between guide signs and features as shown in FIG. 5 by learning.
  • the columns indicating the color of the guide signs include a color (green) column a1 indicated by the feature C11 and the feature C21 and a color (purple) column a2 indicated by the feature C31. Since both the guide sign G1 and the guide sign G2 have a color corresponding to the column a1, “1” is set in the column a1 and “0” is set in the column a2. Since the guide sign G3 has a color corresponding to the column a2, “1” is set in the column a2, and “0” is set in the column a1.
  • the column indicating the sign of the guide sign includes a column c1 of a symbol (arrow) indicated by the feature C16 and the feature C17. Since the symbols (arrows) indicated by the feature C16 and the feature C17 are not included in the guide sign G2 and the guide sign G3, “1” is set in the column c1 corresponding to the guide sign G1, and the guide sign G2 and the guide sign G3 are displayed. “0” is set in the corresponding column c1.
  • FIG. 6 is a flowchart showing the feature detection process, and shows details of the process in step ST2 of FIG.
  • the feature detection unit 11 the features of the guide signs G1, G2, and G3 shown in FIG. 5 are learned in advance, and the guide sign G4 shown in FIGS. 3D, 4D, and 5D is a recognition target.
  • the feature detection unit 11 detects features relating to the color, characters, and symbols of the guide sign G4 from the region where the guide sign G4 exists in the captured image detected by the guide sign detection unit 10.
  • the feature detection unit 11 may use a convolutional neural network (hereinafter referred to as CNN) as a method for detecting features.
  • CNN convolutional neural network
  • a CNN that learns a combination of an image of a guide sign collected in advance and the contents of the indication of the guide sign is used.
  • the feature detection unit 11 according to the first embodiment uses a CNN that learns a combination of a feature in an image of a guide sign and information (color, character, symbol, etc.) indicated by the feature.
  • the feature detection method is not limited to CNN as long as the feature of the guide sign can be detected from the captured image of the guide sign.
  • step ST1a-1 If the color green information sign G4 (step ST1a-1; Yes), the feature detection unit 11 sets "1" to the feature x 1 (step ST2a-1). On the other hand, if the color of the signposts G4 is familiar in green (step ST1a-1; No), the feature detection unit 11 sets "0" to the feature x 1 (step ST2a-2). Subsequently, although not shown, the feature detection unit 11 determines whether the color of the guide sign G4 is purple. Here, since the color indicated by the feature C41 shown in FIG. 4D is green, as shown in FIG. 5, “1” is set in the column a1 of the guide sign G4, and “0” is set in the column a2. Is done.
  • the feature detection unit 11 sets “1” to the feature xI + 1 (step ST2a-3). If the guide sign G4 does not include the letter “E” (step ST1a-2; No), the feature detection unit 11 sets “0” for the feature xI + 1 (step ST2a-4). Subsequently, although not shown, the feature detection unit 11 determines whether or not the guide sign G4 includes the letter “T”.
  • the feature C42 shown in FIG. 4D is the character “E” in the guide sign G4, and the character indicated by the feature C42 of the guide sign G4 and the character indicated by the feature C12 of the guide sign G1 are the same characters. “1” is set in the corresponding column b1.
  • the feature C43 shown in FIG. 4D is the character “T” in the guide sign G4, and the character indicated by the feature C43 of the guide sign G4 and the character indicated by the feature C13 of the guide sign G1 are the same character. “1” is set in the field b2.
  • 4D is the character “C” in the guide sign G4, and the character indicated by the feature C44 of the guide sign G4 and the character indicated by the feature C14 of the guide sign G1 are the same characters. “1” is set in the field b3.
  • the feature C45 shown in FIG. 4D is a character in the guide sign G4, and the character indicated by the feature C45 of the guide sign G4 and the character indicated by the feature C15 of the guide sign G1 are the same character. Is set to “1”.
  • the guide sign G4 does not include the character indicated by the feature C22 shown in FIG. 4B, the character indicated by the feature C23, the character indicated by the feature C35 shown in FIG. 4C, and the character indicated by the feature C36, as shown in FIG. “0” is set in the columns b5 to b8 of the guide sign G4.
  • the feature detection unit 11 sets “1” to the feature xI + J + 1 (step ST2a-5). If the guide sign G4 does not include the symbol “arrow” (step ST1a-3; No), the feature detection unit 11 sets “0” to the feature xI + J + 1 (step ST2a-6).
  • the symbol indicated by the feature C46 illustrated in FIG. 4D is an arrow, “1” is set in the column c1 of the guide sign G4 as illustrated in FIG.
  • the feature detection unit 11 When the presence or absence of the feature of the learning data in the guidance sign G4 is determined, the feature detection unit 11 generates a feature vector x having the digital value set according to the presence or absence of the feature as an element and outputs the feature vector x to the guidance sign recognition unit 12 ( Step ST3a).
  • FIG. 7 is a flowchart showing guidance sign recognition processing, and shows details of step ST3 in FIG.
  • the guide sign recognizing unit 12 inputs the feature vector x of the recognition target guide sign generated by the feature detecting unit 11 (step ST1b). Thereafter, the guide sign recognition unit 12 proceeds to a repetition loop from 1 to D. D is the number of known guide signs stored in the fifth storage device 6.
  • x is a feature vector of the guide sign a
  • the guide sign recognition unit 12 performs the similarity c 1 between the first guide sign and the recognition target guide sign to the similarity c D between the Dth guide sign and the recognition target guide sign. After calculating the among these similarities c d, signs corresponding to the maximum similarity, and outputs the recognized as the signs of the recognition target (step ST4b).
  • the cosine similarity between the feature vector of the guide sign G1 and the feature vector of the guide sign G4 to be recognized is 1.0. It becomes.
  • the cosine similarity between the feature vector of the guide sign G2 and the feature vector of the guide sign G4 is 0.24
  • the cosine similarity between the feature vector of the guide sign G3 and the feature vector of the guide sign G4 is 0.5. It becomes.
  • the guide sign recognition unit 12 recognizes that the guide sign G4 is the guide sign G1 because the similarity of the guide sign G1 is the highest.
  • the conventional recognition method could not recognize a guide sign with a different design from the learning data.
  • the similarity between the guidance sign to be recognized and the guidance sign of the learning data is determined based on the feature of the guidance sign. For this reason, even if the guidance sign G4 to be recognized has a different design from the guidance sign G1 that is the learning data, it includes many of the same features, and is thus recognized as a guidance sign with the same marking content.
  • the guidance sign recognition apparatus 1 includes a processing circuit for executing the processing from step ST1 to step ST3 in FIG.
  • This processing circuit may be dedicated hardware, or may be a CPU (Central Processing Unit) that executes a program stored in a memory.
  • CPU Central Processing Unit
  • FIG. 8A is a block diagram showing a hardware configuration for realizing the function of the guide sign recognition apparatus 1.
  • FIG. 8B is a block diagram showing a hardware configuration for executing software for realizing the function of the guidance sign recognition apparatus 1.
  • the information input / output interface 100 includes the guide sign recognition device 1, the first storage device 2, the second storage device 3, the third storage device 4, the fourth storage device 5 and the like shown in FIG. This is an interface that relays information exchange with the fifth storage device 6.
  • the guidance sign detection unit 10 inputs the image data of the guidance sign to be recognized from the first storage device 2 and the learning data from the third storage device 4 via the information input / output interface 100.
  • the feature detection unit 11 inputs learning data from the fourth storage device 5 via the information input / output interface 100.
  • the guide sign recognition unit 12 inputs characteristic data of known guide signs from the fifth storage device 6 via the information input / output interface 100 and outputs the recognition result to the second storage device 3.
  • the first storage device 2, the second storage device 3, the third storage device 4, the fourth storage device 5, and the fifth storage device 6 may be storage devices included in the guidance sign recognition device 1.
  • a storage device provided independently of the sign recognition device 1 may be used. These storage devices may be storage devices provided so as to be communicable from the guide sign recognition device 1, for example, storage devices existing on the cloud.
  • the display interface 101 is an interface that relays information output from the guide sign recognition apparatus 1 to a display (not shown).
  • the guide sign recognition unit 12 outputs the recognition result to the display device via the display device interface 101.
  • the display unit displays the recognition result input from the guide sign recognition apparatus 1 via the display unit interface 101.
  • the processing circuit 102 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated), or the like. Circuit), FPGA (Field-Programmable Gate Array), or a combination thereof.
  • the functions of the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12 in the guide sign recognition device 1 may be realized by separate processing circuits, and these functions are collectively realized by one processing circuit. Also good.
  • the functions of the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12 in the guide sign recognition device 1 are software, firmware, or a combination of software and firmware. It is realized by.
  • the software or firmware is described as a program and stored in the memory 104.
  • the processor 103 reads out and executes the program stored in the memory 104, thereby realizing the functions of the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12 in the guide sign recognition device 1. That is, the guidance sign recognition apparatus 1 includes a memory 104 for storing a program that, when executed by the processor 103, results in the processing from step ST1 to step ST3 shown in FIG. These programs cause the computer to execute the procedure or method of the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12.
  • the memory 104 may be a computer-readable storage medium storing a program for causing a computer to function as the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12.
  • the memory 104 includes, for example, a nonvolatile memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically-EPROM), or a volatile memory such as an EEPROM (Electrically-EPROM).
  • a nonvolatile memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically-EPROM), or a volatile memory such as an EEPROM (Electrically-EPROM).
  • a nonvolatile memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically-EPROM), or a volatile memory such as an EEPROM (Electrically-EPROM).
  • EEPROM Electrically
  • the functions of the guidance sign detection unit 10, the feature detection unit 11, and the guidance sign recognition unit 12 may be partially realized with dedicated hardware and partly realized with software or firmware.
  • the guidance sign detection unit 10 realizes the function by a processing circuit that is dedicated hardware, and the feature detection unit 11 and the guidance sign recognition unit 12 read and execute a program stored in the memory 104 by the processor 103. To realize the function.
  • the processing circuit can realize the above functions by hardware, software, firmware, or a combination thereof.
  • the guide sign in the captured image and the known guide sign are used by using the feature of the guide sign in the captured image and the feature of the known guide sign.
  • the similarity is calculated, and the guide sign in the captured image is recognized from the known guide sign based on the calculated similarity. Since the recognition is performed based on the similarity calculated from the feature of the guide sign, it is not necessary to prepare the entire guide sign image for recognition. Thereby, the data amount required for recognition of a guidance sign can be reduced.
  • the guidance sign recognition apparatus can reduce the amount of data necessary for recognition of the guidance sign, it can be used, for example, in a driving assistance apparatus that automatically recognizes the guidance sign and supports driving.
  • 1 guidance sign recognition device 2 first storage device, 3rd storage device, 4th 3rd storage device, 5th 4th storage device, 6th 5th storage device, 10 guidance sign detection unit, 11 feature detection unit, 12 guidance sign Recognition unit, 100 information input / output interface, 101 display interface, 102 processing circuit, 103 processor, 104 memory.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

This guide sign identification device (1) uses the features of a guide sign in a captured image and the features of an existing guide sign to calculate the similarity between the features of the guide sign in the captured image and the features of the existing guide sign, and identifies the guide sign in the captured image from the existing guide sign on the basis of the calculated similarity.

Description

案内標識認識装置および案内標識認識方法Guide sign recognition apparatus and guide sign recognition method
　この発明は、画像から案内標識を認識する案内標識認識装置および案内標識認識方法に関する。 The present invention relates to a guide sign recognition device and a guide sign recognition method for recognizing a guide sign from an image.
　従来から、画像から道路標識を認識する技術が提案されている。
　例えば、特許文献１には、画像中のオブジェクトから道路標識の特徴を有した幾何学的図形を判定し、判定した幾何学的図形に対応したテンプレート画像と道路標識の候補画像とを照合した結果に基づいて道路標識を認識する画像処理方法が記載されている。 Conventionally, techniques for recognizing road signs from images have been proposed.
For example, Patent Document 1 discloses a result of determining a geometric figure having a road sign characteristic from an object in an image and collating a template image corresponding to the determined geometric figure with a road sign candidate image. An image processing method for recognizing road signs based on the above is described.
特開２０１７－７８９８９号公報JP 2017-78989 A
　特許文献１に記載された方法は、道路標識のうち、速度標識のように画一的なデザインの標識の認識には有効であるが、多様なデザインが存在する案内標識を認識する場合は、多様なデザインに対応した多数のテンプレート画像を用意する必要がある。これにより、多数のテンプレート画像と候補画像とを照合する必要があるため、認識が完了するまでに膨大な計算時間がかかる。 The method described in Patent Document 1 is effective for recognizing a uniform sign such as a speed sign among road signs, but when recognizing a guide sign having various designs, It is necessary to prepare a large number of template images corresponding to various designs. Accordingly, since it is necessary to collate a large number of template images with candidate images, it takes an enormous amount of calculation time to complete recognition.
　この発明は上記課題を解決するものであって、案内標識の認識に要するデータ量を削減することができる案内標識認識装置を得ることを目的とする。 The present invention solves the above-described problems, and an object of the present invention is to provide a guide sign recognition device that can reduce the amount of data required for recognition of guide signs.
　この発明に係る案内標識認識装置は、案内標識検出部、特徴検出部および案内標識認識部を備える。案内標識検出部は、撮像画像中の案内標識が存在する領域を検出する。特徴検出部は、案内標識検出部によって検出された領域から案内標識の特徴を検出する。案内標識認識部が、特徴検出部によって検出された案内標識の特徴と既知の案内標識の特徴とを用いて撮像画像中の案内標識と既知の案内標識との類似度を算出し、算出した類似度に基づいて、既知の案内標識から撮像画像中の案内標識を認識する。 The guidance sign recognition apparatus according to the present invention includes a guidance sign detection unit, a feature detection unit, and a guidance sign recognition unit. The guide sign detection unit detects a region where the guide sign exists in the captured image. The feature detection unit detects the feature of the guide sign from the area detected by the guide sign detection unit. The guide sign recognition unit calculates the similarity between the guide sign in the captured image and the known guide sign using the feature of the guide sign detected by the feature detection unit and the feature of the known guide sign, and the calculated similarity Based on the degree, the guide sign in the captured image is recognized from the known guide sign.
　この発明によれば、案内標識認識装置は、撮像画像中の案内標識の特徴と既知の案内標識の特徴とを用いて撮像画像中の案内標識と既知の案内標識との類似度を算出し、類似度に基づいて、既知の案内標識から撮像画像中の案内標識を認識する。案内標識の特徴から算出した類似度を基準として認識を行うので、認識のために案内標識の画像全体を用意する必要はない。これにより、案内標識の認識に必要なデータ量を削減することができる。 According to this invention, the guide sign recognition device calculates the similarity between the guide sign in the captured image and the known guide sign using the feature of the guide sign in the captured image and the feature of the known guide sign, Based on the similarity, the guide sign in the captured image is recognized from the known guide sign. Since the recognition is performed based on the similarity calculated from the feature of the guide sign, it is not necessary to prepare the entire guide sign image for recognition. Thereby, the data amount required for recognition of a guidance sign can be reduced.
この発明の実施の形態１に係る案内標識認識装置の構成を示すブロック図である。It is a block diagram which shows the structure of the guidance sign recognition apparatus which concerns on Embodiment 1 of this invention. 実施の形態１に係る案内標識認識方法を示すフローチャートである。4 is a flowchart showing a guide sign recognition method according to the first embodiment. 図３Ａは、案内標識Ｇ１を示す図である。図３Ｂは、案内標識Ｇ２を示す図である。図３Ｃは、案内標識Ｇ３を示す図である。図３Ｄは、案内標識Ｇ４を示す図である。FIG. 3A is a diagram showing a guide sign G1. FIG. 3B shows the guide sign G2. FIG. 3C shows the guide sign G3. FIG. 3D is a diagram showing a guide sign G4. 図４Ａは、案内標識Ｇ１およびその特徴を示す図である。図４Ｂは、案内標識Ｇ２およびその特徴を示す図である。図４Ｃは、案内標識Ｇ３およびその特徴を示す図である。図４Ｄは、案内標識Ｇ４およびその特徴を示す図である。FIG. 4A is a diagram showing a guide sign G1 and its features. FIG. 4B is a diagram showing the guide sign G2 and its features. FIG. 4C is a diagram showing the guide sign G3 and its features. FIG. 4D is a diagram showing a guide sign G4 and its features. 案内標識の特徴を示す図である。It is a figure which shows the characteristic of a guidance sign. 特徴検出処理を示すフローチャートである。It is a flowchart which shows a feature detection process. 案内標識認識処理を示すフローチャートである。It is a flowchart which shows a guidance sign recognition process. 図８Ａは、実施の形態１に係る案内標識認識装置の機能を実現するハードウェア構成を示すブロック図である。図８Ｂは、実施の形態１に係る案内標識認識装置の機能を実現するソフトウェアを実行するハードウェア構成を示すブロック図である。FIG. 8A is a block diagram showing a hardware configuration for realizing the function of the guidance sign recognition apparatus according to Embodiment 1. FIG. 8B is a block diagram showing a hardware configuration for executing software for realizing the function of the guidance sign recognition apparatus according to Embodiment 1.
　以下、この発明をより詳細に説明するため、この発明を実施するための形態について、添付の図面に従って説明する。
実施の形態１．
　図１は、この発明の実施の形態１に係る案内標識認識装置１の構成を示すブロック図である。案内標識認識装置１は、第１記憶装置２に記憶された撮像画像を入力し、入力した撮像画像から案内標識を認識し、認識結果を第２記憶装置３に出力する。第１記憶装置２には、例えば、車両に搭載された撮像装置によって撮像された車外の撮像画像が一時的に記憶される。第２記憶装置３には、案内標識認識装置１による案内標識の自動認識の結果が記憶される。 Hereinafter, in order to describe the present invention in more detail, modes for carrying out the present invention will be described with reference to the accompanying drawings.
Embodiment 1 FIG.
FIG. 1 is a block diagram showing a configuration of a guide sign recognition apparatus 1 according to Embodiment 1 of the present invention. The guide sign recognition device 1 inputs the captured image stored in the first storage device 2, recognizes the guide sign from the input captured image, and outputs the recognition result to the second storage device 3. In the first storage device 2, for example, a captured image outside the vehicle captured by an imaging device mounted on the vehicle is temporarily stored. The second storage device 3 stores the result of automatic recognition of guide signs by the guide sign recognition device 1.
　案内標識認識装置１は、第３記憶装置４、第４記憶装置５および第５記憶装置６に接続されている。第３記憶装置４には、既知の案内標識が撮像された画像から案内標識の画像領域を検出するための学習データが記憶されている。第４記憶装置５には、撮像画像中の案内標識から既知の案内標識の特徴を検出するための学習データが記憶されている。第５記憶装置６には、既知の案内標識の標示内容とこの案内標識の特徴データとが記憶されている。 The guidance sign recognition device 1 is connected to the third storage device 4, the fourth storage device 5, and the fifth storage device 6. The third storage device 4 stores learning data for detecting an image area of a guide sign from an image obtained by capturing a known guide sign. The fourth storage device 5 stores learning data for detecting a feature of a known guide sign from the guide sign in the captured image. The fifth storage device 6 stores the sign contents of a known guide sign and the feature data of the guide sign.
　案内標識認識装置１は、案内標識検出部１０、特徴検出部１１および案内標識認識部１２を備える。案内標識検出部１０は、第３記憶装置４から入力した学習データを用いて、画像から案内標識が存在する領域を検出するように事前に学習されている。案内標識検出部１０は、第１記憶装置２から撮像画像を入力し、入力した撮像画像中の案内標識が存在する領域を検出する。案内標識検出部１０によって検出された案内標識の画像領域を示すデータは、特徴検出部１１に出力される。 The guidance sign recognition device 1 includes a guidance sign detection unit 10, a feature detection unit 11, and a guidance sign recognition unit 12. The guide sign detection unit 10 is learned in advance so as to detect a region where the guide sign exists from the image using the learning data input from the third storage device 4. The guide sign detection unit 10 receives a captured image from the first storage device 2 and detects a region where the guide sign is present in the input captured image. Data indicating the image area of the guide sign detected by the guide sign detection unit 10 is output to the feature detection unit 11.
　特徴検出部１１は、第４記憶装置５から入力した学習データを用いて、案内標識検出部１０によって検出された画像領域から案内標識の特徴を検出するように事前に学習されている。案内標識の特徴は、既知の様々な案内標識の特徴である。特徴の検出方法は、複数の特徴を１つの検出器で検出してもよく、複数の特徴のそれぞれについて個別の検出器で検出してもよい。特徴検出部１１は、案内標識検出部１０から案内標識の画像領域を示すデータを入力し、入力したデータが示す画像領域から案内標識の特徴データを検出する。特徴検出部１１によって検出された特徴データは、案内標識認識部１２に出力される。 The feature detection unit 11 is learned in advance so as to detect the feature of the guide sign from the image area detected by the guide sign detection unit 10 using the learning data input from the fourth storage device 5. Guide sign features are various known guide sign features. In the feature detection method, a plurality of features may be detected by one detector, or each of the plurality of features may be detected by an individual detector. The feature detector 11 receives data indicating the image area of the guide sign from the guide sign detector 10 and detects the feature data of the guide sign from the image area indicated by the input data. The feature data detected by the feature detection unit 11 is output to the guide sign recognition unit 12.
　案内標識認識部１２は、特徴検出部１１によって検出された案内標識の特徴と第５記憶装置６から入力した既知の案内標識の特徴とを用いて、撮像画像中の案内標識と既知の案内標識との類似度を算出する。案内標識認識部１２は、算出した類似度に基づいて既知の案内標識から撮像画像中の案内標識を認識する。例えば、案内標識認識部１２は、既知の案内標識のうち、類似度が最大の案内標識が撮像画像中の案内標識であると認識する。 The guide sign recognition unit 12 uses the guide sign feature detected by the feature detection unit 11 and the known guide sign feature input from the fifth storage device 6 to use the guide sign in the captured image and the known guide sign. The similarity is calculated. The guide sign recognition unit 12 recognizes the guide sign in the captured image from the known guide sign based on the calculated similarity. For example, the guide sign recognizing unit 12 recognizes that the guide sign having the maximum similarity among the known guide signs is the guide sign in the captured image.
　次に動作について説明する。
　図２は、実施の形態１に係る案内標識認識方法を示すフローチャートである。
　案内標識検出部１０は、第１記憶装置２から入力した撮像画像中の案内標識が存在する領域を検出する（ステップＳＴ１）。例えば、案内標識検出部１０は、ニューラルネットワークを用いて撮像画像から案内標識が存在する画像領域を検出する。ニューラルネットワークは、第３記憶装置４から入力した学習データを用いて画像から案内標識が存在する領域を検出するように事前に学習されている。なお、案内標識検出部１０は、撮像画像中のエッジ部分を抽出して案内標識を特定し、特定した案内標識が含まれる矩形領域を検出してもよい。 Next, the operation will be described.
FIG. 2 is a flowchart showing a guide sign recognition method according to the first embodiment.
The guide sign detection unit 10 detects an area where the guide sign exists in the captured image input from the first storage device 2 (step ST1). For example, the guide sign detection unit 10 detects an image area where the guide sign exists from the captured image using a neural network. The neural network is learned in advance so as to detect a region where a guide sign is present from an image using learning data input from the third storage device 4. In addition, the guidance sign detection part 10 may extract the edge part in a captured image, specify a guidance sign, and may detect the rectangular area | region where the specified guidance sign is included.
　特徴検出部１１は、案内標識検出部１０によって検出された領域から、案内標識の特徴を検出する（ステップＳＴ２）。例えば、特徴検出部１１は、ニューラルネットワークを用いて案内標識の画像領域から案内標識の特徴を検出する。ニューラルネットワークは、第４記憶装置５から入力した学習データを用いて画像領域から案内標識の特徴を検出するように事前に学習されている。案内標識の特徴には、案内標識の色、文字および記号に関する特徴がある。案内標識の色に関する特徴としては、既存の様々な案内標識の背景色が挙げられる。案内標識の文字に関する特徴は、案内標識に記載されている文字である。案内標識に“ＡＢＣＤ”という文字列が記載されていた場合、案内標識の文字に関する特徴は“Ａ”、“Ｂ”、“Ｃ”、“Ｄ”である。案内標識の記号に関する特徴は、案内標識に記載されている記号である。 The feature detection unit 11 detects the feature of the guide sign from the region detected by the guide sign detection unit 10 (step ST2). For example, the feature detection unit 11 detects the feature of the guide sign from the image area of the guide sign using a neural network. The neural network is learned in advance so as to detect the feature of the guide sign from the image area using the learning data input from the fourth storage device 5. The characteristics of the guide signs include characteristics regarding the color, characters, and symbols of the guide signs. As a characteristic regarding the color of the guide sign, there are background colors of various existing guide signs. The characteristic regarding the character of a guidance sign is the character described in the guidance sign. When the character string “ABCD” is described in the guide sign, the characteristics regarding the character of the guide sign are “A”, “B”, “C”, and “D”. The characteristic regarding the sign of a guide sign is the sign described in the guide sign.
　特徴検出部１１は、案内標識検出部１０によって検出された領域から検出した案内標識の色、文字および記号に関する特徴と、学習済みの既知の案内標識の色、文字および記号に関する特徴とを比較する。特徴検出部１１は、両者の比較結果に基づいて、撮像画像中の案内標識における既知の案内標識の特徴の有無を判定し、この判定の結果を用いて撮像画像中の案内標識の特徴を示す特徴ベクトルを生成する。例えば、既知の案内標識の文字に関する特徴は、“Ａ”、“Ｂ”、“Ｃ”、“Ｄ”であり、撮像画像中の案内標識には、“Ａ”および“Ｂ”のみが記載されていた場合、撮像画像中の案内標識の特徴を示す特徴ベクトルにおける文字“Ａ”および“Ｂ”に対応する要素に値１が設定され、文字“Ｃ”および“Ｄ”に対応する要素には値０が設定される。 The feature detection unit 11 compares the features related to the color, characters, and symbols of the guide sign detected from the area detected by the guide sign detection unit 10 with the features related to the learned colors, characters, and symbols of the known guide sign. . The feature detection unit 11 determines the presence / absence of a known guide sign feature in the guide sign in the captured image based on the comparison result between the two, and indicates the feature of the guide sign in the captured image using the result of this determination. Generate a feature vector. For example, the features related to the characters of known guide signs are “A”, “B”, “C”, “D”, and only “A” and “B” are described in the guide signs in the captured image. In this case, the element corresponding to the characters “A” and “B” in the feature vector indicating the feature of the guide sign in the captured image is set to a value of 1, and the elements corresponding to the characters “C” and “D” The value 0 is set.
　案内標識認識部１２は、特徴検出部１１によって検出された案内標識の特徴と第５記憶装置６から入力した既知の案内標識の特徴とを用いて撮像画像中の案内標識と既知の案内標識との類似度を算出し、算出した類似度に基づいて、既知の案内標識から撮像画像中の案内標識を認識する（ステップＳＴ３）。例えば、第５記憶装置６には、既知の案内標識の標示内容とこの案内標識の特徴を示す特徴ベクトルが記憶されている。案内標識認識部１２は、特徴検出部１１によって検出された案内標識の特徴を示す特徴ベクトルと、既知の案内標識の特徴を示す特徴ベクトルとを用いて両者の類似度を算出し、算出した類似度が最も大きいと判定された既知の案内標識が撮像画像中の案内標識であると認識する。 The guide sign recognition unit 12 uses the feature of the guide sign detected by the feature detection unit 11 and the feature of the known guide sign input from the fifth storage device 6, and the guide sign in the captured image and the known guide sign And the guide sign in the captured image is recognized from the known guide sign based on the calculated similarity (step ST3). For example, the fifth storage device 6 stores a known guidance sign content and a feature vector indicating the feature of the guidance sign. The guide sign recognition unit 12 calculates the similarity between the feature vector indicating the feature of the guide sign detected by the feature detection unit 11 and the feature vector indicating the feature of the known guide sign, and calculates the similarity. It is recognized that the known guide sign determined to have the highest degree is the guide sign in the captured image.
　次に、特徴検出処理について詳細に説明する。
　図３Ａは、案内標識Ｇ１を示す図であり、案内標識Ｇ１は、料金所のレーン数とＥＴＣ（Ｅｌｅｃｔｒｏｎｉｃ　Ｔｏｌｌ　Ｃｏｌｌｅｃｔｉｏｎ　Ｓｙｓｔｅｍ；登録商標、以下、登録商標であることの記載を省略する）専用レーンの位置を示す案内標識である。図３Ｂは、案内標識Ｇ２を示す図であり、案内標識Ｇ２は、ＥＴＣを使用しない一般用のレーンであることを示す案内標識である。図３Ｃは、案内標識Ｇ３を示す図であり、案内標識Ｇ３は、一般に対応しておらず、ＥＴＣ専用のレーンであることを示す案内標識である。図３Ｄは、案内標識Ｇ４を示す図であり、案内標識Ｇ４は、案内標識認識装置１の認識対象の案内標識である。 Next, the feature detection process will be described in detail.
FIG. 3A is a diagram showing a guide sign G1. The guide sign G1 is the number of lanes of a toll booth and ETC (Electronic Toll Collection System; registered trademark, hereinafter abbreviated to be a registered trademark) dedicated lane. It is a guide sign indicating the position. FIG. 3B is a diagram showing a guide sign G2, and the guide sign G2 is a guide sign indicating that it is a general lane that does not use ETC. FIG. 3C is a diagram showing a guide sign G3. The guide sign G3 is a guide sign indicating that the lane is not generally compatible and dedicated to ETC. FIG. 3D is a diagram showing a guide sign G4, which is a guide sign to be recognized by the guide sign recognition apparatus 1.
　特徴検出部１１は、案内標識Ｇ１、案内標識Ｇ２および案内標識Ｇ３のそれぞれの撮像画像を用いて、画像から、色、文字および記号に関する特徴を検出するように学習されているものとする。図４Ａは、案内標識Ｇ１およびその特徴を示す図である。図４Ｂは、案内標識Ｇ２およびその特徴を示す図である。図４Ｃは、案内標識Ｇ３およびその特徴を示す図である。図４Ｄは、案内標識Ｇ４およびその特徴を示す図である。 Suppose that the feature detection unit 11 has been learned to detect features related to colors, characters, and symbols from images using the respective captured images of the guide sign G1, the guide sign G2, and the guide sign G3. FIG. 4A is a diagram showing a guide sign G1 and its features. FIG. 4B is a diagram showing the guide sign G2 and its features. FIG. 4C is a diagram showing the guide sign G3 and its features. FIG. 4D is a diagram showing a guide sign G4 and its features.
　図４Ａに示す特徴Ｃ１１は、案内標識Ｇ１の色（緑）であり、図４Ｂに示す特徴Ｃ２１も案内標識Ｇ２の色である。特徴Ｃ１１と特徴Ｃ２１が示す色は同じである。図４Ｃに示す特徴Ｃ３１は、案内標識Ｇ３の色であり、特徴Ｃ３１が示す色（紫）は、特徴Ｃ１１と特徴Ｃ２１が示す色とは異なる色である。 The feature C11 shown in FIG. 4A is the color (green) of the guide sign G1, and the feature C21 shown in FIG. 4B is also the color of the guide sign G2. The color indicated by the feature C11 and the feature C21 is the same. A feature C31 illustrated in FIG. 4C is the color of the guide sign G3, and the color (purple) indicated by the feature C31 is a color different from the colors indicated by the feature C11 and the feature C21.
　図４Ａに示す特徴Ｃ１２は、案内標識Ｇ１中の文字であり、この文字（Ｅ）は、図４Ｃに示す特徴Ｃ３２と同じ文字である。図４Ａに示す特徴Ｃ１３は、案内標識Ｇ１中の文字であり、この文字（Ｔ）は、図４Ｃに示す特徴Ｃ３３と同じ文字である。図４Ａに示す特徴Ｃ１４は、案内標識Ｇ１中の文字であり、この文字（Ｃ）は、図４Ｃに示す特徴Ｃ３４と同じ文字である。 The feature C12 shown in FIG. 4A is a character in the guide sign G1, and this character (E) is the same character as the feature C32 shown in FIG. 4C. A feature C13 shown in FIG. 4A is a character in the guide sign G1, and this character (T) is the same character as the feature C33 shown in FIG. 4C. A feature C14 shown in FIG. 4A is a character in the guide sign G1, and this character (C) is the same character as the feature C34 shown in FIG. 4C.
　図４Ｂに示す特徴Ｃ２２は、案内標識Ｇ２中の文字であり、この文字は、図４Ａに示す特徴Ｃ１２、特徴Ｃ１３、特徴Ｃ１４および特徴Ｃ１５のいずれとも異なる文字である。図４Ｂに示す特徴Ｃ２３は、案内標識Ｇ２中の文字であり、この文字は、特徴Ｃ１２、特徴Ｃ１３、特徴Ｃ１４、特徴Ｃ１５および特徴Ｃ２２のいずれとも異なる文字である。 A feature C22 shown in FIG. 4B is a character in the guide sign G2, and this character is different from any of the features C12, C13, C14, and C15 shown in FIG. 4A. A feature C23 illustrated in FIG. 4B is a character in the guide sign G2, and this character is different from any of the feature C12, the feature C13, the feature C14, the feature C15, and the feature C22.
　図４Ｃに示す特徴Ｃ３５は、案内標識Ｇ３中の文字であり、この文字は、特徴Ｃ１２、特徴Ｃ１３、特徴Ｃ１４、特徴Ｃ１５、特徴Ｃ２２および特徴Ｃ２３のいずれとも異なる文字である。また、図４Ｃに示す特徴Ｃ３６は、案内標識Ｇ３中の文字であり、この文字は、特徴Ｃ１２、特徴Ｃ１３、特徴Ｃ１４、特徴Ｃ１５、特徴Ｃ２２、特徴Ｃ２３および特徴Ｃ３５のいずれとも異なる文字である。 A feature C35 shown in FIG. 4C is a character in the guide sign G3, and this character is different from any of the feature C12, the feature C13, the feature C14, the feature C15, the feature C22, and the feature C23. A feature C36 illustrated in FIG. 4C is a character in the guide sign G3, and this character is different from any of the feature C12, the feature C13, the feature C14, the feature C15, the feature C22, the feature C23, and the feature C35. .
　図４Ａに示す特徴Ｃ１６および特徴Ｃ１７は、案内標識Ｇ１中の記号（矢印）である。
　特徴検出部１１は、案内標識Ｇ１、案内標識Ｇ２および案内標識Ｇ３のそれぞれの撮像画像を用いて、画像から、前述した、色、文字および記号に関する特徴を検出するように学習されている。図５は、案内標識の特徴を示す図である。特徴検出部１１には、学習によって、図５に示すような案内標識と特徴との対応データが設定されている。 A feature C16 and a feature C17 shown in FIG. 4A are symbols (arrows) in the guide sign G1.
The feature detection unit 11 is learned to detect the above-described features related to colors, characters, and symbols from the images using the captured images of the guide sign G1, the guide sign G2, and the guide sign G3. FIG. 5 is a diagram showing the characteristics of the guide sign. The feature detection unit 11 is set with correspondence data between guide signs and features as shown in FIG. 5 by learning.
　図５において、案内標識の色を示す欄には、特徴Ｃ１１と特徴Ｃ２１が示す色（緑）の欄ａ１と特徴Ｃ３１が示す色（紫）の欄ａ２とがある。案内標識Ｇ１および案内標識Ｇ２は、両方とも欄ａ１に対応する色を有するので、欄ａ１に“１”が設定され、欄ａ２には“０”が設定されている。案内標識Ｇ３は、欄ａ２に対応する色を有するので、欄ａ２に“１”が設定され、欄ａ１には“０”が設定されている。 In FIG. 5, the columns indicating the color of the guide signs include a color (green) column a1 indicated by the feature C11 and the feature C21 and a color (purple) column a2 indicated by the feature C31. Since both the guide sign G1 and the guide sign G2 have a color corresponding to the column a1, “1” is set in the column a1 and “0” is set in the column a2. Since the guide sign G3 has a color corresponding to the column a2, “1” is set in the column a2, and “0” is set in the column a1.
　図５において、案内標識の文字を示す欄には、特徴Ｃ１２および特徴Ｃ３２が示す文字の欄ｂ１、特徴Ｃ１３と特徴Ｃ３３が示す文字の欄ｂ２、特徴Ｃ１４と特徴Ｃ３４が示す文字の欄ｂ３、特徴Ｃ１５が示す文字の欄ｂ４、特徴Ｃ２２が示す文字の欄ｂ５、特徴Ｃ２３が示す文字の欄ｂ６、特徴Ｃ３５が示す文字の欄ｂ７、および特徴Ｃ３６が示す文字の欄ｂ８がある。 In FIG. 5, in the column indicating the characters of the guide signs, the character column b1 indicated by the feature C12 and the feature C32, the character column b2 indicated by the feature C13 and the feature C33, the character column b3 indicated by the feature C14 and the feature C34, There are a character column b4 indicated by the feature C15, a character column b5 indicated by the feature C22, a character column b6 indicated by the feature C23, a character column b7 indicated by the feature C35, and a character column b8 indicated by the feature C36.
　案内標識Ｇ１の特徴Ｃ１２が示す文字と案内標識Ｇ３の特徴Ｃ３２が示す文字とは同じ文字（Ｅ）であることから、案内標識Ｇ１と案内標識Ｇ３の欄ｂ１に“１”が設定され、案内標識Ｇ２には、特徴Ｃ１２が示す文字（Ｅ）はないので、欄ｂ１には“０”が設定されている。 Since the character indicated by the feature C12 of the guide sign G1 and the character indicated by the feature C32 of the guide sign G3 are the same character (E), “1” is set in the column b1 of the guide sign G1 and the guide sign G3, and the guide Since the sign G2 does not have the character (E) indicated by the feature C12, “0” is set in the column b1.
　案内標識Ｇ１の特徴Ｃ１３が示す文字と案内標識Ｇ３の特徴Ｃ３３が示す文字とは同じ文字であることから、案内標識Ｇ１と案内標識Ｇ３の欄ｂ２に“１”が設定され、案内標識Ｇ２には、特徴Ｃ１３が示す文字はないので、欄ｂ２には“０”が設定されている。
　案内標識Ｇ１の特徴Ｃ１４が示す文字と案内標識Ｇ３の特徴Ｃ３４が示す文字とは同じ文字であることから、案内標識Ｇ１と案内標識Ｇ３の欄ｂ３に“１”が設定され、案内標識Ｇ２には、特徴Ｃ１４が示す文字はないので、欄ｂ３には“０”が設定されている。 Since the character indicated by the feature C13 of the guide sign G1 and the character indicated by the feature C33 of the guide sign G3 are the same character, “1” is set in the column b2 of the guide sign G1 and the guide sign G3. Since there is no character indicated by the feature C13, "0" is set in the field b2.
Since the character indicated by the feature C14 of the guide sign G1 and the character indicated by the feature C34 of the guide sign G3 are the same character, “1” is set in the column b3 of the guide sign G1 and the guide sign G3, and the guide sign G2 is displayed. Since there is no character indicated by the feature C14, "0" is set in the field b3.
　案内標識Ｇ１の特徴Ｃ１５が示す文字は、案内標識Ｇ２と案内標識Ｇ３にはないので、案内標識Ｇ１に対応する欄ｂ４には“１”が設定され、案内標識Ｇ２および案内標識Ｇ３に対応する欄ｂ４には“０”が設定されている。案内標識Ｇ２の特徴Ｃ２２が示す文字は案内標識Ｇ１と案内標識Ｇ３にはないので、案内標識Ｇ２に対応する欄ｂ５には、“１”が設定され、案内標識Ｇ１および案内標識Ｇ３に対応する欄ｂ５には“０”が設定されている。 Since the character indicated by the feature C15 of the guide sign G1 is not in the guide sign G2 and the guide sign G3, “1” is set in the column b4 corresponding to the guide sign G1, and the characters corresponding to the guide sign G2 and the guide sign G3 are set. In the column b4, “0” is set. Since the character indicated by the feature C22 of the guide sign G2 is not in the guide sign G1 and the guide sign G3, “1” is set in the column b5 corresponding to the guide sign G2, and the characters corresponding to the guide sign G1 and the guide sign G3 are set. In the column b5, “0” is set.
　案内標識Ｇ２の特徴Ｃ２３が示す文字は、案内標識Ｇ１と案内標識Ｇ３にはないので、案内標識Ｇ２に対応する欄ｂ６には“１”が設定され、案内標識Ｇ１および案内標識Ｇ３に対応する欄ｂ６には“０”が設定されている。案内標識Ｇ３の特徴Ｃ３５が示す文字は案内標識Ｇ１と案内標識Ｇ２にはないので、案内標識Ｇ３に対応する欄ｂ７には、“１”が設定され、案内標識Ｇ１および案内標識Ｇ２に対応する欄ｂ７には“０”が設定されている。 Since the character indicated by the feature C23 of the guide sign G2 is not in the guide sign G1 and the guide sign G3, “1” is set in the column b6 corresponding to the guide sign G2, and the characters corresponding to the guide sign G1 and the guide sign G3 are set. In the column b6, “0” is set. Since the character indicated by the feature C35 of the guide sign G3 is not in the guide sign G1 and the guide sign G2, “1” is set in the column b7 corresponding to the guide sign G3, and the characters corresponding to the guide sign G1 and the guide sign G2 are set. In the column b7, “0” is set.
　案内標識Ｇ３の特徴Ｃ３６が示す文字は、案内標識Ｇ１と案内標識Ｇ２にはないので、案内標識Ｇ３に対応する欄ｂ８には、“１”が設定され、案内標識Ｇ１および案内標識Ｇ２に対応する欄ｂ８には“０”が設定されている。 Since the character indicated by the feature C36 of the guide sign G3 does not exist in the guide sign G1 and the guide sign G2, “1” is set in the column b8 corresponding to the guide sign G3, and corresponds to the guide sign G1 and the guide sign G2. “0” is set in the column b8 to be performed.
　図５において、案内標識の記号を示す欄には、特徴Ｃ１６および特徴Ｃ１７が示す記号（矢印）の欄ｃ１がある。特徴Ｃ１６および特徴Ｃ１７が示す記号（矢印）は、案内標識Ｇ２および案内標識Ｇ３にはないので、案内標識Ｇ１に対応する欄ｃ１には“１”が設定され、案内標識Ｇ２および案内標識Ｇ３に対応する欄ｃ１には“０”が設定されている。 In FIG. 5, the column indicating the sign of the guide sign includes a column c1 of a symbol (arrow) indicated by the feature C16 and the feature C17. Since the symbols (arrows) indicated by the feature C16 and the feature C17 are not included in the guide sign G2 and the guide sign G3, “1” is set in the column c1 corresponding to the guide sign G1, and the guide sign G2 and the guide sign G3 are displayed. “0” is set in the corresponding column c1.
　図６は、特徴検出処理を示すフローチャートであって、図２のステップＳＴ２の処理の詳細を示している。特徴検出部１１には、図５に示した案内標識Ｇ１，Ｇ２，Ｇ３の特徴が事前に学習されており、図３Ｄ、図４Ｄおよび図５Ｄに示した案内標識Ｇ４が認識対象である。まず、特徴検出部１１は、案内標識検出部１０によって検出された撮像画像中の案内標識Ｇ４が存在する領域から、案内標識Ｇ４の色、文字および記号に関する特徴を検出する。 FIG. 6 is a flowchart showing the feature detection process, and shows details of the process in step ST2 of FIG. In the feature detection unit 11, the features of the guide signs G1, G2, and G3 shown in FIG. 5 are learned in advance, and the guide sign G4 shown in FIGS. 3D, 4D, and 5D is a recognition target. First, the feature detection unit 11 detects features relating to the color, characters, and symbols of the guide sign G4 from the region where the guide sign G4 exists in the captured image detected by the guide sign detection unit 10.
　特徴検出部１１では、特徴を検出する手法として、畳み込みニューラルネットワーク（以下、ＣＮＮと記載する）を用いてもよい。従来のＣＮＮを用いた画像認識では、事前に収集した案内標識の画像と案内標識の標示内容との組み合わせを学習させたＣＮＮが用いられる。これに対して、実施の形態１における特徴検出部１１では、案内標識の画像中の特徴とその特徴が示す情報（色、文字、記号など）との組み合わせを学習させたＣＮＮを用いる。これにより、学習データである既知の案内標識と認識対象の未知の案内標識とのデザインが異なっていても、両者が同じ特徴を有していれば、その特徴が検出される。
　なお、特徴を検出する手法は、案内標識の撮像画像から案内標識の特徴を検出することができる手法であれば、ＣＮＮに限定されるものではない。 The feature detection unit 11 may use a convolutional neural network (hereinafter referred to as CNN) as a method for detecting features. In conventional image recognition using a CNN, a CNN that learns a combination of an image of a guide sign collected in advance and the contents of the indication of the guide sign is used. On the other hand, the feature detection unit 11 according to the first embodiment uses a CNN that learns a combination of a feature in an image of a guide sign and information (color, character, symbol, etc.) indicated by the feature. Thereby, even if the design of the known guide sign that is the learning data and the unknown guide sign to be recognized are different, the feature is detected if both have the same feature.
Note that the feature detection method is not limited to CNN as long as the feature of the guide sign can be detected from the captured image of the guide sign.
　特徴検出部１１は、案内標識Ｇ４の色に関するＩ（Ｉは、１以上の自然数）個の設問として、案内標識Ｇ４の色が緑か否かを判定する（ステップＳＴ１ａ－１）。前述した例では、学習データの案内標識の色として緑と紫が学習されており、Ｉ＝１，２である。特徴検出部１１は、案内標識Ｇ４の色が緑であるか否かの判定と、案内標識Ｇ４の色が紫であるか否かの判定とを行う。 The feature detection unit 11 determines whether the color of the guide sign G4 is green as I questions (I is a natural number of 1 or more) regarding the color of the guide sign G4 (step ST1a-1). In the example described above, green and purple are learned as the colors of the guide signs of the learning data, and I = 1, 2. The feature detection unit 11 determines whether the color of the guide sign G4 is green and determines whether the color of the guide sign G4 is purple.
　案内標識Ｇ４の色が緑であれば（ステップＳＴ１ａ－１；Ｙｅｓ）、特徴検出部１１は特徴ｘ１に“１”を設定する（ステップＳＴ２ａ－１）。一方、案内標識Ｇ４の色が緑ではなれば（ステップＳＴ１ａ－１；Ｎｏ）、特徴検出部１１は特徴ｘ１に“０”を設定する（ステップＳＴ２ａ－２）。続いて、図示を省略したが、特徴検出部１１は、案内標識Ｇ４の色が紫か否かを判定する。ここでは、図４Ｄに示した特徴Ｃ４１が示す色は緑であるので、図５に示すように、案内標識Ｇ４の欄ａ１には“１”が設定され、欄ａ２には“０”が設定される。 If the color green information sign G4 (step ST1a-1; Yes), the feature detection unit 11 sets "1" to the feature x 1 (step ST2a-1). On the other hand, if the color of the signposts G4 is familiar in green (step ST1a-1; No), the feature detection unit 11 sets "0" to the feature x 1 (step ST2a-2). Subsequently, although not shown, the feature detection unit 11 determines whether the color of the guide sign G4 is purple. Here, since the color indicated by the feature C41 shown in FIG. 4D is green, as shown in FIG. 5, “1” is set in the column a1 of the guide sign G4, and “0” is set in the column a2. Is done.
　特徴検出部１１は、案内標識Ｇ４の文字に関するＪ（Ｊは、１以上の自然数）個の設問として、案内標識Ｇ４が文字“Ｅ”を含むか否かを判定する（ステップＳＴ１ａ－２）。学習データの案内標識の文字として、図５に示した欄ｂ１～ｂ８に対応する８つの文字が学習されており、Ｊ＝１，２，３，４，５，６，７，８である。特徴検出部１１は、これらの欄に対応する文字を含むか否かの判定を順次行う。 The feature detection unit 11 determines whether or not the guide sign G4 includes the letter “E” as J questions (J is a natural number of 1 or more) regarding the character of the guide sign G4 (step ST1a-2). Eight characters corresponding to the columns b1 to b8 shown in FIG. 5 have been learned as the guidance sign characters of the learning data, and J = 1, 2, 3, 4, 5, 6, 7, 8. The feature detection unit 11 sequentially determines whether or not the characters corresponding to these fields are included.
　案内標識Ｇ４が文字“Ｅ”を含む場合（ステップＳＴ１ａ－２；Ｙｅｓ）、特徴検出部１１は特徴ｘＩ＋１に“１”を設定する（ステップＳＴ２ａ－３）。案内標識Ｇ４が文字“Ｅ”を含まなければ（ステップＳＴ１ａ－２；Ｎｏ）、特徴検出部１１は、特徴ｘＩ＋１に“０”を設定する（ステップＳＴ２ａ－４）。続いて、図示を省略したが、特徴検出部１１は、案内標識Ｇ４が文字“Ｔ”を含むか否かを判定する。 When the guide sign G4 includes the character “E” (step ST1a-2; Yes), the feature detection unit 11 sets “1” to the feature xI + 1 (step ST2a-3). If the guide sign G4 does not include the letter “E” (step ST1a-2; No), the feature detection unit 11 sets “0” for the feature xI + 1 (step ST2a-4). Subsequently, although not shown, the feature detection unit 11 determines whether or not the guide sign G4 includes the letter “T”.
　図４Ｄに示した特徴Ｃ４２は案内標識Ｇ４中の文字“Ｅ”であり、案内標識Ｇ４の特徴Ｃ４２が示す文字と案内標識Ｇ１の特徴Ｃ１２が示す文字は同じ文字であるので、案内標識Ｇ４に対応する欄ｂ１に“１”が設定される。図４Ｄに示した特徴Ｃ４３は案内標識Ｇ４中の文字“Ｔ”であり、案内標識Ｇ４の特徴Ｃ４３が示す文字と案内標識Ｇ１の特徴Ｃ１３が示す文字は同じ文字であるので、案内標識Ｇ４の欄ｂ２に“１”が設定される。図４Ｄに示した特徴Ｃ４４は案内標識Ｇ４中の文字“Ｃ”であり、案内標識Ｇ４の特徴Ｃ４４が示す文字と案内標識Ｇ１の特徴Ｃ１４が示す文字は同じ文字であるので、案内標識Ｇ４の欄ｂ３に“１”が設定される。図４Ｄに示した特徴Ｃ４５は案内標識Ｇ４中の文字であり、案内標識Ｇ４の特徴Ｃ４５が示す文字と案内標識Ｇ１の特徴Ｃ１５が示す文字は同じ文字であるので、案内標識Ｇ４の欄ｂ４には“１”が設定される。 The feature C42 shown in FIG. 4D is the character “E” in the guide sign G4, and the character indicated by the feature C42 of the guide sign G4 and the character indicated by the feature C12 of the guide sign G1 are the same characters. “1” is set in the corresponding column b1. The feature C43 shown in FIG. 4D is the character “T” in the guide sign G4, and the character indicated by the feature C43 of the guide sign G4 and the character indicated by the feature C13 of the guide sign G1 are the same character. “1” is set in the field b2. The feature C44 shown in FIG. 4D is the character “C” in the guide sign G4, and the character indicated by the feature C44 of the guide sign G4 and the character indicated by the feature C14 of the guide sign G1 are the same characters. “1” is set in the field b3. The feature C45 shown in FIG. 4D is a character in the guide sign G4, and the character indicated by the feature C45 of the guide sign G4 and the character indicated by the feature C15 of the guide sign G1 are the same character. Is set to “1”.
　案内標識Ｇ４は、図４Ｂに示した特徴Ｃ２２が示す文字、特徴Ｃ２３が示す文字、図４Ｃに示した特徴Ｃ３５が示す文字、および特徴Ｃ３６が示す文字を含まないので、図５に示すように、案内標識Ｇ４の欄ｂ５～ｂ８には“０”が設定される。 Since the guide sign G4 does not include the character indicated by the feature C22 shown in FIG. 4B, the character indicated by the feature C23, the character indicated by the feature C35 shown in FIG. 4C, and the character indicated by the feature C36, as shown in FIG. “0” is set in the columns b5 to b8 of the guide sign G4.
　特徴検出部１１は、案内標識Ｇ４の記号に関するＫ（Ｋは、１以上の自然数）個の設問として、案内標識Ｇ４が記号“矢印”を含むか否かを判定する（ステップＳＴ１ａ－３）。前述した例では、学習データの案内標識の記号として矢印が学習されており、Ｋ＝１である。特徴検出部１１は、案内標識Ｇ４に記号“矢印”が含まれるか否かを判定する。 The feature detection unit 11 determines whether or not the guide sign G4 includes the symbol “arrow” as K questions (K is a natural number of 1 or more) regarding the sign of the guide sign G4 (step ST1a-3). In the above-described example, an arrow is learned as a guide sign symbol of learning data, and K = 1. The feature detection unit 11 determines whether or not the symbol “arrow” is included in the guide sign G4.
　案内標識Ｇ４が記号“矢印”を含む場合（ステップＳＴ１ａ－３；Ｙｅｓ）、特徴検出部１１は特徴ｘＩ＋Ｊ＋１に“１”を設定する（ステップＳＴ２ａ－５）。案内標識Ｇ４が記号“矢印”を含まなければ（ステップＳＴ１ａ－３；Ｎｏ）、特徴検出部１１は、特徴ｘＩ＋Ｊ＋１に“０”を設定する（ステップＳＴ２ａ－６）。ここでは、図４Ｄに示した特徴Ｃ４６が示す記号は矢印であるので、図５に示すように、案内標識Ｇ４の欄ｃ１には“１”が設定される。 When the guide sign G4 includes the symbol “arrow” (step ST1a-3; Yes), the feature detection unit 11 sets “1” to the feature xI + J + 1 (step ST2a-5). If the guide sign G4 does not include the symbol “arrow” (step ST1a-3; No), the feature detection unit 11 sets “0” to the feature xI + J + 1 (step ST2a-6). Here, since the symbol indicated by the feature C46 illustrated in FIG. 4D is an arrow, “1” is set in the column c1 of the guide sign G4 as illustrated in FIG.
　案内標識Ｇ４における学習データの特徴の有無を判定すると、特徴検出部１１は、特徴の有無に応じて設定したデジタル値を要素とした特徴ベクトルｘを生成して案内標識認識部１２に出力する（ステップＳＴ３ａ）。特徴ベクトルｘは、下記式（１）で表すことができる。図５に示すように、図４Ａに示した案内標識Ｇ１と図４Ｄに示した案内標識Ｇ４とでは、デザインが異なるものの、同じ特徴を有している。
　ｘ＝（ｘ１，・・・，ｘＩ，ｘＩ＋１，・・・，ｘＩ＋Ｊ，ｘＩ＋Ｊ＋１）Ｔ　　・・・（１） When the presence or absence of the feature of the learning data in the guidance sign G4 is determined, the feature detection unit 11 generates a feature vector x having the digital value set according to the presence or absence of the feature as an element and outputs the feature vector x to the guidance sign recognition unit 12 ( Step ST3a). The feature vector x can be expressed by the following equation (1). As shown in FIG. 5, the guide sign G1 shown in FIG. 4A and the guide sign G4 shown in FIG.
x = (x 1 ,..., x I , x I + 1 ,..., x I + J , x I + J + 1 ) T (1)
　次に、案内標識認識処理について詳細に説明する。
　図７は、案内標識認識処理を示すフローチャートであり、図２のステップＳＴ３の詳細を示している。案内標識認識部１２は、特徴検出部１１によって生成された認識対象の案内標識の特徴ベクトルｘを入力する（ステップＳＴ１ｂ）。この後、案内標識認識部１２は、１～Ｄまでの繰り返しループに移行する。Ｄは、第５記憶装置６に記憶された既知の案内標識の数である。 Next, the guidance sign recognition process will be described in detail.
FIG. 7 is a flowchart showing guidance sign recognition processing, and shows details of step ST3 in FIG. The guide sign recognizing unit 12 inputs the feature vector x of the recognition target guide sign generated by the feature detecting unit 11 (step ST1b). Thereafter, the guide sign recognition unit 12 proceeds to a repetition loop from 1 to D. D is the number of known guide signs stored in the fifth storage device 6.
　案内標識認識部１２は、第５記憶装置６から第ｄ（ｄ＝１～Ｄ）番目の案内標識の特徴ベクトルｙｄを入力する（ステップＳＴ２ｂ）。次に、案内標識認識部１２は、認識対象の案内標識の特徴ベクトルｘと既知の案内標識の特徴ベクトルｙｄとを用いて、認識対象の案内標識と既知の案内標識との類似度ｃｄを算出する（ステップＳＴ３ｂ）。例えば、案内標識ａと案内標識ｂの類似度は、下記式（２）で求められるコサイン類似度を用いてもよい。下記式（２）において、ｘは案内標識ａの特徴ベクトルであり、ｙは案内標識ｂの特徴ベクトルである。
　ｃｏｓ（案内標識ａ，案内標識ｂ）＝ｘ・ｙ／（｜ｘ｜｜ｙ｜）　　　・・・（２） Signpost recognition unit 12 from the fifth storage device 6 inputs the first d (d = 1 ~ D) th signs feature vector y d (step ST2b). Then, information sign recognition unit 12 uses the feature vector y d of the feature vector x and the known signs of signs to be recognized, the similarity c d between signs and known signs to be recognized Is calculated (step ST3b). For example, the cosine similarity obtained by the following formula (2) may be used as the similarity between the guide sign a and the guide sign b. In the following formula (2), x is a feature vector of the guide sign a, and y is a feature vector of the guide sign b.
cos (guide sign a, guide sign b) = x · y / (| x || y |) (2)
　案内標識認識部１２は、上記繰り返しループにおいて、第１番目の案内標識と認識対象の案内標識との類似度ｃ１から第Ｄ番目の案内標識と認識対象の案内標識との類似度ｃＤまでを算出すると、これらの類似度ｃｄのうち、最大の類似度に対応する案内標識が認識対象の案内標識であると認識して出力する（ステップＳＴ４ｂ）。 In the above iterative loop, the guide sign recognition unit 12 performs the similarity c 1 between the first guide sign and the recognition target guide sign to the similarity c D between the Dth guide sign and the recognition target guide sign. After calculating the among these similarities c d, signs corresponding to the maximum similarity, and outputs the recognized as the signs of the recognition target (step ST4b).
　例えば、上記式（１）および上記式（２）と図５に示した値を用いると、案内標識Ｇ１の特徴ベクトルと認識対象の案内標識Ｇ４の特徴ベクトルとのコサイン類似度は、１．０となる。一方、案内標識Ｇ２の特徴ベクトルと案内標識Ｇ４の特徴ベクトルとのコサイン類似度は、０．２４となり、案内標識Ｇ３の特徴ベクトルと案内標識Ｇ４の特徴ベクトルとのコサイン類似度は、０．５となる。案内標識認識部１２は、案内標識Ｇ１の類似度が最も大きいので、案内標識Ｇ４が案内標識Ｇ１であると認識される。 For example, using the above formulas (1) and (2) and the values shown in FIG. 5, the cosine similarity between the feature vector of the guide sign G1 and the feature vector of the guide sign G4 to be recognized is 1.0. It becomes. On the other hand, the cosine similarity between the feature vector of the guide sign G2 and the feature vector of the guide sign G4 is 0.24, and the cosine similarity between the feature vector of the guide sign G3 and the feature vector of the guide sign G4 is 0.5. It becomes. The guide sign recognition unit 12 recognizes that the guide sign G4 is the guide sign G1 because the similarity of the guide sign G1 is the highest.
　従来の認識手法では、学習データと異なるデザインの案内標識を認識することができなかった。これに対して、実施の形態１に係る案内標識認識装置１では、案内標識の特徴に基づいて認識対象の案内標識と学習データの案内標識との類似度を判断する。このため、認識対象の案内標識Ｇ４が、学習データである案内標識Ｇ１とは異なるデザインであっても、同じ特徴を多く含んでいるため、同じ標示内容の案内標識であると認識される。 The conventional recognition method could not recognize a guide sign with a different design from the learning data. On the other hand, in the guidance sign recognition apparatus 1 according to the first embodiment, the similarity between the guidance sign to be recognized and the guidance sign of the learning data is determined based on the feature of the guidance sign. For this reason, even if the guidance sign G4 to be recognized has a different design from the guidance sign G1 that is the learning data, it includes many of the same features, and is thus recognized as a guidance sign with the same marking content.
　なお、案内標識の特徴として、色、文字および記号を例に挙げたが、特徴から類似度の算出が可能であれば、これら３つの特徴よりも少なくても多くてもよい。 Note that colors, characters, and symbols are given as examples of the features of the guide signs. However, if the similarity can be calculated from the features, the number may be less or more than these three features.
　案内標識認識装置１における案内標識検出部１０、特徴検出部１１および案内標識認識部１２の機能は、処理回路によって実現される。すなわち、案内標識認識装置１は、図２におけるステップＳＴ１からステップＳＴ３までの処理を実行するための処理回路を備えている。この処理回路は、専用のハードウェアであってもよいが、メモリに記憶されたプログラムを実行するＣＰＵ（Ｃｅｎｔｒａｌ　Ｐｒｏｃｅｓｓｉｎｇ　Ｕｎｉｔ）であってもよい。 The functions of the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12 in the guide sign recognition device 1 are realized by a processing circuit. That is, the guidance sign recognition apparatus 1 includes a processing circuit for executing the processing from step ST1 to step ST3 in FIG. This processing circuit may be dedicated hardware, or may be a CPU (Central Processing Unit) that executes a program stored in a memory.
　図８Ａは、案内標識認識装置１の機能を実現するハードウェア構成を示すブロック図である。図８Ｂは、案内標識認識装置１の機能を実現するソフトウェアを実行するハードウェア構成を示すブロック図である。図８Ａおよび図８Ｂにおいて、情報入出力インタフェース１００は、案内標識認識装置１と、図１に示した第１記憶装置２、第２記憶装置３、第３記憶装置４、第４記憶装置５および第５記憶装置６との間の情報のやり取りを中継するインタフェースである。 FIG. 8A is a block diagram showing a hardware configuration for realizing the function of the guide sign recognition apparatus 1. FIG. 8B is a block diagram showing a hardware configuration for executing software for realizing the function of the guidance sign recognition apparatus 1. 8A and 8B, the information input / output interface 100 includes the guide sign recognition device 1, the first storage device 2, the second storage device 3, the third storage device 4, the fourth storage device 5 and the like shown in FIG. This is an interface that relays information exchange with the fifth storage device 6.
　例えば、案内標識検出部１０は、情報入出力インタフェース１００を介して、第１記憶装置２から認識対象の案内標識の画像データを入力し、第３記憶装置４から学習データを入力する。特徴検出部１１は、情報入出力インタフェース１００を介して、第４記憶装置５から学習データを入力する。案内標識認識部１２は、情報入出力インタフェース１００を介して、第５記憶装置６から既知の案内標識の特徴データを入力し、認識結果を第２記憶装置３に出力する。なお、第１記憶装置２、第２記憶装置３、第３記憶装置４、第４記憶装置５および第５記憶装置６は、案内標識認識装置１が備える記憶装置であってもよいが、案内標識認識装置１とは独立して設けられた記憶装置であってもよい。これらの記憶装置は、案内標識認識装置１から通信可能に設けられた記憶装置、例えば、クラウド上に存在する記憶装置であってもよい。 For example, the guidance sign detection unit 10 inputs the image data of the guidance sign to be recognized from the first storage device 2 and the learning data from the third storage device 4 via the information input / output interface 100. The feature detection unit 11 inputs learning data from the fourth storage device 5 via the information input / output interface 100. The guide sign recognition unit 12 inputs characteristic data of known guide signs from the fifth storage device 6 via the information input / output interface 100 and outputs the recognition result to the second storage device 3. The first storage device 2, the second storage device 3, the third storage device 4, the fourth storage device 5, and the fifth storage device 6 may be storage devices included in the guidance sign recognition device 1. A storage device provided independently of the sign recognition device 1 may be used. These storage devices may be storage devices provided so as to be communicable from the guide sign recognition device 1, for example, storage devices existing on the cloud.
　表示器インタフェース１０１は、案内標識認識装置１から不図示の表示器へ出力される情報を中継するインタフェースである。案内標識認識部１２は、表示器インタフェース１０１を介して認識結果を表示器に出力する。表示器は、表示器インタフェース１０１を介して案内標識認識装置１から入力した認識結果を表示する。 The display interface 101 is an interface that relays information output from the guide sign recognition apparatus 1 to a display (not shown). The guide sign recognition unit 12 outputs the recognition result to the display device via the display device interface 101. The display unit displays the recognition result input from the guide sign recognition apparatus 1 via the display unit interface 101.
　上記処理回路が図８Ａに示す専用のハードウェアの処理回路１０２である場合、処理回路１０２は、例えば、単一回路、複合回路、プログラム化したプロセッサ、並列プログラム化したプロセッサ、ＡＳＩＣ（Ａｐｐｌｉｃａｔｉｏｎ　Ｓｐｅｃｉｆｉｃ　Ｉｎｔｅｇｒａｔｅｄ　Ｃｉｒｃｕｉｔ）、ＦＰＧＡ（Ｆｉｅｌｄ－Ｐｒｏｇｒａｍｍａｂｌｅ　Ｇａｔｅ　Ａｒｒａｙ）、または、これらを組み合わせたものが該当する。案内標識認識装置１における案内標識検出部１０、特徴検出部１１および案内標識認識部１２の機能を別々の処理回路で実現してもよく、これらの機能をまとめて１つの処理回路で実現してもよい。 When the processing circuit is the dedicated hardware processing circuit 102 shown in FIG. 8A, the processing circuit 102 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated), or the like. Circuit), FPGA (Field-Programmable Gate Array), or a combination thereof. The functions of the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12 in the guide sign recognition device 1 may be realized by separate processing circuits, and these functions are collectively realized by one processing circuit. Also good.
　上記処理回路が図８Ｂに示すプロセッサ１０３である場合、案内標識認識装置１における案内標識検出部１０、特徴検出部１１および案内標識認識部１２の機能は、ソフトウェア、ファームウェアまたはソフトウェアとファームウェアとの組み合わせによって実現される。なお、ソフトウェアまたはファームウェアは、プログラムとして記述されてメモリ１０４に記憶される。 When the processing circuit is the processor 103 shown in FIG. 8B, the functions of the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12 in the guide sign recognition device 1 are software, firmware, or a combination of software and firmware. It is realized by. The software or firmware is described as a program and stored in the memory 104.
　プロセッサ１０３は、メモリ１０４に記憶されたプログラムを読み出して実行することにより、案内標識認識装置１における案内標識検出部１０、特徴検出部１１および案内標識認識部１２の機能を実現する。すなわち、案内標識認識装置１は、プロセッサ１０３によって実行されるときに、図２に示したステップＳＴ１からステップＳＴ３までの処理が結果的に実行されるプログラムを記憶するためのメモリ１０４を備える。これらのプログラムは、案内標識検出部１０、特徴検出部１１および案内標識認識部１２の手順または方法をコンピュータに実行させる。メモリ１０４は、コンピュータを、案内標識検出部１０、特徴検出部１１および案内標識認識部１２として機能させるためのプログラムが記憶されたコンピュータ可読記憶媒体であってもよい。 The processor 103 reads out and executes the program stored in the memory 104, thereby realizing the functions of the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12 in the guide sign recognition device 1. That is, the guidance sign recognition apparatus 1 includes a memory 104 for storing a program that, when executed by the processor 103, results in the processing from step ST1 to step ST3 shown in FIG. These programs cause the computer to execute the procedure or method of the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12. The memory 104 may be a computer-readable storage medium storing a program for causing a computer to function as the guide sign detection unit 10, the feature detection unit 11, and the guide sign recognition unit 12.
　メモリ１０４には、例えば、ＲＡＭ（Ｒａｎｄｏｍ　Ａｃｃｅｓｓ　Ｍｅｍｏｒｙ）、ＲＯＭ（Ｒｅａｄ　Ｏｎｌｙ　Ｍｅｍｏｒｙ）、フラッシュメモリ、ＥＰＲＯＭ（Ｅｒａｓａｂｌｅ　Ｐｒｏｇｒａｍｍａｂｌｅ　Ｒｅａｄ　Ｏｎｌｙ　Ｍｅｍｏｒｙ）、ＥＥＰＲＯＭ（Ｅｌｅｃｔｒｉｃａｌｌｙ－ＥＰＲＯＭ）などの不揮発性または揮発性の半導体メモリ、磁気ディスク、フレキシブルディスク、光ディスク、コンパクトディスク、ミニディスク、ＤＶＤなどが該当する。 The memory 104 includes, for example, a nonvolatile memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically-EPROM), or a volatile memory such as an EEPROM (Electrically-EPROM). Magnetic disks, flexible disks, optical disks, compact disks, mini disks, DVDs, and the like are applicable.
　案内標識検出部１０、特徴検出部１１および案内標識認識部１２の機能について一部を専用のハードウェアで実現し、一部をソフトウェアまたはファームウェアで実現してもよい。例えば、案内標識検出部１０は、専用のハードウェアである処理回路で機能を実現し、特徴検出部１１および案内標識認識部１２は、プロセッサ１０３がメモリ１０４に記憶されたプログラムを読み出して実行することによって機能を実現する。このように、処理回路は、ハードウェア、ソフトウェア、ファームウェアまたはこれらの組み合わせにより上記機能を実現することができる。 The functions of the guidance sign detection unit 10, the feature detection unit 11, and the guidance sign recognition unit 12 may be partially realized with dedicated hardware and partly realized with software or firmware. For example, the guidance sign detection unit 10 realizes the function by a processing circuit that is dedicated hardware, and the feature detection unit 11 and the guidance sign recognition unit 12 read and execute a program stored in the memory 104 by the processor 103. To realize the function. As described above, the processing circuit can realize the above functions by hardware, software, firmware, or a combination thereof.
　以上のように、実施の形態１に係る案内標識認識装置１では、撮像画像中の案内標識の特徴と既知の案内標識の特徴とを用いて撮像画像中の案内標識と既知の案内標識との類似度を算出し、算出した類似度に基づいて既知の案内標識から撮像画像中の案内標識を認識する。案内標識の特徴から算出した類似度を基準として認識を行うので、認識のために案内標識の画像全体を用意する必要はない。これにより、案内標識の認識に必要なデータ量を削減することができる。 As described above, in the guide sign recognition apparatus 1 according to Embodiment 1, the guide sign in the captured image and the known guide sign are used by using the feature of the guide sign in the captured image and the feature of the known guide sign. The similarity is calculated, and the guide sign in the captured image is recognized from the known guide sign based on the calculated similarity. Since the recognition is performed based on the similarity calculated from the feature of the guide sign, it is not necessary to prepare the entire guide sign image for recognition. Thereby, the data amount required for recognition of a guidance sign can be reduced.
　なお、本発明は上記実施の形態に限定されるものではなく、本発明の範囲内において、実施の形態のそれぞれの自由な組み合わせまたは実施の形態のそれぞれの任意の構成要素の変形もしくは実施の形態のそれぞれにおいて任意の構成要素の省略が可能である。 It should be noted that the present invention is not limited to the above-described embodiment, and within the scope of the present invention, each free combination of the embodiments or any component modification or embodiment of the embodiments. It is possible to omit arbitrary components in each of the above.
　この発明に係る案内標識認識装置は、案内標識の認識に必要なデータ量を削減することができるので、例えば、案内標識を自動認識して運転を支援する運転支援装置に利用可能である。 Since the guidance sign recognition apparatus according to the present invention can reduce the amount of data necessary for recognition of the guidance sign, it can be used, for example, in a driving assistance apparatus that automatically recognizes the guidance sign and supports driving.
　１　案内標識認識装置、２　第１記憶装置、３　第２記憶装置、４　第３記憶装置、５　第４記憶装置、６　第５記憶装置、１０　案内標識検出部、１１　特徴検出部、１２　案内標識認識部、１００　情報入出力インタフェース、１０１　表示器インタフェース、１０２　処理回路、１０３　プロセッサ、１０４　メモリ。 1 guidance sign recognition device, 2 first storage device, 3rd storage device, 4th 3rd storage device, 5th 4th storage device, 6th 5th storage device, 10 guidance sign detection unit, 11 feature detection unit, 12 guidance sign Recognition unit, 100 information input / output interface, 101 display interface, 102 processing circuit, 103 processor, 104 memory.

Claims (4)

  1. 　撮像画像中の案内標識が存在する領域を検出する案内標識検出部と、
    　前記案内標識検出部によって検出された領域から案内標識の特徴を検出する特徴検出部と、
    　前記特徴検出部によって検出された案内標識の特徴と既知の案内標識の特徴とを用いて前記撮像画像中の案内標識と前記既知の案内標識との類似度を算出し、算出した類似度に基づいて、前記既知の案内標識から前記撮像画像中の案内標識を認識する案内標識認識部とを備えたこと
    　を特徴とする案内標識認識装置。     A guide sign detection unit for detecting a region where a guide sign exists in the captured image;
    A feature detection unit for detecting a feature of the guide sign from the area detected by the guide sign detection unit;
    Based on the calculated similarity, the similarity between the guide sign in the captured image and the known guide sign is calculated using the feature of the guide sign detected by the feature detection unit and the feature of the known guide sign. And a guide sign recognition unit for recognizing a guide sign in the captured image from the known guide sign.
  2. 　前記特徴検出部は、前記案内標識検出部によって検出された領域から、案内標識の色、文字および記号に関する特徴を検出し、
    　前記案内標識認識部は、前記特徴検出部によって検出された案内標識の色、文字および記号に関する特徴と前記既知の案内標識の色、文字および記号に関する特徴とを用いて、前記撮像画像中の案内標識と前記既知の案内標識との類似度を算出し、算出した類似度に基づいて、前記既知の案内標識から前記撮像画像中の案内標識を認識すること
    　を特徴とする請求項１記載の案内標識認識装置。     The feature detection unit detects features related to the color, characters and symbols of the guide sign from the area detected by the guide sign detection unit,
    The guide sign recognizing unit uses the features related to the color, characters, and symbols of the guide sign detected by the feature detecting unit and the features related to the color, characters, and symbols of the known guide sign, to guide in the captured image. The guide according to claim 1, wherein a similarity between a sign and the known guide sign is calculated, and a guide sign in the captured image is recognized from the known guide sign based on the calculated similarity. Sign recognition device.
  3. 　前記特徴検出部は、前記撮像画像中の案内標識の特徴と前記既知の案内標識の特徴とを比較して前記撮像画像中の案内標識における前記既知の案内標識の特徴の有無を判定し、前記既知の案内標識の特徴の有無を判定した結果を用いて、前記撮像画像中の案内標識の特徴を示す特徴ベクトルを生成し、
    　前記案内標識認識部は、前記既知の案内標識の特徴を示す特徴ベクトルと前記撮像画像中の案内標識の特徴を示す特徴ベクトルとを用いて、前記撮像画像中の案内標識と前記既知の案内標識との類似度を算出し、算出した類似度に基づいて、前記既知の案内標識から前記撮像画像中の案内標識を認識すること
    　を特徴とする請求項１記載の案内標識認識装置。     The feature detection unit compares the feature of the guide sign in the captured image with the feature of the known guide sign to determine the presence or absence of the feature of the known guide sign in the guide sign in the captured image, Using the result of determining the presence or absence of a feature of a known guide sign, generating a feature vector indicating the feature of the guide sign in the captured image,
    The guide sign recognizing unit uses the feature vector indicating the feature of the known guide sign and the feature vector indicating the feature of the guide sign in the captured image, and the guide sign in the captured image and the known guide sign. The guide sign recognition apparatus according to claim 1, wherein a guide sign in the captured image is recognized from the known guide sign based on the calculated similarity.
  4. 　案内標識検出部が、撮像画像中の案内標識が存在する領域を検出するステップと、
    　特徴検出部が、前記案内標識検出部によって検出された領域から案内標識の特徴を検出するステップと、
    　案内標識認識部が、前記特徴検出部によって検出された案内標識の特徴と既知の案内標識の特徴とを用いて前記撮像画像中の案内標識と前記既知の案内標識との類似度を算出し、算出した類似度に基づいて、前記既知の案内標識から前記撮像画像中の案内標識を認識するステップとを備えたこと
    　を特徴とする案内標識認識方法。     A guide sign detection unit detecting a region where the guide sign exists in the captured image;
    A feature detecting unit detecting a feature of the guide sign from the region detected by the guide sign detecting unit;
    The guide sign recognition unit calculates the similarity between the guide sign in the captured image and the known guide sign using the feature of the guide sign detected by the feature detection unit and the feature of the known guide sign, And a step of recognizing a guide sign in the captured image from the known guide sign based on the calculated similarity.
PCT/JP2018/014795 2018-04-06 2018-04-06 Guide sign identification device and guide sign identification method WO2019193762A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/JP2018/014795 WO2019193762A1 (en) 2018-04-06 2018-04-06 Guide sign identification device and guide sign identification method
JP2020511583A JP6762451B2 (en) 2018-04-06 2018-04-06 Guide sign recognition device and guide sign recognition method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2018/014795 WO2019193762A1 (en) 2018-04-06 2018-04-06 Guide sign identification device and guide sign identification method

Publications (1)

Publication Number Publication Date
WO2019193762A1 true WO2019193762A1 (en) 2019-10-10

Family

ID=68100582

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/014795 WO2019193762A1 (en) 2018-04-06 2018-04-06 Guide sign identification device and guide sign identification method

Country Status (2)

Country Link
JP (1) JP6762451B2 (en)
WO (1) WO2019193762A1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013054522A (en) * 2011-09-02 2013-03-21 Pasuko:Kk Road appurtenances detecting device, road appurtenances detecting method and program
JP2015158739A (en) * 2014-02-21 2015-09-03 日本電信電話株式会社 Image sorting device, image classification method, and image classification program

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4380838B2 (en) * 1999-04-08 2009-12-09 アジア航測株式会社 Video image automatic road sign recognition method, road sign automatic recognition device, and road sign automatic recognition program
JP6591257B2 (en) * 2015-10-21 2019-10-16 株式会社パスコ Image processing apparatus, image processing method, and program

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013054522A (en) * 2011-09-02 2013-03-21 Pasuko:Kk Road appurtenances detecting device, road appurtenances detecting method and program
JP2015158739A (en) * 2014-02-21 2015-09-03 日本電信電話株式会社 Image sorting device, image classification method, and image classification program

Also Published As

Publication number Publication date
JP6762451B2 (en) 2020-09-30
JPWO2019193762A1 (en) 2020-09-17

Similar Documents

Publication Publication Date Title
US10769487B2 (en) Method and device for extracting information from pie chart
JP7051267B2 (en) Image detection methods, equipment, electronic equipment, storage media, and programs
CN109086668B (en) Unmanned aerial vehicle remote sensing image road information extraction method based on multi-scale generation countermeasure network
CN111489403A (en) Method and device for generating virtual feature map by utilizing GAN
WO2017079522A1 (en) Subcategory-aware convolutional neural networks for object detection
CN109582880A (en) Interest point information processing method, device, terminal and storage medium
US20180349716A1 (en) Apparatus and method for recognizing traffic signs
CN112997190A (en) License plate recognition method and device and electronic equipment
CN113420745B (en) Image-based target identification method, system, storage medium and terminal equipment
CN112200193B (en) Distributed license plate recognition method, system and device based on multi-attribute fusion
CN111382625A (en) Road sign identification method and device and electronic equipment
CN112232368B (en) Target recognition model training method, target recognition method and related devices thereof
WO2023138538A1 (en) Vehicle-mounted video image stabilization method and apparatus, vehicle and storage medium
CN116107591A (en) Deployment model construction method based on corn cases
CN112052907A (en) Target detection method and device based on image edge information and storage medium
CN108992033B (en) Grading device, equipment and storage medium for vision test
CN111523351A (en) Neural network training method and device and electronic equipment
CN114120259A (en) Empty parking space identification method and system, computer equipment and storage medium
CN114267076B (en) Image identification method, device, equipment and storage medium
WO2019193762A1 (en) Guide sign identification device and guide sign identification method
CN116580230A (en) Target detection method and training method of classification model
JP2011081614A (en) Recognition system, recognition method, and program
JP2023069083A (en) Learning apparatus, learning method, learning program, object detection apparatus, object detection method, object detection method, learning support system, learning support method, and learning support program
CN109087351B (en) Method and device for carrying out closed-loop detection on scene picture based on depth information
KR20120062168A (en) Apparatus and method for recogniting sub-trajectory

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18913981

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2020511583

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18913981

Country of ref document: EP

Kind code of ref document: A1