TWI828495B - Traffic road intelligent detection method and cloud road surface identification module - Google Patents

Traffic road intelligent detection method and cloud road surface identification module Download PDF

Info

Publication number
TWI828495B
TWI828495B TW111150393A TW111150393A TWI828495B TW I828495 B TWI828495 B TW I828495B TW 111150393 A TW111150393 A TW 111150393A TW 111150393 A TW111150393 A TW 111150393A TW I828495 B TWI828495 B TW I828495B
Authority
TW
Taiwan
Prior art keywords
road surface
cloud
data
road
recognition module
Prior art date
Application number
TW111150393A
Other languages
Chinese (zh)
Inventor
顏嘉俊
Original Assignee
鴻銘資訊有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 鴻銘資訊有限公司 filed Critical 鴻銘資訊有限公司
Priority to TW111150393A priority Critical patent/TWI828495B/en
Application granted granted Critical
Publication of TWI828495B publication Critical patent/TWI828495B/en

Links

Landscapes

  • Traffic Control Systems (AREA)

Abstract

本發明係涉及一種交通路面智慧檢測方法及雲端路面辨識模組,其主要以邊緣端行動裝置來錄影分析框選道路的PCI鋪面特徵的MP4及其所對應F-IMU值、B-IMU值及GPS之數據成數個後設資料(metadata),再將該數個後設資料分別上傳至一雲端伺服器後,再由該雲端伺服器所建置的一雲端路面辨識模組依據各個後設資料,分別辨識出一PCI鋪面狀況指標的特徵值並儲存之,最後再將辨識結果資訊會依據GPS位置資訊而顯示於一地理資訊系統上,以利於交通道路管理人員及養護人員查詢。 The invention relates to a traffic road surface intelligent detection method and a cloud road surface identification module, which mainly uses an edge-end mobile device to record and analyze the MP4 of the PCI pavement characteristics of the selected road and its corresponding F-IMU value, B-IMU value and The GPS data is converted into several metadata, and then the several metadata are uploaded to a cloud server respectively, and then a cloud road surface recognition module built by the cloud server is based on each metadata. , respectively identify the characteristic value of a PCI pavement condition indicator and store it. Finally, the identification result information will be displayed on a geographical information system based on the GPS location information to facilitate the inquiry of traffic road management personnel and maintenance personnel.

Description

交通路面智慧檢測方法及雲端路面辨識模組 Traffic road intelligent detection method and cloud road surface identification module

本發明關於交通道路管理技術領域,尤指一種交通道路的路面智慧檢測方法及雲端路面辨識模組之範疇。 The present invention relates to the technical field of traffic road management, and in particular, to the category of a road intelligent detection method and a cloud road surface recognition module for traffic roads.

按,「路平、燈亮、水溝通」一直是各縣市政府皆致力達成的公共需求,其中,路平又是訴求之首。但現行一般路平養護是採人工目視定期巡檢、民眾申訴等二種方式來進行,而現有的路檢設備不是「非常貴」就是「非常複雜」,如本國新型專利第TWM386314U號「道路檢測車之鋪面平坦度量測單元」。 According to reports, "flat roads, bright lights, and water connections" have always been public needs that all counties and municipal governments are striving to achieve, and among them, smooth roads are at the top of the list. However, the current general road level maintenance is carried out by two methods: manual visual inspections and public complaints. The existing road inspection equipment is either "very expensive" or "very complicated". For example, the country's new patent No. TWM386314U "Road Inspection Vehicle pavement flatness measurement unit".

路面狀況代表著許多人或車的用路安全,等到民眾申訴,往往已經非常嚴重或有交通意外發生。因此,本發明人希望能找一個方式,讓道路養護主管單位能精確找出需整修的路段,讓人民對路平維護更有感。 Road condition represents the safety of many people or vehicles. By the time people complain, it is often very serious or traffic accidents have occurred. Therefore, the inventor hopes to find a way to allow the road maintenance authority to accurately identify the road sections that need to be repaired, so that the people can be more aware of road maintenance.

緣此,為了能夠配合智慧生活、智慧城市的開發,於是本發明人乃窮極心思開發出一種交通路面智慧檢測方法及雲端路面辨識模組,故本發明之主要目的在於:提供低成本建置的一種交通路面智慧檢測方法及雲端路面辨識模組;而本發明之次要目的:在於提供提升交通道路鋪面辨識精準度的一種交通路面智慧檢測方法及雲端路面辨識模組。 Therefore, in order to cooperate with the development of smart life and smart cities, the inventor has put great effort into developing a smart traffic road detection method and a cloud road surface recognition module. Therefore, the main purpose of the present invention is to provide a low-cost construction A traffic road surface intelligent detection method and a cloud road surface identification module; and the secondary purpose of the present invention is to provide an intelligent traffic road surface detection method and a cloud road surface identification module that improve the accuracy of traffic road pavement identification.

為達到上述目的,本發明運用的技術手段如下:一種雲端路面辨識模組,其在一雲端伺服器內建置一雲端路面辨識模組,且該雲端路面辨識模 組會依據各已框選的PCI鋪面特徵的影音串流及其所對應的慣性姿態之數據來分別辨識出不良路面的PCI鋪面狀況指標的特徵值並儲存;其中,前述該慣性姿態之數據係指位於邊緣端的一移動載具的前端、後端所設一前慣性測量單元及一後慣性測量單元所量測到的三軸姿態角、三軸加速度及三軸地球磁場的絕對指向之前慣性姿態、後慣性姿態的數據資料。 In order to achieve the above object, the technical means used by the present invention are as follows: a cloud road surface recognition module, which builds a cloud road road recognition module in a cloud server, and the cloud road road recognition module The group will identify and store the characteristic values of the PCI pavement condition indicators of bad road surfaces based on the video streams of the selected PCI pavement characteristics and the corresponding inertial posture data. Among them, the aforementioned inertial posture data is Refers to the previous inertial attitude of the three-axis attitude angle, the three-axis acceleration and the absolute pointing of the three-axis earth's magnetic field measured by a front inertial measurement unit and a rear inertial measurement unit installed at the front and rear ends of a mobile vehicle at the edge. , data of rear inertial attitude.

關於本發明一種交通路面智慧檢測方法,係包含有:一邊緣端資料蒐集步驟,係藉由至少一邊緣端行動裝置蒐集交通各道路不良路面的數個串流影音及所對應的一前慣性姿態、一後慣性姿態、一GPS資訊之數據資訊;一邊緣端不良路面偵測框選步驟,係針對該數個串流影音分別進行初步辨識以框選出一PCI鋪面特徵;一數據資訊上傳雲端步驟,係整合道路不良路面的數個串流影音及其所對應的前慣性姿態、後慣性姿態、GPS資訊成數個後設資料上傳至雲端伺服器作儲存;一雲端不良路面辨識步驟,係在該雲端伺服器更包含有一雲端路面辨識模組,且該雲端路面辨識模組會依據不良路面的該數個串流影音及其所對應的前慣性姿態、後慣性姿態之數據,分別辨識出該數個串流影音的鋪面狀況指標的特徵值並作儲存,其中該前慣性姿態、該後慣性姿態之數據係指位於邊緣端的一移動載具的前端、後端所設一前慣性測量單元及一後慣性測量單元所量測到的三軸姿態角、三軸加速度及三軸地球磁場的絕對指向之前後慣性姿態的數據資料;及一路面辨識結果輸出步驟,係依據前步驟的辨識結果資訊會依據GPS的位置資訊顯示於一地理資訊系統上。 Regarding a traffic road surface intelligent detection method of the present invention, it includes: an edge end data collection step, which collects several streaming videos and a corresponding forward inertial posture of bad road surfaces of each traffic road through at least one edge end mobile device. , the data information of a rear inertial attitude and a GPS information; a step of edge-end bad road detection and frame selection, which is to conduct preliminary identification of the several streaming videos to frame a PCI pavement feature; a step of uploading data information to the cloud , the system integrates several streaming videos of bad road surfaces and their corresponding front inertial attitude, rear inertial attitude, and GPS information into several metadata and uploads them to the cloud server for storage; a cloud bad road surface identification step is performed in the The cloud server also includes a cloud road surface recognition module, and the cloud road road recognition module will identify the data based on the several streaming videos on the bad road surface and their corresponding front inertial posture and rear inertial posture data. The characteristic value of the store condition indicator of the streaming video is stored, in which the data of the front inertial attitude and the rear inertial attitude refer to a front inertial measurement unit and a rear end of a mobile vehicle located at the edge. The data of the three-axis attitude angle, the three-axis acceleration and the absolute pointing of the three-axis earth's magnetic field measured by the rear inertial measurement unit are the front and rear inertial attitude data; and a road surface identification result output step is based on the identification result information of the previous step. Location information based on GPS is displayed on a geographic information system.

上述雲端路面辨識模組,係使用由Tensorflow開發軟體所開發的該雲端路面辨識模組並以scikit-learn作為機器學習的函式庫來進行資料前處理 與建立模型,且該雲端路面辨識模組會依據交通道路之不良路面的後設資料,分別辨識出不良路面的該數個串流影音的鋪面狀況指標的特徵值並作儲存。 The above-mentioned cloud road surface recognition module uses the cloud road road recognition module developed by Tensorflow development software and uses scikit-learn as the machine learning function library for data pre-processing. and establish a model, and the cloud road surface recognition module will separately identify and store the characteristic values of the pavement condition indicators of the several streaming videos based on the metadata of the bad road surface of the traffic road.

其中該雲端路面辨識模組係採用監督式深度學習模型框架,並使用以YOLO演算法所識別數個不良路面串流影音的鋪面狀況指標的特徵值及其所對應的前慣性姿態、後慣性姿態等數據來作為監督式學習的一訓練樣本之標記檔,進而讓該鋪面狀況指標的特徵值與前慣性姿態數據、後慣性姿態數據產生對應關聯,以增進不良路面的辨識效果。 The cloud road surface recognition module adopts a supervised deep learning model framework and uses the YOLO algorithm to identify the characteristic values of pavement condition indicators of several bad road streaming videos and their corresponding front inertia posture and rear inertia posture. The data is used as a mark file of a training sample for supervised learning, and then the characteristic value of the pavement condition indicator is correspondingly associated with the front inertial attitude data and the rear inertial attitude data to improve the identification effect of bad road surfaces.

其中該雲端路面辨識模組的訓練及測試係依序經過一資料前處理步驟、一建構機器學習模型並訓練步驟、一評估模型步驟及一預測步驟,而在該資料前處理步驟中將訓練樣本區分為一75%訓練集及一25%測試集。 The training and testing of the cloud road surface recognition module sequentially go through a data pre-processing step, a machine learning model construction and training step, an evaluation model step and a prediction step, and in the data pre-processing step, the training samples It is divided into a 75% training set and a 25% test set.

其中該訓練樣本的標記檔更包含有GPS的速度數據。 The mark file of the training sample also contains GPS speed data.

其中該鋪面狀況指標的特徵值依據ASTM D6433-11所定義之破壞項目合計共有19項破壞,分別為鱷魚狀裂縫、冒油、塊狀裂縫、凸與凹陷、波浪型路面、凹陷、邊緣裂縫、反射裂縫、路肩高差、縱與橫裂縫、補錠、粒料光滑、坑洞、跨越鐵道、車轍、推擠、滑動裂縫、隆起及風化與剝脫。 Among them, the characteristic value of the pavement condition index is based on the damage items defined by ASTM D6433-11. There are a total of 19 damages, including alligator cracks, oil leakage, block cracks, convexities and depressions, wavy pavement, depressions, edge cracks, Reflective cracks, shoulder height differences, longitudinal and transverse cracks, patches, granular smoothing, potholes, railroad crossings, ruts, jostling, sliding cracks, bulges, and weathering and spalling.

據此,本發明藉由上述技術手段,可以達到如下功效: Accordingly, the present invention can achieve the following effects through the above technical means:

1.本發明僅使用包含有一移動載具、一雲端伺服器、一邊緣端行動裝置、一AI影像雲端路面辨識模組等,且採用邊緣運算架構,分擔雲端伺服器的運算負擔,故使用本發明所建置的成本低,又可以協助提供路面驗收及快速判斷路面情況。 1. The present invention only uses a mobile vehicle, a cloud server, an edge mobile device, an AI image cloud road recognition module, etc., and adopts an edge computing architecture to share the computing burden of the cloud server. Therefore, this invention is used The invention has low construction cost and can assist in providing road surface inspection and quick judgment of road conditions.

2.本發明的雲端路面辨識模組係採用監督式的深度學習模型框架,且以前慣性姿態、後慣性姿態及PCI鋪面狀況指標的特徵值以作為不 良路面影像辨識之訓練樣本的標記檔,所訓練出來的雲端路面辨識模組具有高精準的辨識效果。 2. The cloud road surface recognition module of the present invention adopts a supervised deep learning model framework, and the characteristic values of the previous inertial attitude, the rear inertial attitude and the PCI pavement condition indicator are used as different Using the labeled files of training samples for good road surface image recognition, the cloud road surface recognition module trained has highly accurate recognition results.

A:交通路面智慧檢測方法 A: Traffic road intelligent detection method

a、b、c、d、e:步驟 a, b, c, d, e: steps

B:交通路面智慧檢測系統 B: Traffic road intelligent detection system

C:移動載具 C:Mobile vehicle

C1:手機架 C1: mobile phone holder

1:邊緣端行動裝置 1: Edge mobile devices

11:鏡頭單元 11: Lens unit

12:顯示單元 12:Display unit

13:GPS定位單元 13:GPS positioning unit

14:前慣性測量單元 14: Front inertial measurement unit

15:記憶體單元 15:Memory unit

16:通信單元 16: Communication unit

17:運算處理器 17:Arithmetic processor

2:慣性測量裝置 2: Inertial measurement device

21:後慣性測量單元 21:Rear inertial measurement unit

22:傳輸單元 22:Transmission unit

3:整合應用程式單元 3: Integrate application units

4:雲端伺服器 4:Cloud server

41:雲端路面辨識模組 41: Cloud road surface recognition module

5:地理資訊系統 5:Geographic information system

〔圖1〕本發明「交通路面智慧檢測方法」之步驟流程圖。 [Figure 1] The step flow chart of the "intelligent traffic road detection method" of the present invention.

〔圖2〕本發明「交通路面智慧檢測系統」架構之示意圖。 [Figure 2] Schematic diagram of the architecture of the "intelligent traffic road detection system" of the present invention.

〔圖3〕本發明「移動載具」選用機車行駛於路面之錄影角度之示意圖。 [Figure 3] A schematic diagram of the video recording angle of a motorcycle traveling on the road for the "mobile vehicle" of the present invention.

〔圖4〕本發明「移動載具」選用汽車行駛於路面之錄影角度之示意圖。 [Figure 4] A schematic diagram of the video recording angle of a car driving on the road for the "mobile vehicle" of the present invention.

〔圖5〕本發明「邊緣端行動裝置」框選不良路鋪面特徵並記錄其對應的GPS、F-IMU值、B-IMU值、MP4數據之作動示意圖。 [Figure 5] Schematic diagram of the operation of the "edge mobile device" of the present invention to select bad road pavement features and record their corresponding GPS, F-IMU values, B-IMU values, and MP4 data.

〔圖6〕本發明「交通路面智慧檢測系統」之運作流程圖。 [Figure 6] The operation flow chart of the "intelligent traffic road detection system" of the present invention.

〔圖7〕本發明「雲端伺服器的影像辨識模組」演算及其建置之流程圖。 [Figure 7] The flow chart of the calculation and construction of the "image recognition module of the cloud server" of the present invention.

〔圖8〕本發明「邊緣端行動裝置」架構之示意圖。 [Figure 8] Schematic diagram of the architecture of the "edge mobile device" of the present invention.

〔圖9〕本發明「慣性測量裝置」架構之示意圖。 [Figure 9] Schematic diagram of the structure of the "inertial measurement device" of the present invention.

〔圖10〕本發明關於不良路面之鋪面示意圖。 [Fig. 10] A schematic diagram of the pavement of poor road surface according to the present invention.

〔圖11〕本發明關於「前慣性測量數據」及「後慣性測量數據」之示意圖。 [Fig. 11] A schematic diagram of the "front inertia measurement data" and "rear inertia measurement data" of the present invention.

本發明係關於一種交通路面智慧檢測方法及雲端路面辨識模組,其主要提供低成本的建置系統,以精準地辨識交通道路不良路面的狀況,來協助路面驗收、養護人快速判斷路面情況,而有利於通知養護單位來進行道路維護養護;其中該交通路面智慧檢測方法A,如圖1所示,其包含有:一邊緣端資料蒐集步驟a、一邊緣端不良路面偵測框選步驟b、一數據資訊上傳雲端步 驟c、一雲端不良路面辨識步驟d;茲將上述各步驟配合圖1及圖2,分別說明如後。 The present invention relates to an intelligent detection method of traffic road surface and a cloud road surface identification module. It mainly provides a low-cost construction system to accurately identify the condition of bad road surface on traffic roads to assist road acceptance and maintenance personnel to quickly judge the road surface conditions. It is helpful to notify the maintenance unit to carry out road maintenance; the traffic road intelligent detection method A, as shown in Figure 1, includes: an edge end data collection step a, an edge end bad road surface detection frame selection step b , One step of uploading data information to the cloud Step c. Step d of identification of bad road surface in the cloud. Each of the above steps is described below with reference to Figures 1 and 2.

所述該邊緣端資料蒐集步驟a,如圖3及圖4所示,係藉由至少一個或數個邊緣端行動裝置蒐集交通各道路不良路面的數個串流影音(可以是AVI、WMV、MOV、MP4、RM等格式,以下統稱MP4)及所對應的一前慣性姿態、一後慣性姿態、一GPS資訊之數據資訊;其中各道路不良路面的數個串流影音的較佳錄影長度大約為150公分長的不良路面。 The edge end data collection step a, as shown in Figure 3 and Figure 4, is to collect several streaming videos (which can be AVI, WMV, MOV, MP4, RM and other formats (hereinafter collectively referred to as MP4) and the corresponding data information of one front inertial attitude, one rear inertial attitude, and one GPS information; among them, the optimal recording length of several streaming videos on bad road surfaces is approximately It is a 150 cm long patch of bad road surface.

即本步驟a主要由一移動載具C的前端、後端分別安裝有一邊緣端行動裝置1、一慣性測量裝置2後,其中移動載具C較佳則選用機車,次佳則為汽車,而邊緣端行動裝置1較佳則選用智慧型手機(或使用平板電腦,與鏡頭分離之裝置,例如:以一筆電配合一攝影鏡頭等裝置),其中所謂慣性姿態(inertial attitude)之數據係由慣性測量單元(Inertial Measurement Unit,簡稱IMU)所感測到一物體的三軸姿態角、三軸加速度及三軸地球磁場的絕對指向之數據;故前慣性姿態係指該移動載具C的前端的慣性姿態,本發明將其稱F-IMU值;而後慣性姿態係指該移動載具C的後端的慣性姿態,本發明將其稱B-IMU值。 That is, this step a mainly consists of installing an edge-end mobile device 1 and an inertial measurement device 2 on the front and rear ends of a mobile vehicle C. The best choice for mobile vehicle C is a motorcycle, the second best is a car, and the edge-end The mobile device 1 is preferably a smart phone (or a tablet computer, a device that is separated from the lens, such as a laptop with a photographic lens), in which the so-called inertial attitude data is obtained by an inertial measurement unit ( Data on the three-axis attitude angle, three-axis acceleration and the absolute direction of the three-axis earth's magnetic field sensed by the Inertial Measurement Unit (IMU); therefore, the front inertial attitude refers to the inertial attitude of the front end of the mobile vehicle C. The invention calls this the F-IMU value; the rear inertial attitude refers to the inertial attitude of the rear end of the mobile vehicle C, and the invention calls it the B-IMU value.

當移動載具C行駛於道路上以進行不良路面攝影並記錄當下的F-IMU值、B-IMU值、GPS之數據,其中該F-IMU值、該GPS之數據係由該邊緣端行動裝置1負責感測蒐集,而該B-IMU值數據係由該慣性測量裝置2負責感測蒐集,蒐集後的該B-IMU值數據經由無線或有線傳輸傳送到該邊緣端行動裝置1。 When the mobile vehicle C is driving on the road to take pictures of bad road surfaces and record the current F-IMU value, B-IMU value, and GPS data, the F-IMU value and the GPS data are obtained from the edge mobile device. 1 is responsible for sensing and collecting, and the inertial measurement device 2 is responsible for sensing and collecting the B-IMU value data. The collected B-IMU value data is transmitted to the edge mobile device 1 through wireless or wired transmission.

所述該邊緣端不良路面偵測框選步驟b,如圖5所示,係針對交通道路的該數個串流影音(MP4)分別以Tensorflow開發軟體的物件偵測(Object detector)進行初步辨識並框選(Bounding Box)出一PCI鋪面特徵並同時分別記錄其GPS、F-IMU值、B-IMU值之數據來結合成數個後設資料(metadata);特別一提,該物件偵測(Object detector)係為Tensorflow開發軟體的一個物件辨識功能。 The edge edge bad road surface detection frame selection step b, as shown in Figure 5, is based on the object detection (Object detection) of the Tensorflow development software for the several streaming videos (MP4) on the traffic road. detector) to conduct preliminary identification and Bounding Box to identify a PCI pavement feature and simultaneously record its GPS, F-IMU value, and B-IMU value data respectively to combine them into several metadata; in particular, The object detector is an object recognition function of the Tensorflow development software.

所述數據資訊上傳雲端步驟c,如圖6所示,係整合前步驟針對交通道路的不良路面所蒐集到的數個後設資料分別上傳至雲端伺服器4作儲存。 The data information is uploaded to the cloud in step c, as shown in Figure 6. The system integrates the previous steps and uploads several metadata collected for the bad road surface of the traffic road to the cloud server 4 for storage.

所述該雲端不良路面辨識步驟d,如圖7所示,係在該雲端伺服器4更包含有由Tensorflow開發軟體所開發的一雲端路面辨識模組41並以scikit-learn作為機器學習的函式庫來進行資料前處理與建立模型,且該雲端路面辨識模組41會依據交通道路之不良路面的數個後設資料,分別辨識出不良路面的該數個串流影音的鋪面狀況指標(Pavement Condition Index,簡稱PCI)的特徵值並作儲存,而不良路面的PCI指標的特徵值係依據ASTM D6433-11所定義之破壞項目合計共有四大種類及19項破壞,分別為NO.01鱷魚狀裂縫(Alligator Craking)、NO.02冒油(Bleeding)、NO.03塊狀裂縫(Block Cracking)、NO.04凸與凹陷(Bumps and Sags)、NO.05波浪型路面(Corrugation)、NO.06凹陷(Depression)、NO.07邊緣裂縫(Edge Cracking)、NO.08反射裂縫(Jt.Reflection Cracking)、NO.09路肩高差(Lane/Shoulder Drop Off)、NO.10縱與橫裂縫(Long & Trans Cracking)、NO.11補錠(Patching & Util Cut Patching)、NO.12粒料光滑(Polished Aggregate)、NO.13坑洞(Potholes)、NO.14跨越鐵道(Railroad crossing)、NO.15車轍(Rutting)、NO.16推擠(Shoving)、NO.17滑動裂縫(Slippage Cracking)、NO.18隆起(Swell)、NO.19風化與剝脫(Weathering/Raveling),如底下表1所示者。 The cloud bad road surface identification step d, as shown in Figure 7, is that the cloud server 4 further includes a cloud road surface identification module 41 developed by Tensorflow development software and uses scikit-learn as a machine learning function. A library is used to perform data pre-processing and build models, and the cloud road surface identification module 41 will respectively identify the pavement condition indicators of the several streaming videos of the bad road surface based on several meta-data of the bad road surface of the traffic road ( The characteristic values of the Pavement Condition Index (PCI) are stored. The characteristic values of the PCI index of bad pavement are based on the damage items defined by ASTM D6433-11. There are four major categories and 19 damage items in total, which are NO.01 crocodile. Alligator Craking, NO.02 Bleeding, NO.03 Block Cracking, NO.04 Bumps and Sags, NO.05 Corrugation, NO. .06 Depression, NO.07 Edge Cracking, NO.08 Reflection Cracking, NO.09 Lane/Shoulder Drop Off, NO.10 Longitudinal and transverse cracks (Long & Trans Cracking), NO.11 Patching & Util Cut Patching, NO.12 Polished Aggregate, NO.13 Potholes (Potholes), NO.14 Railroad crossing, NO.15 Rutting, NO.16 Shoving, NO.17 Slippage Cracking, NO.18 Swell, NO.19 Weathering/Raveling, as shown below Those shown in Table 1.

Figure 111150393-A0305-02-0007-1
Figure 111150393-A0305-02-0007-1
Figure 111150393-A0305-02-0008-2
Figure 111150393-A0305-02-0008-2

進一步,在本步驟d中,該雲端路面辨識模組41係採用監督式深度學習模型(deep learning modle)框架,並使用以YOLO演算法所識別數個不良路面的串流影音的鋪面狀況指標的特徵值及其所對應的F-IMU值、B-IMU值等數據來作為監督式學習(supervised learning)的訓練樣本(training sample)之標記檔(label),進而讓該雲端路面辨識模組41可以辨識該鋪面狀況指標的特徵值與F-IMU值、B-IMU值等之數據產生對應關聯,以增進交通道路不良路面的辨識效果;進一步,該訓練樣本係由75%訓練集(training data)及25%測試集(testing set)所組成,而該訓練樣本的標記檔可以額外再加入GPS的速度數據之參數,進而提升辨識的精準度。 Further, in this step d, the cloud road surface recognition module 41 adopts a supervised deep learning model framework and uses the pavement condition indicators of streaming videos of several bad road surfaces identified by the YOLO algorithm. The feature value and its corresponding F-IMU value, B-IMU value and other data are used as the label of the training sample for supervised learning, thereby allowing the cloud road surface recognition module 41 The characteristic values of the pavement condition index can be identified and correlated with the data of F-IMU value, B-IMU value, etc. to improve the identification effect of bad road pavement on traffic roads; further, the training sample is composed of 75% training data (training data ) and 25% of the test set, and the marker file of the training sample can be additionally added with parameters of GPS speed data to improve the accuracy of identification.

特別一提,路面辨識模型在訓練集上進行調適。對於監督式學習,訓練集是由用來調適參數(例如:人工神經網路中神經元之間連結的權重)的範例組成的集合。在實施當中,訓練集通常是由輸入向量(純量)和輸出向量(純量)組成的資料對。其中輸出向量(純量)被稱為目標或標籤。在訓練過程中,當前路面辨識模型會對訓練集中的每個範例進行預測,並將預測結果與目標進行比較。根據比較的結果,學習演算法會更新路面辨識模型的參數。模型調適的過程可能同時包括特徵選擇和參數估計。 In particular, the road surface recognition model is adapted on the training set. For supervised learning, a training set is a collection of examples used to tune parameters (such as the weights of connections between neurons in an artificial neural network). In implementation, the training set is usually a pair of input vectors (scalars) and output vectors (scalars). where the output vector (scalar) is called the target or label. During training, the current road surface recognition model makes predictions for each example in the training set and compares the predictions with the target. Based on the comparison results, the learning algorithm updates the parameters of the road surface recognition model. The process of model tuning may include both feature selection and parameter estimation.

而測試集則被用來提供對最終的路面辨識模模型之無偏評估。 The test set is used to provide an unbiased evaluation of the final road surface identification model.

進一步,該雲端路面辨識模組41的訓練步驟,如圖7所示,如下述: Further, the training steps of the cloud road surface recognition module 41, as shown in Figure 7, are as follows:

步驟1:資料前處理 Step 1: Data pre-processing

1.以scikit-learn的train test split( )函式將樣本資料隨機排序(shuffle)後分成兩部分:75%樣本為訓練集,25%樣本為測式集(可自行設定比例,但以經驗來說,這樣的比例是不錯的);2.stratify=y:依據原目標的分類比例來分層;3.random_state=0:設定整數固定種子值(其他整數亦可),讓每次實驗跑的資料都相同;4.train_test_split( )函式回覆4個NumPy陣列;5.#X_train,X_test訓練與測試資料,都是二維陣列,每一列是一筆樣本資料;6.#y_train,y_test訓練與測試答案,都是一維陣列,每一個元素是一個類別資料; 1. Use scikit-learn's train test split() function to randomly sort (shuffle) the sample data and divide it into two parts: 75% of the samples are the training set, and 25% of the samples are the test set (the proportion can be set by yourself, but based on experience Generally speaking, this ratio is good); 2. stratify=y: Stratify according to the classification ratio of the original target; 3. random_state=0: Set an integer fixed seed value (other integers are also acceptable), so that each experiment can be run The data are the same; 4. The train_test_split() function returns 4 NumPy arrays; 5. #X_train, The test answers are all one-dimensional arrays, and each element is a category data;

步驟2:建構機器學習模型並訓練 Step 2: Construct a machine learning model and train it

建立一個機器學習模型,給定訓練資料與答案,利用fit( )擬合方法來訓練模型; Establish a machine learning model, given training data and answers, and use the fit() fitting method to train the model;

步驟3:評估模型(Evaluating the model) Step 3: Evaluating the model

利用score( )方法計算訓練效能:給定測試資料與答案,計算預測正確的比例;及 Use the score() method to calculate training performance: given test data and answers, calculate the proportion of correct predictions; and

步驟4:預測(prediction) Step 4: prediction

訓練完畢且效能符合標準即可進行預測:給定新資料,利用predict( )方法來預測結果。 After training is completed and the performance meets the standard, prediction can be made: given new data, use the predict() method to predict the results.

因此,在本步驟d中,該雲端路面辨識模組41會藉由訓練樣本經過一連串的訓練、測試及調較,進而取得一個最佳的辨識模型;故該雲端路面辨識模組41實際的辨識過程中,會將該邊緣端行動裝置1所傳來的MP4檔預先轉換成數個影像後,再分別經降噪後才由該雲端路面辨識模組41進行辨識。 Therefore, in this step d, the cloud road surface recognition module 41 will obtain an optimal recognition model through a series of training, testing and adjustment using training samples; therefore, the cloud road road surface recognition module 41 actually recognizes During the process, the MP4 files transmitted from the edge mobile device 1 will be converted into several images in advance, and then the images will be recognized by the cloud road surface recognition module 41 after noise reduction.

所述該路面辨識結果輸出步驟e,係依據前步驟的辨識結果資訊會依據GPS的位置資訊顯示於一地理資訊系統5上。 The road surface recognition result output step e is based on the recognition result information of the previous step being displayed on a geographical information system 5 based on the GPS location information.

請參閱圖2、圖3及圖8所示,關於一種交通路面智慧檢測系統B,係由前述本發明交通路面智慧檢測方法A所建置執行並實施,進而達到本發明之目的,其包含有:一邊緣端行動裝置、一慣性測量裝置2、一整合應用程式單元3及一雲端伺服器4等裝置;茲配合圖式針對上述各裝置分別說明如後。 Please refer to Figure 2, Figure 3 and Figure 8. Regarding a traffic road intelligent detection system B, which is constructed and implemented by the aforementioned traffic road surface intelligent detection method A of the present invention, thereby achieving the purpose of the present invention, it includes: : An edge mobile device, an inertial measurement device 2, an integrated application unit 3 and a cloud server 4 and other devices; the above-mentioned devices are described separately with reference to the drawings as follows.

所述該邊緣端行動裝置1,如圖8所示,係提供可執行邊緣端運算(edge computing)之裝置,係裝設於一移動載具C的前端處,例如:機車的握把處,且該邊緣端行動裝置1內建更包含有一鏡頭單元11、一顯示單元12、一GPS定位單元13、一前慣性測量單元14、一記憶體單元15及一通信單元16分別電性連接一運算處理器17;其中該鏡頭單元11會對道路的不良路面進行錄影,而該GPS定位單元13則感測該移動載具C的GPS資訊(亦可稱不良路面的GPS資訊);該前慣性測量單元14則感測該移動載具C前端處的F-IMU值,而該記憶體單元15則提供儲存該行動裝置1的整合應用程式單元3,以及MP4、GPS、F-IMU值、B-IMU值等之數個後設資料(metadata);該通信單元16則藉由行動網路、網際網路與該雲端 伺服器4耦接後,可上傳該數個後設資料,而該運算處理器17執行該整合應用程式單元3以提供該不良路面之錄影MP4並框選,且同時取得感測後的與該MP4對應的GPS、F-IMU值、B-IMU值,以完成數次後設資料之收集;進一步,該移動載具C較佳選用如圖3的機車,次佳為如圖4的汽車,且該機車前端的握把處附設有具自動或手動調整角度功能的一手機架C1,該手機架C1離地面高度為0.5~1.6公尺之間,且該移動載具的速度設為低於120km/hr以下,以取得清晰的影像;若使用汽車,則該智慧型手機安裝利地面高度為0.5~1.6公尺之間的保險桿、測後照鏡或擋風玻璃處;而該邊緣端行動裝置1係設為智慧型手機(或使用平板電腦、與鏡頭分離之裝置,例如:以一筆電配合一攝影鏡頭等裝置)並可以將該智慧型手機裝設於該機車的手機架C1上,且該智慧型手機的鏡頭單元11拍攝道路的角度θ設為22~50度之間;特別一提,機車及智慧型手機均為非常簡便的工具,在騎乘道路的過程中,當人員目視道路有不良路面時,則直接騎向該不良路面處便可對其錄影來取得數個後設資料。 The edge mobile device 1, as shown in Figure 8, is a device that can perform edge computing and is installed at the front end of a mobile vehicle C, such as the handlebar of a motorcycle. The edge mobile device 1 further includes a lens unit 11, a display unit 12, a GPS positioning unit 13, a front inertial measurement unit 14, a memory unit 15 and a communication unit 16 that are each electrically connected to a computing unit. Processor 17; wherein the lens unit 11 records the bad road surface of the road, and the GPS positioning unit 13 senses the GPS information of the mobile vehicle C (which can also be called the GPS information of the bad road surface); the front inertial measurement The unit 14 senses the F-IMU value at the front end of the mobile vehicle C, and the memory unit 15 provides the integrated application unit 3 for storing the mobile device 1, as well as MP4, GPS, F-IMU value, B- Several metadata of the IMU value; the communication unit 16 communicates with the cloud through the mobile network, the Internet After the server 4 is coupled, the plurality of metadata can be uploaded, and the computing processor 17 executes the integrated application unit 3 to provide the video MP4 of the bad road surface and frame selection, and at the same time obtain the sensed and the The GPS, F-IMU value, and B-IMU value corresponding to MP4 are used to complete the collection of metadata several times; further, the mobile vehicle C is preferably a motorcycle as shown in Figure 3, and the second best is a car as shown in Figure 4. And the grip of the front end of the motorcycle is equipped with a mobile phone holder C1 with automatic or manual angle adjustment function. The height of the mobile phone holder C1 from the ground is between 0.5 and 1.6 meters, and the speed of the mobile vehicle is set to less than Below 120km/hr to obtain a clear image; if using a car, the smartphone should be installed on the bumper, rear view mirror or windshield with a ground height between 0.5 and 1.6 meters; and the edge end The mobile device 1 is set as a smartphone (or uses a tablet computer, a device separated from the lens, such as a laptop with a camera lens, etc.) and the smartphone can be installed on the mobile phone holder C1 of the motorcycle. , and the angle θ of the lens unit 11 of the smartphone to capture the road is set to between 22 and 50 degrees; in particular, motorcycles and smartphones are very simple tools. During the process of riding the road, when people When there is a bad road surface visually, you can ride directly to the bad road surface and record it to obtain several metadata.

該慣性測量裝置2,如圖3及圖9所示,係裝置於該移動載具C的後端處且更包含有一後慣性測量單元21及一傳輸單元22;其中該後慣性測量單元21感測該移動載具C後端處的B-IMU值,且該傳輸單元22可以是無線或有線,而將後慣性姿態之數據則預先傳輸至該邊緣端行動裝置1作整合,所獲得的後設資料,如底下表2、表3配合圖10所示,則為其中一後設資料(metadata):

Figure 111150393-A0305-02-0011-3
Figure 111150393-A0305-02-0012-4
The inertial measurement device 2, as shown in Figures 3 and 9, is installed at the rear end of the mobile vehicle C and further includes a rear inertial measurement unit 21 and a transmission unit 22; wherein the rear inertial measurement unit 21 senses The B-IMU value at the rear end of the mobile vehicle C is measured, and the transmission unit 22 can be wireless or wired, and the data of the rear inertial attitude is pre-transmitted to the edge mobile device 1 for integration, and the obtained rear Assuming data, as shown in Table 2 and Table 3 below together with Figure 10, is one of the metadata:
Figure 111150393-A0305-02-0011-3
Figure 111150393-A0305-02-0012-4

Figure 111150393-A0305-02-0012-5
Figure 111150393-A0305-02-0012-5

特別一提,該傳輸單元22較佳則選用藍芽(bluetooth)傳輸。 In particular, the transmission unit 22 preferably adopts Bluetooth transmission.

該整合應用程式單元3,如圖8所示,係預存並被安裝至該邊緣端行動裝置1的記憶體單元15,並由該運算處理器17呼叫執行以針對不良路面進行錄影並蒐集該數個後設資料作儲存,並同時令該通信單元16上傳到雲端伺服器4。 The integrated application unit 3, as shown in Figure 8, is pre-stored and installed in the memory unit 15 of the edge mobile device 1, and is called and executed by the computing processor 17 to record bad road surfaces and collect the data. The metadata is stored, and the communication unit 16 is uploaded to the cloud server 4 at the same time.

該雲端伺服器4,係接收該邊緣端行動裝置1所傳來含MP4、GPS、F-IMU值、B-IMU值之數個後設資料,且該雲端伺服器4更包含有一雲端路面辨識模組41,而該雲端路面辨識模組41會依據該數個後設資料分別辨識出不良路面的PCI鋪面狀況指標的特徵值並儲存之,並最終辨識結果會提供藉由該GPS的位置資訊輸出顯示於一地理資訊系統上。 The cloud server 4 receives several metadata including MP4, GPS, F-IMU values, and B-IMU values from the edge mobile device 1, and the cloud server 4 further includes a cloud road surface recognition Module 41, and the cloud road surface recognition module 41 will respectively identify the characteristic values of the PCI pavement condition index of the bad road surface based on the several metadata and store them, and the final recognition result will provide location information through the GPS The output is displayed on a geographic information system.

因此,本發明提供一種交通路面智慧檢測方法及雲端路面辨識模組之較佳解決方案,待把該邊緣端行動裝置1裝好於該移動載具C並啟動後,執行該整合應用程式單元3,便會開啟該邊緣端行動裝置1的辨識功能,且同時校正該鏡頭單元11的拍攝角度θ後,呼叫該邊緣端行動裝置1中GPS、F-IMU、鏡頭單元11及該慣性測量裝置2的B-IMU; 然後,續載入Tensorflow函式庫並向雲端伺服器4呼叫路面標記檔後,開始運用Tensorflow初步辨識PCI鋪面特徵,當辨識到裂縫、坑洞等時,開始記錄GPS、F-IMU、B-IMU及MP4之後設資料;最後啟動背景傳輸,如果有網路則傳輸所記錄的後設資料到雲端伺服器4作進一步作詳細辨識PCI鋪面特徵,即該雲端路面辨識模組41會根據F-IMU、B-IMU數據,如圖11所示,找出特徵波形及各項九軸參數,由於不同的PCI鋪面特徵會產生,最後該雲端路面辨識模組41會依據不同的九軸參數,進而辨識出各PCI鋪面特徵,倘若沒網路則傳輸所記錄的後設資料傳輸到本端並預先作儲存,等有網路時在上傳至雲端。 Therefore, the present invention provides a better solution for a smart traffic road detection method and a cloud road surface recognition module. After the edge mobile device 1 is installed on the mobile vehicle C and started, the integrated application unit 3 is executed. , the recognition function of the edge-end mobile device 1 will be turned on, and after correcting the shooting angle θ of the lens unit 11, the GPS, F-IMU, lens unit 11 and the inertial measurement device 2 in the edge-end mobile device 1 will be called. B-IMU; Then, continue to load the Tensorflow function library and call the road marking file to the cloud server 4, and then use Tensorflow to initially identify the PCI pavement features. When cracks, potholes, etc. are identified, start recording GPS, F-IMU, B- IMU and MP4 post-configuration data; finally start background transmission, if there is a network, the recorded post-configuration data will be transmitted to the cloud server 4 for further detailed identification of PCI pavement characteristics, that is, the cloud road surface recognition module 41 will be based on F- IMU and B-IMU data, as shown in Figure 11, find the characteristic waveform and various nine-axis parameters. Since different PCI pavement characteristics will be generated, the cloud road surface identification module 41 will finally use different nine-axis parameters to determine Identify the characteristics of each PCI interface. If there is no network, the recorded metadata will be transferred to the local end and stored in advance, and then uploaded to the cloud when there is a network.

提供用於前述交通路面智慧檢測方法的一種邊緣收集裝置,則其僅包含有:前述該邊緣端行動裝置1、前述該慣性測量裝置2及前述該整合應用程式單元3等,而該邊緣收集裝置則專門以蒐集不良鋪面的MP4、GPS、F-IMU值、B-IMU值等之數個後設資料,並上傳至雲端伺服器4以進行後續處理。特別一提,如圖13所示,本發明進行資訊蒐集時,由於隨時有GPS、智慧型手機的F-IMU、B-IMU、影像等相關數據資料進入,此為大數據型之數據來源,即本發明可利用大數據技術來進行資料處理,換言之,當從某設備(資料來源),發現有狀況發生(問題特徵),定義為某狀況事件,並通知相關人員進行處理(腳本式回應),據以提升設備問題等之解決效率。 Provide an edge collection device for the aforementioned intelligent detection method of traffic road surface, which only includes: the aforementioned edge-side mobile device 1, the aforementioned inertial measurement device 2, the aforementioned integrated application unit 3, etc., and the edge collection device It is specifically designed to collect several metadata such as MP4, GPS, F-IMU value, B-IMU value, etc. of defective pavements, and upload them to the cloud server 4 for subsequent processing. In particular, as shown in Figure 13, when the present invention collects information, since relevant data such as GPS, F-IMU, B-IMU, and images of smartphones enter at any time, this is a big data source. That is, the present invention can use big data technology for data processing. In other words, when a situation occurs (problem characteristics) from a certain device (data source), it is defined as a certain situation event, and relevant personnel are notified for processing (scripted response) , to improve the efficiency of solving equipment problems.

A:交通路面智慧檢測方法 A: Traffic road intelligent detection method

a、b、c、d、e:步驟 a, b, c, d, e: steps

Claims (11)

一種雲端路面辨識模組,其在一雲端伺服器內建置一雲端路面辨識模組,且該雲端路面辨識模組會依據各已框選的PCI鋪面特徵的影音串流及其所對應的慣性姿態之數據來分別辨識出不良路面的PCI鋪面狀況指標的特徵值並儲存;其中,前述該慣性姿態之數據係指位於邊緣端的一移動載具的前端、後端所設一前慣性測量單元及一後慣性測量單元所量測到的三軸姿態角、三軸加速度及三軸地球磁場的絕對指向之前慣性姿態、後慣性姿態的數據資料。 A cloud road surface recognition module, which builds a cloud road surface recognition module in a cloud server, and the cloud road road recognition module will base on the audio and video streams of selected PCI pavement features and their corresponding inertia The attitude data are used to identify and store the characteristic values of the PCI pavement condition indicators of bad roads respectively; among them, the aforementioned inertial attitude data refers to a front inertial measurement unit located at the front end and rear end of a mobile vehicle located at the edge. 1. The data of the three-axis attitude angle, the three-axis acceleration and the absolute pointing of the three-axis earth's magnetic field measured by the inertial measurement unit before and after the inertial attitude. 如請求項1所述雲端路面辨識模組,係使用由Tensorflow開發軟體所開發的該雲端路面辨識模組並以scikit-learn作為機器學習的函式庫來進行資料前處理與建立模型,且該雲端路面辨識模組會依據交通道路之不良路面的後設資料,分別辨識出不良路面的該數個串流影音的鋪面狀況指標的特徵值並作儲存。 The cloud road surface recognition module described in request item 1 uses the cloud road road recognition module developed by Tensorflow development software and uses scikit-learn as a machine learning function library for data preprocessing and model building, and the The cloud road surface recognition module will identify and store the characteristic values of the pavement condition indicators of the several streaming videos based on the meta-data of the bad road surface of the traffic road. 如請求項2所述雲端路面辨識模組,其中該雲端路面辨識模組係採用監督式深度學習模型框架,並使用以YOLO演算法所識別數個不良路面串流影音的鋪面狀況指標的特徵值及其所對應的前慣性姿態、後慣性姿態等數據來作為監督式學習的一訓練樣本之標記檔,進而讓該鋪面狀況指標的特徵值與前慣性姿態數據、後慣性姿態數據產生對應關聯,以增進不良路面的辨識效果。 The cloud road surface identification module as described in claim 2, wherein the cloud road surface identification module adopts a supervised deep learning model framework and uses the YOLO algorithm to identify the characteristic values of pavement condition indicators of several bad road surface streaming videos. And its corresponding front inertial attitude, rear inertial attitude and other data are used as a mark file of a training sample for supervised learning, so that the characteristic value of the pavement condition indicator is correspondingly associated with the front inertial attitude data and the rear inertial attitude data. To improve the identification effect of bad road surfaces. 如請求項3所述雲端路面辨識模組,其中該雲端路面辨識模組的訓練及測試係依序經過一資料前處理步驟、一建構機器學習模型並訓練步驟、一評估模型步驟及一預測步驟,而在該資料前處理步驟中將訓練樣本區分為一75%訓練集及一25%測試集,且該訓練樣本的標記檔更包含有GPS的速度數據。 The cloud road surface recognition module described in claim 3, wherein the training and testing of the cloud road road recognition module sequentially go through a data pre-processing step, a machine learning model construction and training step, an evaluation model step and a prediction step. , and in the data pre-processing step, the training samples are divided into a 75% training set and a 25% test set, and the mark file of the training sample further includes GPS speed data. 如請求項1所述雲端路面辨識模組,其中該鋪面狀況指標的特徵值依據ASTM D6433-11所定義之破壞項目合計共有19項破壞,分別為鱷魚狀裂縫、冒油、塊狀裂縫、凸與凹陷、波浪型路面、凹陷、邊緣裂縫、反射裂縫、路肩高差、縱與橫裂縫、補錠、粒料光滑、坑洞、跨越鐵道、車轍、推擠、滑動裂縫、隆起及風化與剝脫。 For the cloud road surface identification module described in request item 1, the characteristic value of the pavement condition indicator is based on the damage items defined by ASTM D6433-11. There are a total of 19 types of damage, including crocodile cracks, oil leakage, massive cracks, and bulges. Related to depressions, wavy pavement, depressions, edge cracks, reflective cracks, shoulder height differences, longitudinal and transverse cracks, patches, granular smoothing, potholes, railroad crossings, ruts, jostling, sliding cracks, bulges and weathering and peeling Take off. 一種交通路面智慧檢測方法,係包含有:一邊緣端資料蒐集步驟,係藉由至少一邊緣端行動裝置蒐集交通各道路不良路面的數個串流影音及所對應的一前慣性姿態、一後慣性姿態、一GPS資訊之數據資訊;一邊緣端不良路面偵測框選步驟,係針對該數個串流影音分別進行初步辨識以框選出一PCI鋪面特徵;一數據資訊上傳雲端步驟,係整合道路不良路面的數個串流影音及其所對應的前慣性姿態、後慣性姿態、GPS資訊成數個後設資料上傳至雲端伺服器作儲存;一雲端不良路面辨識步驟,係在該雲端伺服器更包含有一雲端路面辨識模組,且該雲端路面辨識模組會依據不良路面的該數個串流影音及其所對應的前慣性姿態、後慣性姿態之數據,分別辨識出該數個串流影音的鋪面狀況指標的特徵值並作儲存,其中該前慣性姿態、該後慣性姿態之數據係指位於邊緣端的一移動載具的前端、後端所設一前慣性測量單元及一後慣性測量單元所量測到的三軸姿態角、三軸加速度及三軸地球磁場的絕對指向之前後慣性姿態的數據資料;及一路面辨識結果輸出步驟,係依據前步驟的辨識結果資訊會依據GPS的位置資訊顯示於一地理資訊系統上。 A traffic road surface intelligent detection method includes: an edge end data collection step, which collects several streaming videos of bad road surfaces on each traffic road and the corresponding front inertial posture, and a rear end through at least one edge end mobile device. Inertial posture, data information of GPS information; an edge-end bad road surface detection frame selection step, which is to perform preliminary identification on the several streaming videos to frame a PCI pavement feature; a data information uploading step to the cloud, which is integrated Several streaming videos of bad road surfaces and their corresponding front inertial attitude, rear inertial attitude, and GPS information are uploaded to a cloud server as metadata for storage; a cloud bad road surface identification step is performed on the cloud server It also includes a cloud road surface recognition module, and the cloud road road recognition module will identify the several streams based on the data of the stream videos on the bad road surface and their corresponding front inertial posture and rear inertial posture. The characteristic values of the audio and video pavement condition indicators are stored, in which the data of the front inertial attitude and the rear inertial attitude refer to a front inertial measurement unit and a rear inertial measurement unit located at the front end and rear end of a mobile vehicle at the edge. The data of the three-axis attitude angle, the three-axis acceleration and the absolute pointing of the three-axis earth's magnetic field measured by the unit before and after the inertial attitude; and a road identification result output step, which is based on the identification result information of the previous step and will be based on the GPS Location information is displayed on a geographic information system. 如請求項6所述交通路面智慧檢測方法,其中在該邊緣端不良路面偵測框選步驟中,係針對交通道路的該數個串流影音分別以Tensorflow開發軟體的物件偵測進行初步辨識並框選出一PCI鋪面特徵並同時記錄其GPS、前慣性姿態、後慣性姿態之數據來結合成數個後設資料。 The intelligent detection method of traffic road as described in claim 6, wherein in the edge-end bad road detection frame selection step, the object detection of the Tensorflow development software is used to initially identify and identify the several streaming videos on the traffic road respectively. Select a PCI pavement feature and simultaneously record its GPS, front inertial attitude, and rear inertial attitude data to combine into several metadata. 如請求項7所述交通路面智慧檢測方法,其中在該雲端不良路面辨識步驟中的雲端路面辨識模組係使用由Tensorflow開發軟體所開發的該雲端路面辨識模組並以scikit-learn作為機器學習的函式庫來進行資料前處理與建立模型,且該雲端路面辨識模組會依據交通道路之不良路面的後設資料,分別辨識出不良路面的該數個串流影音的鋪面狀況指標的特徵值並作儲存。 The traffic road surface intelligent detection method described in claim 7, wherein the cloud road surface identification module in the cloud bad road surface identification step uses the cloud road surface identification module developed by Tensorflow development software and uses scikit-learn as the machine learning A function library is used to perform data preprocessing and build models, and the cloud road surface recognition module will identify the characteristics of the pavement condition indicators of the several streaming videos of bad roads based on the meta-data of bad road surfaces on traffic roads. value and store it. 如請求項8所述交通路面智慧檢測方法,其中該雲端路面辨識模組係採用監督式深度學習模型框架,並使用以YOLO演算法所識別數個不良路面串流影音的鋪面狀況指標的特徵值及其所對應的前慣性姿態、後慣性姿態等數據來作為監督式學習的一訓練樣本之標記檔,進而讓該鋪面狀況指標的特徵值與前慣性姿態數據、後慣性姿態數據產生對應關聯,以增進不良路面的辨識效果。 The intelligent traffic road detection method described in claim 8, wherein the cloud road surface recognition module adopts a supervised deep learning model framework and uses the YOLO algorithm to identify the characteristic values of pavement condition indicators of several bad road streaming videos And its corresponding front inertial attitude, rear inertial attitude and other data are used as a mark file of a training sample for supervised learning, so that the characteristic value of the pavement condition indicator is correspondingly associated with the front inertial attitude data and the rear inertial attitude data. To improve the identification effect of bad road surfaces. 如請求項9所述交通路面智慧檢測方法,其中該雲端路面辨識模組的訓練及測試係依序經過一資料前處理步驟、一建構機器學習模型並訓練步驟、一評估模型步驟及一預測步驟,而在該資料前處理步驟中將訓練樣本區分為一75%訓練集及一25%測試集,且該訓練樣本的標記檔更包含有GPS的速度數據。 The intelligent traffic road detection method described in claim 9, wherein the training and testing of the cloud road surface recognition module sequentially go through a data pre-processing step, a machine learning model construction and training step, an evaluation model step and a prediction step. , and in the data pre-processing step, the training samples are divided into a 75% training set and a 25% test set, and the mark file of the training sample further includes GPS speed data. 如請求項6所述交通路面智慧檢測方法,其中在該雲端不良路面辨識步驟中,該鋪面狀況指標的特徵值依據ASTM D6433-11所定義之破壞項 目合計共有19項破壞,分別為鱷魚狀裂縫、冒油、塊狀裂縫、凸與凹陷、波浪型路面、凹陷、邊緣裂縫、反射裂縫、路肩高差、縱與橫裂縫、補錠、粒料光滑、坑洞、跨越鐵道、車轍、推擠、滑動裂縫、隆起及風化與剝脫。 The traffic road surface intelligent detection method described in claim 6, wherein in the cloud bad road surface identification step, the characteristic value of the pavement condition indicator is based on the damage items defined by ASTM D6433-11 There are a total of 19 damages in the project, including crocodile cracks, oil leakage, block cracks, convex and concave, wavy pavement, depression, edge cracks, reflective cracks, shoulder height difference, longitudinal and transverse cracks, patches, and granular materials. Slicks, potholes, track crossings, ruts, jostling, slip cracks, heaving and weathering and spalling.
TW111150393A 2022-12-28 2022-12-28 Traffic road intelligent detection method and cloud road surface identification module TWI828495B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
TW111150393A TWI828495B (en) 2022-12-28 2022-12-28 Traffic road intelligent detection method and cloud road surface identification module

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
TW111150393A TWI828495B (en) 2022-12-28 2022-12-28 Traffic road intelligent detection method and cloud road surface identification module

Publications (1)

Publication Number Publication Date
TWI828495B true TWI828495B (en) 2024-01-01

Family

ID=90459061

Family Applications (1)

Application Number Title Priority Date Filing Date
TW111150393A TWI828495B (en) 2022-12-28 2022-12-28 Traffic road intelligent detection method and cloud road surface identification module

Country Status (1)

Country Link
TW (1) TWI828495B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210012441A1 (en) * 2012-12-10 2021-01-14 Mark Hamilton Schmidt Systems, methods and apparatus for property defect management
TW202143123A (en) * 2020-05-04 2021-11-16 覺華工程科技股份有限公司 Artificial intelligence road inspection and detection system wherein the automated artificial intelligence inspection data can be recorded and uploaded in real time after identifying the damage on the road
CN113850228A (en) * 2021-10-18 2021-12-28 北方民族大学 Road crack detection method and system based on multi-mode fusion
CN114518094A (en) * 2020-11-16 2022-05-20 阿里巴巴集团控股有限公司 Road detection method and system
TW202232373A (en) * 2021-02-02 2022-08-16 覺華工程科技股份有限公司 Smart motorcycle road inspection and detection system have performance increase of reducing the volume, simplifying the apparatus, and lowering the cost
CN115170783A (en) * 2022-06-28 2022-10-11 重庆邮电大学 Expressway pavement crack detection method using street view image
US20220373473A1 (en) * 2020-10-05 2022-11-24 Novi Llc Surface defect monitoring system

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210012441A1 (en) * 2012-12-10 2021-01-14 Mark Hamilton Schmidt Systems, methods and apparatus for property defect management
TW202143123A (en) * 2020-05-04 2021-11-16 覺華工程科技股份有限公司 Artificial intelligence road inspection and detection system wherein the automated artificial intelligence inspection data can be recorded and uploaded in real time after identifying the damage on the road
US20220373473A1 (en) * 2020-10-05 2022-11-24 Novi Llc Surface defect monitoring system
CN114518094A (en) * 2020-11-16 2022-05-20 阿里巴巴集团控股有限公司 Road detection method and system
TW202232373A (en) * 2021-02-02 2022-08-16 覺華工程科技股份有限公司 Smart motorcycle road inspection and detection system have performance increase of reducing the volume, simplifying the apparatus, and lowering the cost
CN113850228A (en) * 2021-10-18 2021-12-28 北方民族大学 Road crack detection method and system based on multi-mode fusion
CN115170783A (en) * 2022-06-28 2022-10-11 重庆邮电大学 Expressway pavement crack detection method using street view image

Similar Documents

Publication Publication Date Title
CN109870456B (en) Rapid detection system and method for road surface health condition
Hadjidemetriou et al. Automated pavement patch detection and quantification using support vector machines
CN112800913B (en) Pavement damage data space-time analysis method based on multi-source feature fusion
CN112800911B (en) Pavement damage rapid detection and natural data set construction method
EP2079984B1 (en) Method of constructing a digital road model
CN111027430B (en) Traffic scene complexity calculation method for intelligent evaluation of unmanned vehicles
JP2019196680A (en) Pavement information gathering inspection system, pavement information gathering inspection method, and program
Jang et al. Framework of data acquisition and integration for the detection of pavement distress via multiple vehicles
US20240167962A1 (en) System and method for automatic monitoring of pavement condition
Benmhahe et al. Automated pavement distress detection, classification and measurement: A review
CN112070756A (en) Three-dimensional road surface disease measuring method based on unmanned aerial vehicle oblique photography
Camilleri et al. Detecting road potholes using computer vision techniques
Shu et al. Pavement crack detection method of street view images based on deep learning
Syed et al. Pothole detection under diverse conditions using object detection models
KR102562757B1 (en) Prediction and recognition method of road marking information and road maintenance method
CN114971166A (en) Lightweight road pavement service quality inspection system
TWI828495B (en) Traffic road intelligent detection method and cloud road surface identification module
CN111723643B (en) Target detection method based on fixed-area periodic image acquisition
TWI821089B (en) Traffic road intelligent detection system and edge collection device
Alshammari et al. 3Pod: Federated Learning-based 3 Dimensional Pothole Detection for Smart Transportation
TW200925354A (en) Robot and method for automatically detecting flatness and surface breakage
Mehajabin et al. Real-time deep learning based road deterioration detection for smart cities
CN108573220B (en) Road crack identification method based on group multi-source data
Kandoi et al. Pothole detection using accelerometer and computer vision with automated complaint redressal
Li et al. Road Damage Evaluation via Stereo Camera and Deep Learning Neural Network