WO2023151401A1 - 异常市政设施的定位方法、装置及其应用 - Google Patents

异常市政设施的定位方法、装置及其应用 Download PDF

Info

Publication number
WO2023151401A1
WO2023151401A1 PCT/CN2022/141315 CN2022141315W WO2023151401A1 WO 2023151401 A1 WO2023151401 A1 WO 2023151401A1 CN 2022141315 W CN2022141315 W CN 2022141315W WO 2023151401 A1 WO2023151401 A1 WO 2023151401A1
Authority
WO
WIPO (PCT)
Prior art keywords
abnormal
municipal
facility
street view
image
Prior art date
Application number
PCT/CN2022/141315
Other languages
English (en)
French (fr)
Inventor
陈斌
章东平
钟梓尹
徐志坚
Original Assignee
城云科技（中国）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 城云科技（中国）有限公司 filed Critical 城云科技（中国）有限公司
Publication of WO2023151401A1 publication Critical patent/WO2023151401A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Definitions

  • the present application relates to the technical fields of urban management and image recognition, in particular to a method, device and application for locating abnormal municipal facilities.
  • Municipal facilities refer to public facilities funded by the government, legal persons, or citizens, and generally refer to various buildings, structures, and equipment in the planning area.
  • Urban roads including bridges
  • urban rail transit water supply, drainage, gas, heat, landscaping, environmental sanitation, road lighting, industrial waste, medical waste, domestic waste treatment equipment, venues and other facilities and ancillary facilities.
  • This application provides a method, device and application for locating abnormal municipal facilities. By collecting road video images and street view video images at the same time, the road video images are detected to find abnormal municipal facilities and perform fast and accurate positioning.
  • the embodiment of the present application provides a method for locating abnormal municipal facilities, including the following steps:
  • the suspected abnormal municipal facility is a municipal facility of the same type as any one of the abnormal municipal facilities, and The location of the suspected abnormal municipal facility is the same as and/or adjacent to the abnormal street view;
  • the corresponding positioning information is obtained. If the abnormal state is displayed as missing, the positioning information is the position information of the suspected abnormal municipal facility closest to the position of the abnormal street view, Otherwise, the positioning information is the position information of the suspected abnormal municipal facility corresponding to the original image most similar to the real-time image of the abnormal municipal facility.
  • the municipal facility information database includes the information of each municipal facility Facility types, location information and original images
  • the street view sample database includes at least one street view sample image and each of the street view sample images is marked with street view location information.
  • the location information of each of the municipal facilities includes original location information and real-time location information; before "building the municipal facility information database", it includes: binding a unique electronic identity tag for each of the municipal facilities , the unique electronic identity tag is used to update the real-time location information of each municipal facility.
  • "obtaining the original image and location information of at least one suspected abnormal municipal facility according to the abnormal status of each of the abnormal municipal facilities” includes: if the abnormal status of the abnormal municipal facility is displayed as missing, from the Obtain the original image and original location information of at least one suspected abnormal municipal facility from the municipal facility information database, wherein the suspected abnormal municipal facility is a municipal facility of the same facility type and the original location information is the same as and/or adjacent to the location of the abnormal street view ; Otherwise, obtain the original image and real-time location information of at least one suspected abnormal municipal facility from the municipal facility information database, wherein the suspected abnormal municipal facility is a municipal facility of the same facility type and the real-time location information is the same as the location of the abnormal street view and/or nearby.
  • "obtaining the street view position information of the street view sample image most similar to the street view video image in the street view sample library as the abnormal street view position" includes: using the first image matching model to calculate the street view video image The first similarity with each of the street view sample images, acquiring the street view position information of the street view sample image with the highest first similarity as the abnormal street view position.
  • "otherwise the location information is the position information of the suspected abnormal municipal facility corresponding to the original image most similar to the real-time image of the abnormal municipal facility” includes: using the second image matching model to calculate the location information of each location The second similarity between the real-time image of the abnormal municipal facility and the original image of each suspected abnormal municipal facility, and the location information is the real-time location information of the suspected abnormal municipal facility with the largest second similarity.
  • "obtaining a real-time image of at least one abnormal municipal facility and the abnormal state and facility type of each abnormal municipal facility according to the road surface video image” includes: inputting the road surface video image into the abnormal municipal facility
  • the detection model obtains at least one bounding box of abnormal municipal facilities, the abnormal state and facility type of each abnormal municipal facility in each said bounding box, and intercepts corresponding real-time images.
  • the embodiment of the present application provides a device for locating abnormal municipal facilities, including the following modules:
  • An acquisition module configured to acquire road surface video images and corresponding street view video images at the same moment
  • a detection module configured to obtain a real-time image of at least one abnormal municipal facility and the abnormal state and facility type of each abnormal municipal facility according to the road surface video image;
  • the street view matching module is used to obtain the street view position information of the street view sample image most similar to the street view video image as an abnormal street view position;
  • a facility matching module configured to acquire the original image and location information of at least one suspected abnormal municipal facility according to the abnormal state of each of the abnormal municipal facilities, wherein the suspected abnormal municipal facility is a facility type related to any one of the abnormal municipal facilities The same municipal facility, and the location of the suspected abnormal municipal facility is the same as and/or adjacent to the abnormal street view;
  • a positioning module configured to obtain corresponding positioning information according to the abnormal state of each of the abnormal municipal facilities, if the abnormal state is displayed as missing, the positioning information is the suspected abnormal municipal facility closest to the position of the abnormal street view.
  • the location information of the facility otherwise the positioning information is the location information of the suspected abnormal municipal facility corresponding to the original image most similar to the real-time image of the abnormal municipal facility.
  • an embodiment of the present application provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program as described in any of the above application embodiments.
  • an embodiment of the present application provides a readable storage medium, where a computer program is stored in the readable storage medium, and the computer program includes a program code for controlling a process to execute the process, and the process includes The method for locating abnormal municipal facilities as described in any application embodiment above.
  • This application preliminarily determines the location of the abnormal street scene through the street view video image corresponding to the same moment as the road surface video image, screens out the suspected abnormal municipal facilities within a certain range according to the abnormal street view position, and adopts different methods according to the abnormal status of the abnormal municipal facilities position.
  • a municipal facility information library including location information and original images of municipal facilities and a street view sample library including a large number of street view sample images are constructed in advance to provide data support for rapid detection and positioning.
  • the position of the suspected abnormal municipal facility closest to the abnormal street view position is used as the positioning information of the abnormal municipal facility, otherwise, comparing the real-time image of the abnormal municipal facility with the original image of each suspected abnormal municipal facility in the municipal facility information database, the suspected abnormal municipal facility most similar to the real-time image of the abnormal municipal facility.
  • the location information of is used as the positioning information of the abnormal municipal facility.
  • a unique electronic identity tag is bound for each municipal facility, and the original location information and real-time location information of each municipal facility are saved in the municipal facility information database.
  • the unique electronic identity tag can update the real-time location information of each municipal facility through radio frequency identification technology, and provide more accurate positioning information for abnormal municipal facilities in different abnormal states.
  • FIG. 1 is a schematic flowchart of a method for locating abnormal municipal facilities according to an embodiment of the present application
  • Fig. 2 is a structural block diagram of a positioning device for abnormal municipal facilities according to an embodiment of the present application
  • FIG. 3 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application.
  • the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification.
  • the method may include more or less steps than those described in this specification.
  • a single step described in this specification may be decomposed into multiple steps for description in other embodiments; multiple steps described in this specification may also be combined into a single step in other embodiments describe.
  • This embodiment provides a method for locating abnormal municipal facilities, which is used to obtain fast and accurate positioning information of detected abnormal municipal facilities.
  • the method package is mainly divided into five steps:
  • Step S1 Obtain the road surface video image and the corresponding street view video image at the same moment;
  • Step S2 Obtain a real-time image of at least one abnormal municipal facility and the abnormal state and facility type of each abnormal municipal facility according to the road surface video image;
  • Step S3 Obtain the street view position information of the street view sample image most similar to the street view video image as the abnormal street view position;
  • Step S4 Obtain the original image and location information of at least one suspected abnormal municipal facility according to the abnormal state of each of the abnormal municipal facilities, wherein the suspected abnormal municipal facility is a municipality of the same type as any one of the abnormal municipal facilities facility, and the location of the suspected abnormal municipal facility is the same as and/or adjacent to the abnormal street view;
  • Step S5 Obtain corresponding location information according to the abnormal state of each of the abnormal municipal facilities. If the abnormal state is displayed as missing, the location information is the location of the suspected abnormal municipal facility closest to the location of the abnormal street view Position information, otherwise the positioning information is the position information of the suspected abnormal municipal facility corresponding to the original image most similar to the real-time image of the abnormal municipal facility.
  • step S1 the purpose of acquiring the road surface video image and the corresponding street view video image at the same moment is to ensure that each frame of road surface video image corresponds to the street view video image one-to-one, so that the road surface video image is corresponding to the street view video image.
  • the specific method can be to install two shooting devices on the urban road work vehicle to collect the road video image and the corresponding street view video image at the same time, one of which is used to obtain the video image of the road ahead when the work vehicle is moving, and the other is used to simultaneously acquire The street view video image in front of the work vehicle when it is moving.
  • step S2 the video image of the road surface is detected, and the real-time image of the abnormal municipal facilities, the abnormal state and the type of the abnormal municipal facilities are obtained.
  • the specific method may be to input the road surface video image into the abnormal municipal facility detection model, obtain the bounding box of at least one abnormal municipal facility, the abnormal state and facility type of the abnormal municipal facility in each said bounding box, and according to each said abnormal The bounding box of the municipal facility intercepts the corresponding real-time image from the road surface video image.
  • the abnormal municipal facility detection model can use a conventional target detection network to conduct corresponding training on one or more municipal facilities that need to be detected and identified, and is used to obtain the bounding box of abnormal municipal facilities and the Abnormal state and type of facility.
  • the target detection network can use Yolov5s or YoloX, these models are existing technologies, and will not be described in detail in this embodiment.
  • the abnormal state is mainly used to distinguish whether the abnormal municipal facilities are lost or not.
  • the manhole cover and the wellhead Take the manhole cover and the wellhead as an example. If the wellhead is detected, it means that the manhole cover of the sewer well has been lost, so the abnormal state needs to be expressed as missing. If the manhole cover is detected, but the manhole cover is deformed or damaged to a certain extent, it means that the The manhole cover of the sewer well needs to be maintained in time, so the abnormal state needs to be expressed as deformation or damage.
  • step S3 the street view sample image most similar to the street view video image is searched first, and then the street view position information of the most similar street view sample image is obtained as the initially located abnormal street view position.
  • the specific method may be to use the first image matching model to calculate the first similarity between the street view video image and each of the street view sample images, and obtain the street view position information of the street view sample image with the highest first similarity as the abnormal street view position .
  • the first image matching model used to obtain the first similarity needs to be trained.
  • a large number of street view images of the same location at different times and different angles are collected to train the first image matching model.
  • the first image matching model is a convolutional neural network model NetVLAD with a VLAD layer.
  • NetVLAD is a technology commonly used for scene comparison and recognition in the technical field, and will not be described in detail in this embodiment.
  • step S4 the original image and location information of at least one suspected abnormal municipal facility are obtained according to the abnormal state of each abnormal municipal facility, and the suspected abnormal municipal facility is ensured to meet two conditions: the facilities are of the same type and the same location or adjacent.
  • the original image of the suspected abnormal municipal facilities is entered in advance; to judge whether the two locations are adjacent, the proximity range threshold can be set in advance, that is to say, the municipal facilities that deviate from the abnormal street view position by no more than the proximity range threshold can also be regarded as suspected abnormalities Municipal facilities.
  • step S5 different methods are selected to obtain positioning information according to abnormal municipal facilities in different abnormal states. If the abnormal state is displayed as lost, the location information is the location information of the suspected abnormal municipal facility closest to the location of the abnormal street view, otherwise the location information is the most similar to the real-time image of the abnormal municipal facility The location information of the suspected abnormal municipal facilities corresponding to the original image of .
  • the quality problems or abnormal causes of each abnormal municipal facility can also be analyzed according to its facility category and location information.
  • the method for obtaining the suspected abnormal municipal facility whose original image is most similar to the real-time image of the abnormal municipal facility includes: using the second image matching model to calculate the difference between the real-time image of each of the abnormal municipal facilities and each of the suspected abnormal municipal facilities.
  • the second image matching model used to obtain the second similarity also needs to be trained.
  • a large number of facility images are collected for each type of municipal facility for training.
  • the second image matching model is the same as the first image matching model, both adopting a convolutional neural network model NetVLAD with a VLAD layer.
  • this embodiment uses different methods to obtain corresponding location information for abnormal municipal facilities in different abnormal states.
  • the real-time image of the abnormal municipal facility can be matched with the original image of the suspected abnormal municipal facility, and the location information of the suspected abnormal municipal facility with the highest similarity can be used as the location of the abnormal municipal facility information.
  • the position information of the suspected abnormal municipal facility closest to the abnormal street view position can only be directly used as the location information of the abnormal municipal facility .
  • the municipal facilities information library includes facility types, location information and original images of each municipal facility
  • the street view sample library includes at least one street view sample image and each street view sample image is marked with street view location information.
  • the street view video image at the same moment of the road surface video image is matched with each street view sample image in the street view sample database, and the position information of the street view sample image with the highest similarity is obtained as the abnormal street view position.
  • the location of the abnormal street view is compared with the facility type and location information of each municipal facility in the municipal facility information database, and suspected abnormal municipal facilities are screened out. After obtaining one or more suspected abnormal municipal facilities, the location information of each abnormal municipal facility is further determined according to the abnormal state of the abnormal municipal facility and the information of each suspected abnormal municipal facility in the municipal facility information database.
  • the location information of each municipal facility includes original location information and real-time location information;
  • the municipal facilities are separable structures, therefore, "obtaining the original image and location information of at least one suspected abnormal municipal facility according to the abnormal state of each abnormal municipal facility” includes: If the abnormal state of the abnormal municipal facility is displayed as missing, obtain the original image and original location information of at least one suspected abnormal municipal facility from the municipal facility information database, wherein the suspected abnormal municipal facility is a municipal facility of the same type of facility And the original location information is the same as and/or adjacent to the location of the abnormal street view; otherwise, the original image and real-time location information of at least one suspected abnormal municipal facility is obtained from the municipal facility information database, wherein the suspected abnormal municipal facility is the same facility types of municipal facilities and the real-time location information is the same as and/or adjacent to the location of the abnormal street view.
  • a sewer manhole cover includes a manhole cover and a wellhead, and the manhole cover and the wellhead can be separated. Then the original position information of the sewer manhole cover is the position data collected when the manhole cover is normally placed on the corresponding wellhead, and the original image is the image data collected on site.
  • the original location information can be used to locate the location that needs immediate maintenance, or when the abnormal municipal facility deviates from the original location, the abnormality can be located using real-time location information The location of the municipal facilities, and recover the corresponding abnormal municipal facilities.
  • the sewer well includes a detachable manhole cover and wellhead, and a unique electronic identity tag is bound to the manhole cover, and the entered original location information can indicate the wellhead position of the sewer well.
  • the abnormal state of the well is detected as missing through the collected pavement video images, it means that only the well head has been detected.
  • the collected pavement video images detect other situations in which the abnormal state of the well is not lost, such as a certain degree of deformation or damage, two situations need to be considered.
  • the first situation is that the manhole cover is placed on the
  • the second case is that the manhole cover is not placed on the original wellhead.
  • the manhole cover cannot be located according to the original position information of the lower well. Only by obtaining the real-time position information of the manhole cover can the precise position of the manhole cover be located, and the maintenance personnel should be discharged in time to recover the manhole cover.
  • this embodiment also provides a device for locating abnormal municipal facilities, which is used to implement the method for locating abnormal municipal facilities described in the first embodiment.
  • the device mainly includes the following modules:
  • An acquisition module configured to acquire road surface video images and corresponding street view video images at the same moment
  • a detection module configured to obtain a real-time image of at least one abnormal municipal facility and the abnormal state and facility type of each abnormal municipal facility according to the road surface video image;
  • the street view matching module is used to obtain the street view position information of the street view sample image most similar to the street view video image as an abnormal street view position;
  • a facility matching module configured to acquire the original image and location information of at least one suspected abnormal municipal facility according to the abnormal state of each of the abnormal municipal facilities, wherein the suspected abnormal municipal facility is a facility type related to any one of the abnormal municipal facilities The same municipal facility, and the location of the suspected abnormal municipal facility is the same as and/or adjacent to the abnormal street view;
  • a positioning module configured to obtain corresponding positioning information according to the abnormal state of each of the abnormal municipal facilities, if the abnormal state is displayed as missing, the positioning information is the suspected abnormal municipal facility closest to the position of the abnormal street view.
  • the location information of the facility otherwise the positioning information is the location information of the suspected abnormal municipal facility corresponding to the original image most similar to the real-time image of the abnormal municipal facility.
  • This embodiment also provides an electronic device, referring to FIG. 3 , including a memory 404 and a processor 402, the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the above-mentioned first embodiment.
  • the processor 402 may include a central processing unit (CPU), or an Application Specific Integrated Circuit (ASIC for short), or may be configured to implement one or more integrated circuits in the embodiments of the present application.
  • CPU central processing unit
  • ASIC Application Specific Integrated Circuit
  • the memory 404 may include a mass memory 404 for data or instructions.
  • the memory 404 may include a hard disk drive (Hard Disk Drive, referred to as HDD), a floppy disk drive, a solid state drive (Solid State Drive, referred to as SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape or a general serial Bus (Universal Serial Bus, referred to as USB) driver or a combination of two or more of the above.
  • Storage 404 may include removable or non-removable (or fixed) media, where appropriate.
  • Memory 404 may be internal or external to the data processing arrangement, where appropriate.
  • memory 404 is a non-volatile (Non-Volatile) memory.
  • the memory 404 includes a read-only memory (Read-Only Memory, ROM for short) and a random access memory (Random Access Memory, RAM for short).
  • the ROM can be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, referred to as PROM), an erasable PROM (Erasable Programmable Read-Only Memory, referred to as EPROM), an electronically programmable Erase PROM (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), electrically rewritable ROM (Electrically Alterable Read-Only Memory, referred to as EAROM) or flash memory (FLASH) or a combination of two or more of these.
  • PROM Programmable Read-Only Memory
  • EPROM Erasable Programmable Read-Only Memory
  • EPROM Erasable Programmable Read-Only Memory
  • EEPROM Electrically rewritable ROM
  • FLASH flash memory
  • the RAM can be a Static Random-Access Memory (SRAM for short) or a Dynamic Random-Access Memory (DRAM for short), where the DRAM can be a fast page Mode dynamic random access memory 404 (Fast Page Mode Dynamic Random Access Memory, referred to as FPMDRAM), extended data output dynamic random access memory (Extended Date Out Dynamic Random Access Memory, referred to as EDODRAM), synchronous dynamic random access memory ( Synchronous Dynamic Random-Access Memory, referred to as SDRAM), etc.
  • SRAM Static Random-Access Memory
  • DRAM Dynamic Random-Access Memory
  • FPMDRAM Fast Page Mode Dynamic Random Access Memory
  • EDODRAM Extended Data output dynamic random access memory
  • SDRAM Synchronous Dynamic Random-Access Memory
  • the memory 404 may be used to store or cache various data files required for processing and/or communication, and possibly computer program instructions executed by the processor 402 .
  • the processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the methods for locating abnormal municipal facilities in the above-mentioned embodiments.
  • the above-mentioned electronic device may further include a transmission device 406 and an input-output device 408 , wherein the transmission device 406 is connected to the above-mentioned processor 402 , and the above-mentioned input-output device 408 is connected to the above-mentioned processor 402 .
  • Transmission device 406 may be used to receive or transmit data via a network.
  • the specific example of the above network may include a wired or wireless network provided by the communication provider of the electronic device.
  • the transmission device includes a network interface controller (NIC for short), which can be connected to other network devices through a base station so as to communicate with the Internet.
  • the transmission device 406 may be a radio frequency (Radio Frequency, RF for short) module, which is used to communicate with the Internet in a wireless manner.
  • RF Radio Frequency
  • Input and output devices 408 are used to input or output information.
  • the input information can be the current data table such as the epidemic situation document, feature data, template table, etc.
  • the output information can be feature fingerprints, fingerprint templates, text classification recommendation information, file template configuration mapping table, file Template configuration information table, etc.
  • the processor 402 may be configured to execute the following steps through a computer program:
  • the suspected abnormal municipal facility is a municipal facility of the same type as any one of the abnormal municipal facilities, and The location of the suspected abnormal municipal facility is the same as and/or adjacent to the abnormal street view;
  • the corresponding positioning information is obtained. If the abnormal state is displayed as missing, the positioning information is the position information of the suspected abnormal municipal facility closest to the position of the abnormal street view, Otherwise, the positioning information is the position information of the suspected abnormal municipal facility corresponding to the original image most similar to the real-time image of the abnormal municipal facility.
  • the embodiment of the present application can be implemented as a computer program product.
  • the computer program product includes a software code part.
  • the software code part is used to implement the positioning method for any abnormal municipal facility in the first embodiment above. .
  • this embodiment of the present application may provide a readable storage medium for implementation.
  • a computer program is stored on the readable storage medium; when the computer program is executed by a processor, any one of the methods for locating abnormal municipal facilities in the first embodiment above is realized.
  • the various embodiments may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device, although the invention is not limited thereto. Although aspects of the present invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, as non-limiting examples, these blocks, devices, systems, techniques or methods described herein may be presented in hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof.
  • Embodiments of the invention may be implemented by computer software executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware.
  • Computer software or programs also called program products
  • a computer program product may include one or more computer-executable components configured to perform an embodiment when the program is run.
  • One or more computer-executable components may be at least one software code or a portion thereof.
  • any blocks of the logic flow in the figures may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions.
  • Software may be stored on physical media such as memory chips or memory blocks implemented within the processor, magnetic media such as hard or floppy disks, and optical media such as eg DVD and its data variants, CD. Physical media are non-transient media.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

本申请提供了一种异常市政设施的定位方法，包括：获取同一时刻的路面视频图像和对应的街景视频图像；根据路面视频图像获取至少一异常市政设施的实时图像及每一异常市政设施的异常状态和设施种类；获取与街景视频图像最相似的街景样本图像的街景位置信息作为异常街景位置；根据每一异常市政设施的异常状态获取至少一疑似异常市政设施的原始图像和位置信息；根据每一异常市政设施的异常状态获取对应的定位信息。该方法通过采集同一时刻的路面视频图像和街景视频图像，对路面视频图像进行检测发现异常市政设施，根据街景视频图像进行初步定位，并根据异常市政设施的异常状态和/或实时图像快速而精准地进行定位。

Description

异常市政设施的定位方法、装置及其应用 技术领域
本申请涉及城市管理和图像识别技术领域，特别是涉及一种异常市政设施的定位方法、装置及其应用。
背景技术
随着城市进程化的脚步较快，以及信息化技术的快速发展，社会公众对市政设施的依赖越来越高，同时也对市政设施的管理提出越来越高的要求。
市政设施是指由政府、法人、或公民出资建造的公共设施，一般指规划区内的各种建筑物、构筑物、设备等。城市道路(含桥梁)、城市轨道交通、供水、排水、燃气、热力、园林绿化、环境卫生、道路照明、工业垃圾医疗垃圾、生活垃圾处理设备、场地等设施及附属设施。
但由于市政施舍通常放置于公共场合，容易出现损坏或丢失的情况，现有市政设施的情况主要还是通过人工在城市管理作业车上进行视觉判断，耗费大量时间和人力，并且容易忽略很多较为隐蔽的市政设施，不能及时发现市政施舍的损坏或丢失情况以上传准确的位置信息到维护人员，以至于维护人员无法及时进行维护，给公众的生活造成一定的不便。
发明内容
本申请提供了一种异常市政设施的定位方法、装置及其应用，通过采集同一时刻的路面视频图像和街景视频图像，对路面视频图像进行检测发现异常市政设施并进行快速而精准的定位。
第一方面，本申请实施例提供了一种异常市政设施的定位方法，包括以下步骤：
获取同一时刻的路面视频图像和对应的街景视频图像；
根据所述路面视频图像获取至少一异常市政设施的实时图像及每一所述异常市政设施的异常状态和设施种类；
获取与所述街景视频图像最相似的街景样本图像的街景位置信息作为异常街景位置；
根据每一所述异常市政设施的异常状态获取至少一疑似异常市政设施的原始图像和位置信息，其中所述疑似异常市政设施为与任一所述异常市政设施的设施种类相同的市政设施，且所述疑似异常市政设施与所述异常街景位置相同和/或邻近；
根据每一所述异常市政设施的异常状态获取对应的定位信息，若所述异常状态显示为丢失，所述定位信息为与所述异常街景位置最接近的所述疑似异常市政设施的位置信息，否则所述定位信息为与所述异常市政设施的实时图像最相似的原始图像对应的疑似异常市政设施的位置信息。
在一些申请实施例中，在“获取同一时刻的路面视频图像和对应的街景视频图像”前，包括：构建市政设施信息库和街景样本库，其中所述市政设施信息库包括每一市政设施的设施种类、位置信息和原始图像，所述街景样本库包括至少一街景样本图像且每一所述街景样本图像标注有街景位置信息。
在一些申请实施例中，每一所述市政设施的位置信息包括原始位置信息和实时位置信息；在“构建市政设施信息库”之前，包括：为每一所述市政设施绑定 唯一电子身份标签，所述唯一电子身份标签用于更新每一所述市政设施的实时位置信息。
在一些申请实施例中，“根据每一所述异常市政设施的异常状态获取至少一疑似异常市政设施的原始图像和位置信息”包括：若所述异常市政设施的异常状态显示为丢失，从所述市政设施信息库中获取至少一疑似异常市政设施的原始图像和原始位置信息，其中所述疑似异常市政设施为相同设施种类的市政设施且原始位置信息与所述异常街景位置相同和/或邻近；否则从所述市政设施信息库中获取至少一疑似异常市政设施的原始图像和实时位置信息，其中所述疑似异常市政设施为相同设施种类的市政设施且实时位置信息与所述异常街景位置相同和/或邻近。
在一些申请实施例中，“获取所述街景样本库中与所述街景视频图像最相似的街景样本图像的街景位置信息作为异常街景位置”包括：采用第一图像匹配模型计算所述街景视频图像与每一所述街景样本图像的第一相似度，获取所述第一相似度最高的街景样本图像的街景位置信息作为异常街景位置。
在一些申请实施例中，“否则所述定位信息为与所述异常市政设施的实时图像最相似的原始图像对应的疑似异常市政设施的位置信息”包括：采用第二图像匹配模型计算每一所述异常市政设施的实时图像与每一所述疑似异常市政设施的原始图像的第二相似度，所述定位信息为所述第二相似度最大的疑似异常市政设施的实时位置信息。
在一些申请实施例中，“根据所述路面视频图像获取至少一异常市政设施的实时图像及每一所述异常市政设施的异常状态和设施种类”包括：将所述路面视频图像输入异常市政设施检测模型，获取至少一异常市政设施的包围框、每一所述包围框中异常市政设施的异常状态和设施种类，根据每一所述异常市政设施的包围框从所述路面视频图像中截取对应的实时图像。
第二方面，本申请实施例提供了一种异常市政设施的定位装置，包括以下模块：
获取模块，用于获取同一时刻的路面视频图像和对应的街景视频图像；
检测模块，用于根据所述路面视频图像获取至少一异常市政设施的实时图像及每一所述异常市政设施的异常状态和设施种类；
街景匹配模块，用于获取与所述街景视频图像最相似的街景样本图像的街景位置信息作为异常街景位置；
设施匹配模块，用于根据每一所述异常市政设施的异常状态获取至少一疑似异常市政设施的原始图像和位置信息，其中所述疑似异常市政设施为与任一所述异常市政设施的设施种类相同的市政设施，且所述疑似异常市政设施与所述异常街景位置相同和/或邻近；
定位模块，用于根据每一所述异常市政设施的异常状态获取对应的定位信息，若所述异常状态显示为丢失，所述定位信息为与所述异常街景位置最接近的所述疑似异常市政设施的位置信息，否则所述定位信息为与所述异常市政设施的实时图像最相似的原始图像对应的疑似异常市政设施的位置信息。
第三方面，本申请实施例提供了一种电子装置，包括存储器和处理器，所述存储器中存储有计算机程序，所述处理器被设置为运行所述计算机程序以如上任意申请实施例所述的异常市政设施的定位方法。
第四方面，本申请实施例提供了一种可读存储介质，所述可读存储介质中存储有计算机程序，所述计算机程序包括用于控制过程以执行过程的程序代码，所述过程包括根据如上任意申请实施例所述的异常市政设施的定位方法。
本申请的主要贡献和创新点如下：
本申请通过与路面视频图像同一时刻对应的街景视频图像初步确定异常街景位置，根据异常街景位置在一定范围内筛查出疑似异常市政设施，并根据异常市政设施的异常状态采用不同的方法进行精准定位。
在本申请的一些申请实施例中，提前构建了包括市政设施的位置信息和原始图像的市政设施信息库和包括大量街景样本图像的街景样本库，为快速检测和定位提供数据支撑。
特别的是，针对异常状态显示为丢失的异常市政设施，通过对比异常街景位置与每一疑似异常市政设施在市政设施信息库中的位置信息，将与异常街景位置最近的疑似异常市政设施的位置信息作为该异常市政设施的定位信息，否则对比异常市政设施的实时图像与每一疑似异常市政设施在市政设施信息库中的原始图像，将与异常市政设施的实时图像最相似的疑似异常市政设施的位置信息作为该异常市政设施的定位信息。
值得一提的是，在本申请的一些申请实施例中，为每一市政设施绑定唯一电子身份标签，并在市政设施信息库中保存每一市政设施的原始位置信息和实时位置信息，该唯一电子身份标签可通过射频识别技术更新每一市政设施的实时位置信息，针对不同异常状态的异常市政设施提供更加准确的定位信息。
附图说明
此处所说明的附图用来提供对本申请的进一步理解，构成本申请的一部分，本申请的示意性实施例及其说明用于解释本申请，并不构成对本申请的不当限定。在附图中：
图1是根据本申请实施例的异常市政设施的定位方法流程示意图；
图2是根据本申请实施例的异常市政设施的定位装置结构框图；
图3是根据本申请实施例的电子装置的硬件结构示意图。
具体实施方式
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本说明书一个或多个实施例相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本说明书一个或多个实施例的一些方面相一致的装置和方法的例子。
需要说明的是：在其他实施例中并不一定按照本说明书示出和描述的顺序来执行相应方法的步骤。在一些其他实施例中，其方法所包括的步骤可以比本说明书所描述的更多或更少。此外，本说明书中所描述的单个步骤，在其他实施例中可能被分解为多个步骤进行描述；而本说明书中所描述的多个步骤，在其他实施例中也可能被合并为单个步骤进行描述。
实施例一
本实施例提供了一种异常市政设施的定位方法，用于获取检测到的异常市政设施进行快速且精准的定位信息。
该方法包主要分为五个步骤：
步骤S1：获取同一时刻的路面视频图像和对应的街景视频图像；
步骤S2：根据所述路面视频图像获取至少一异常市政设施的实时图像及每一所述异常市政设施的异常状态和设施种类；
步骤S3：获取与所述街景视频图像最相似的街景样本图像的街景位置信息作为异常街景位置；
步骤S4：根据每一所述异常市政设施的异常状态获取至少一疑似异常市政设施的原始图像和位置信息，其中所述疑似异常市政设施为与任一所述异常市政设施的设施种类相同的市政设施，且所述疑似异常市政设施与所述异常街景位置相同和/或邻近；
步骤S5：根据每一所述异常市政设施的异常状态获取对应的定位信息，若所述异常状态显示为丢失，所述定位信息为与所述异常街景位置最接近的所述疑似异常市政设施的位置信息，否则所述定位信息为与所述异常市政设施的实时图像最相似的原始图像对应的疑似异常市政设施的位置信息。
在步骤S1中，获取同一时刻的路面视频图像和对应的街景视频图像的目的是为了确保每一帧路面视频图像都和街景视频图像一一对应，以便于根据路面视频图像对应的街景视频图像进行初步定位。具体方法可以是在城市道路作业车上安装两个拍摄设备，同时采集路面视频图像和对应的街景视频图像，其中一个用于获取作业车运动时前方的路面你视频图像，另一个用于同时获取作业车运动时前方的街景视频图像。
在步骤S2中，对路面视频图像进行检测，从中获取异常市政设施的实时图像及异常市政设施的异常状态和设施种类。具体方法可以是将所述路面视频图像输入异常市政设施检测模型，获取至少一异常市政设施的包围框、每一所述包围框中异常市政设施的异常状态和设施种类，根据每一所述异常市政设施的包围框从所述路面视频图像中截取对应的实时图像。
其中，异常市政设施检测模型可以采用常规的目标检测网络对一种或者多种需要检测和识别的市政设施都进行了相应的训练，用于获取异常市政设施的包围框及每一异常市政设施的异常状态和设施种类。在一些实施例中，目标检测网络可以采用Yolov5s或者YoloX，这些模型是现有技术，本实施例中不过多赘述。
特别的是，对于可分离的异常状态设施，异常状态主要用于区分异常市政设施是否丢失。以包括井盖和井口的下水井为例，若检测到井口，表示该下水井的井盖已丢失，因此异常状态需要表示为丢失，若检测到井盖，但井盖存在一定程度的形变或破损，表示该下水井的井盖需要及时维护，因此异常状态需要对应表示为形变或者破损。
在步骤S3中，先查询到与所述街景视频图像最相似的街景样本图像，再获取最相似的街景样本图像的街景位置信息作为初步定位的异常街景位置。具体方法可以是采用第一图像匹配模型计算所述街景视频图像与每一所述街景样本图像的第一相似度，获取所述第一相似度最高的街景样本图像的街景位置信息作为异常街景位置。
其中，用于获取第一相似度的第一图像匹配模型是需要经过训练的。本实施例中，对于每一个地点都收集了大量不同时间不同角度相同地点的街景图像对第一图像匹配模型进行训练。
在一些实施例中，所述第一图像匹配模型为带有VLAD层的卷积神经网络模型NetVLAD，NetVLAD是本技术领域中常用来做场景对比识别的技术，本实施例中不过多赘述。
在步骤S4中，根据每一所述异常市政设施的异常状态获取至少一疑似异常市政设施的原始图像和位置信息，并确保疑似异常市政设施满足两个条件：设施种类相同且位置相同或者邻近。其中，疑似异常市政设施的原始图像是事先录入的；判断两者的位置是否邻近，可以提前设置邻近范围阈值，也就是说，偏离异常街景位置不超过邻近范围阈值的市政设施也可以作为疑似异常市政设施。
在步骤S5中，根据不同异常状态的异常市政设施选择不同的方法获取定位信息。若所述异常状态显示为丢失，所述定位信息为与所述异常街景位置最接近的所述疑似异常市政设施的位置信息，否则所述定位信息为与所述异常市政设施的实时图像最相似的原始图像对应的疑似异常市政设施的位置信息。
并且，根据获取到每个异常市政设施的定位信息，还可以根据其的设施类别和位置信息分析每种异常市政设施存在的质量问题或者异常原因。
其中，获取原始图像与所述异常市政设施的实时图像最相似的疑似异常市政设施的方法包括：采用第二图像匹配模型计算每一所述异常市政设施的实时图像与每一所述疑似异常市政设施的原始图像的第二相似度，所述定位信息为所述第二相似度最大的疑似异常市政设施的实时位置信息。
并且，用于获取第二相似度的第二图像匹配模型也是需要经过训练的，本实施例中，对于每一种市政设施都收集了大量的设施图像进行训练。在一些实施例中，第二图像匹配模型是与第一图像匹配模型相同，均采用带有VLAD层的卷积神经网络模型NetVLAD。
值得一提的是，本实施例针对不同异常状态的异常市政设施采用不同的方法获取对应的定位信息。
对于异常状态不是为丢失状态的异常市政设施，可以将异常市政设施的实时图像与疑似异常市政设施的原始图像进行匹配，将相似度最高的疑似异常市政设施的位置信息作为该异常市政设施的定位信息。
对于异常状态为丢失状态的异常市政设施，由于无法通过实时图像与原始图像进行匹配，只能直接将与所述异常街景位置最接近的疑似异常市政设施的位置信息作为该异常市政设施的定位信息。
另外，为快速定位到异常市政设施提供数据支撑，在一些实施例中，在“获取同一时刻的路面视频图像和对应的街景视频图像”前，包括：构建市政设施信息库和街景样本库，其中所述市政设施信息库包括每一市政设施的设施种类、位置信息和原始图像，所述街景样本库包括至少一街景样本图像且每一所述街景样本图像标注有街景位置信息。
因此，在初步确定异常街景位置时，将路面视频图像同一时刻的街景视频图像与街景样本库中每一街景样本图像进行匹配，获取相似度最高的街景样本图像的位置信息作为异常街景位置。并且，将异常街景位置与市政设施信息库中每一市政设施的设施种类和位置信息进行对比，筛查出疑似异常市政设施。在获取到一个或多个疑似异常市政设施后，根据异常市政设施的异常状态和每一疑似异常市政设施在市政设施信息库的信息，进一步确定每一异常市政设施的定位信息。
特别的是，为了更加精准地获取每一异常市政设施的定位信息，在“构建市政设施信息库”之前，包括：为每一所述市政设施绑定唯一电子身份标签，所述唯一电子身份标签用于更新每一所述市政设施的实时位置信息。该唯一电子身份标签可以通过射频识别技术上传市政设施的实时位置信息。相应的，在市政设施信息库中，每一所述市政设施的位置信息包括原始位置信息和实时位置信息；
值得一提的是，在一些实施例中，市政设施属于可分离的结构，因此，“根 据每一所述异常市政设施的异常状态获取至少一疑似异常市政设施的原始图像和位置信息”包括：若所述异常市政设施的异常状态显示为丢失，从所述市政设施信息库中获取至少一疑似异常市政设施的原始图像和原始位置信息，其中所述疑似异常市政设施为相同设施种类的市政设施且原始位置信息与所述异常街景位置相同和/或邻近；否则从所述市政设施信息库中获取至少一疑似异常市政设施的原始图像和实时位置信息，其中所述疑似异常市政设施为相同设施种类的市政设施且实时位置信息与所述异常街景位置相同和/或邻近。
例如下水道井盖包括井盖和井口，井盖和井口可分离，那么下水道井盖的原始位置信息就是当井盖正常安置在对应的井口时采集的位置数据，原始图像就是现场采集的图像数据。
这种做的好处是，在异常市政设施处于丢失状态下，可以通过原始位置信息定位到需要立即进行维护的地点，或者在异常市政设施偏离原始位置的状态下，通过实时位置信息定位到该异常市政设施所处的地点，并回收对应的异常市政设施。
具体的，如果市政设施为下水井，那么下水井包括可分离的井盖和井口，在井盖上绑定唯一电子身份标签，录入的原始位置信息可以表示下水井的井口位置。当通过采集到的路面视频图像检测到下水井的异常状态显示为丢失时，表示只检测到了井口。对于检测到井口的情况，一定是要赶紧排出维修人员到现场进行检查和维护，防止有人意外落入，并且同时也可以尝试通过该井盖的实时位置信息回收井盖。
同样的，当通过采集到的路面视频图像检测到下水井的异常状态显示为非丢失状态的其他情况时，例如一定程度的形变或者破损，需要考虑两种情况，第一种情况是井盖放置于匹配的井口上，第二种情况是井盖并非放置于原有的井口上。
针对第二种情况，根据该下水井的原始位置信息是无法定位得到井盖的，只有通过获取井盖的实时位置信息才可以定位到井盖精确的位置，及时排出维修人员对井盖进行回收。
实施例二
基于相同的构思，本实施例还提供了一种异常市政设施的定位装置，用于实现实施例一中所描述的异常市政设施的定位方法。如图2所示，该装置主要包括以下模块：
获取模块，用于获取同一时刻的路面视频图像和对应的街景视频图像；
检测模块，用于根据所述路面视频图像获取至少一异常市政设施的实时图像及每一所述异常市政设施的异常状态和设施种类；
街景匹配模块，用于获取与所述街景视频图像最相似的街景样本图像的街景位置信息作为异常街景位置；
设施匹配模块，用于根据每一所述异常市政设施的异常状态获取至少一疑似异常市政设施的原始图像和位置信息，其中所述疑似异常市政设施为与任一所述异常市政设施的设施种类相同的市政设施，且所述疑似异常市政设施与所述异常街景位置相同和/或邻近；
定位模块，用于根据每一所述异常市政设施的异常状态获取对应的定位信息，若所述异常状态显示为丢失，所述定位信息为与所述异常街景位置最接近的所述疑似异常市政设施的位置信息，否则所述定位信息为与所述异常市政设施的实时图像最相似的原始图像对应的疑似异常市政设施的位置信息。
实施例三
本实施例还提供了一种电子装置，参考图3，包括存储器404和处理器402，该存储器404中存储有计算机程序，该处理器402被设置为运行计算机程序以执行上述实施例一中的任意一种异常市政设施的定位方法的步骤。
具体地，上述处理器402可以包括中央处理器(CPU)，或者特定集成电路(Application Specific Integrated Circuit，简称为ASIC)，或者可以被配置成实施本申请实施例的一个或多个集成电路。
其中，存储器404可以包括用于数据或指令的大容量存储器404。举例来说而非限制，存储器404可包括硬盘驱动器(Hard Disk Drive，简称为HDD)、软盘驱动器、固态驱动器(Solid State Drive，简称为SSD)、闪存、光盘、磁光盘、磁带或通用串行总线(Universal Serial Bus，简称为USB)驱动器或者两个或更多个以上这些的组合。在合适的情况下，存储器404可包括可移除或不可移除(或固定)的介质。在合适的情况下，存储器404可在数据处理装置的内部或外部。在特定实施例中，存储器404是非易失性(Non-Volatile)存储器。在特定实施例中，存储器404包括只读存储器(Read-Only Memory，简称为ROM)和随机存取存储器(Random Access Memory，简称为RAM)。在合适的情况下，该ROM可以是掩模编程的ROM、可编程ROM(Programmable Read-Only Memory，简称为PROM)、可擦除PROM(Erasable Programmable Read-Only Memory，简称为EPROM)、电可擦除PROM(Electrically Erasable Programmable Read-Only Memory，简称为EEPROM)、电可改写ROM(Electrically Alterable Read-Only Memory，简称为EAROM)或闪存(FLASH)或者两个或更多个以上这些的组合。在合适的情况下，该RAM可以是静态随机存取存储器(Static Random-Access Memory，简称为SRAM)或动态随机存取存储器(Dynamic Random Access Memory，简称为DRAM)，其中，DRAM可以是快速页模式动态随机存取存储器404(Fast Page Mode Dynamic Random Access Memory，简称为FPMDRAM)、扩展数据输出动态随机存取存储器(Extended Date Out Dynamic Random Access Memory，简称为EDODRAM)、同步动态随机存取内存(Synchronous Dynamic Random-Access Memory，简称SDRAM)等。
存储器404可以用来存储或者缓存需要处理和/或通信使用的各种数据文件，以及处理器402所执行的可能的计算机程序指令。
处理器402通过读取并执行存储器404中存储的计算机程序指令，以实现上述实施例中的任意一种异常市政设施的定位方法。
可选地，上述电子装置还可以包括传输设备406以及输入输出设备408，其中，该传输设备406和上述处理器402连接，该输入输出设备408和上述处理器402连接。
传输设备406可以用来经由一个网络接收或者发送数据。上述的网络具体实例可包括电子装置的通信供应商提供的有线或无线网络。在一个实例中，传输设备包括一个网络适配器(Network Interface Controller，简称为NIC)，其可通过基站与其他网络设备相连从而可与互联网进行通讯。在一个实例中，传输设备406可以为射频(Radio Frequency，简称为RF)模块，其用于通过无线方式与互联网进行通讯。
输入输出设备408用于输入或输出信息。在本实施例中，输入的信息可以是当前数据表例如疫情流调文档、特征数据、模板表等，输出的信息可以是特征指纹、指纹模板、文本分类推荐信息、文件模板配置映射表、文件模板配置信息表等。
可选地，在本实施例中，上述处理器402可以被设置为通过计算机程序执行以下步骤：
获取同一时刻的路面视频图像和对应的街景视频图像；
根据所述路面视频图像获取至少一异常市政设施的实时图像及每一所述异常市政设施的异常状态和设施种类；
获取与所述街景视频图像最相似的街景样本图像的街景位置信息作为异常街景位置；
根据每一所述异常市政设施的异常状态获取至少一疑似异常市政设施的原始图像和位置信息，其中所述疑似异常市政设施为与任一所述异常市政设施的设施种类相同的市政设施，且所述疑似异常市政设施与所述异常街景位置相同和/或邻近；
根据每一所述异常市政设施的异常状态获取对应的定位信息，若所述异常状态显示为丢失，所述定位信息为与所述异常街景位置最接近的所述疑似异常市政设施的位置信息，否则所述定位信息为与所述异常市政设施的实时图像最相似的原始图像对应的疑似异常市政设施的位置信息。
需要说明的是，本实施例中的具体示例可以参考上述实施例及可选实施方式中所描述的示例，本实施例在此不再赘述。
另外，结合上述实施例一中的任意一种异常市政设施的定位方法，本申请实施例可一种计算机程序产品来实现。该计算机程序产品该计算机程序产品包括软件代码部分，当所述计算机程序产品在计算机上被运行时，所述软件代码部分用于执行实现上述实施例一中的任意一种异常市政设施的定位方法。
并且，结合上述实施例一中的任意一种异常市政设施的定位方法，本申请实施例可提供一种可读存储介质来实现。该可读存储介质上存储有计算机程序；该计算机程序被处理器执行时实现上述实施例一中的任意一种异常市政设施的定位方法。
通常，各种实施例可以以硬件或专用电路、软件、逻辑或其任何组合来实现。本发明的一些方面可以以硬件来实现，而其他方面可以以可以由控制器、微处理器或其他计算设备执行的固件或软件来实现，但是本发明不限于此。尽管本发明的各个方面可以被示出和描述为框图、流程图或使用一些其他图形表示，但是应当理解，作为非限制性示例，本文中描述的这些框、装置、系统、技术或方法可以以硬件、软件、固件、专用电路或逻辑、通用硬件或控制器或其他计算设备或其某种组合来实现。
本发明的实施例可以由计算机软件来实现，该计算机软件由移动设备的数据处理器诸如在处理器实体中可执行，或者由硬件来实现，或者由软件和硬件的组合来实现。包括软件例程、小程序和/或宏的计算机软件或程序(也称为程序产品)可以存储在任何装置可读数据存储介质中，并且它们包括用于执行特定任务的程序指令。计算机程序产品可以包括当程序运行时被配置为执行实施例的一个或多个计算机可执行组件。一个或多个计算机可执行组件可以是至少一个软件代码或其一部分。另外，在这一点上，应当注意，如图中的逻辑流程的任何框可以表示程序步骤、或者互连的逻辑电路、框和功能、或者程序步骤和逻辑电路、框和功能的组合。软件可以存储在诸如存储器芯片或在处理器内实现的存储块等物理介质、诸如硬盘或软盘等磁性介质、以及诸如例如DVD及其数据变体、CD等光学介质上。物理介质是非瞬态介质。
本领域的技术人员应该明白，以上实施例的各技术特征可以进行任意的组合， 为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对本申请范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请的保护范围应以所附权利要求为准。

Claims (9)

  1. 异常市政设施的定位方法，其特征在于，包括以下步骤：
    获取同一时刻的路面视频图像和对应的街景视频图像；
    根据所述路面视频图像获取至少一异常市政设施的实时图像及每一所述异常市政设施的异常状态和设施种类；
    获取与所述街景视频图像最相似的街景样本图像的街景位置信息作为异常街景位置；
    若所述异常市政设施的异常状态显示为丢失，从市政设施信息库中获取至少一疑似异常市政设施的原始图像和原始位置信息，其中所述疑似异常市政设施为相同设施种类的市政设施且原始位置信息与所述异常街景位置相同和/或邻近；否则从市政设施信息库中获取至少一疑似异常市政设施的原始图像和实时位置信息，其中所述疑似异常市政设施为相同设施种类的市政设施且实时位置信息与所述异常街景位置相同和/或邻近；
    根据每一所述异常市政设施的异常状态获取对应的定位信息，若所述异常状态显示为丢失，所述定位信息为与所述异常街景位置最接近的所述疑似异常市政设施的位置信息，否则所述定位信息为与所述异常市政设施的实时图像最相似的原始图像对应的疑似异常市政设施的位置信息。
  2. 根据权利要求1所述的异常市政设施的定位方法，其特征在于，在“获取同一时刻的路面视频图像和对应的街景视频图像”前，包括：构建市政设施信息库和街景样本库，其中所述市政设施信息库包括每一市政设施的设施种类、位置信息和原始图像，所述街景样本库包括至少一街景样本图像且每一所述街景样本图像标注有街景位置信息。
  3. 根据权利要求2所述的异常市政设施的定位方法，其特征在于，每一所述市政设施的位置信息包括原始位置信息和实时位置信息；在“构建市政设施信息库”之前，包括：为每一所述市政设施绑定唯一电子身份标签，所述唯一电子身份标签用于更新每一所述市政设施的实时位置信息。
  4. 根据权利要求2所述的异常市政设施的定位方法，其特征在于，“获取所述街景样本库中与所述街景视频图像最相似的街景样本图像的街景位置信息作为异常街景位置”包括：采用第一图像匹配模型计算所述街景视频图像与每一所述街景样本图像的第一相似度，获取所述第一相似度最高的街景样本图像的街景位置信息作为异常街景位置。
  5. 根据权利要求3所述的异常市政设施的定位方法，其特征在于，“否则所述定位信息为与所述异常市政设施的实时图像最相似的原始图像对应的疑似异常市政设施的位置信息”包括：采用第二图像匹配模型计算每一所述异常市政设施的实时图像与每一所述疑似异常市政设施的原始图像的第二相似度，所述定位信息为所述第二相似度最大的疑似异常市政设施的实时位置信息。
  6. 根据权利要求3所述的异常市政设施的定位方法，其特征在于，“根据所述路面视频图像获取至少一异常市政设施的实时图像及每一所述异常市政设施的异常状态和设施种类”包括：将所述路面视频图像输入异常市政设施检测模型，获取至少一异常市政设施的包围框、每一所述包围框中异常市政设施的异常状态和设施种类，根据每一所述异常市政设施的包围框从所述路面视频图像中截取对应的实时图像。
  7. 异常市政设施的定位装置，其特征在于，包括以下模块：
    获取模块，用于获取同一时刻的路面视频图像和对应的街景视频图像；
    检测模块，用于根据所述路面视频图像获取至少一异常市政设施的实时图像及每一所述异常市政设施的异常状态和设施种类；
    街景匹配模块，用于获取与所述街景视频图像最相似的街景样本图像的街景位置信息作为异常街景位置；
    设施匹配模块，用于根据每一所述异常市政设施的异常状态获取至少一疑似异常市政设施的原始图像和位置信息，若所述异常市政设施的异常状态显示为丢失，从市政设施信息库中获取至少一疑似异常市政设施的原始图像和原始位置信息，其中所述疑似异常市政设施为相同设施种类的市政设施且原始位置信息与所述异常街景位置相同和/或邻近；否则从市政设施信息库中获取至少一疑似异常市政设施的原始图像和实时位置信息，其中所述疑似异常市政设施为相同设施种类的市政设施且实时位置信息与所述异常街景位置相同和/或邻近；
    定位模块，用于根据每一所述异常市政设施的异常状态获取对应的定位信息，若所述异常状态显示为丢失，所述定位信息为与所述异常街景位置最接近的所述疑似异常市政设施的位置信息，否则所述定位信息为与所述异常市政设施的实时图像最相似的原始图像对应的疑似异常市政设施的位置信息。
  8. 一种电子装置，包括存储器和处理器，所述存储器中存储有计算机程序，所述处理器被设置为运行所述计算机程序以执行权利要求1至6任一所述的异常市政设施的定位方法。
  9. 一种可读存储介质，其特征在于，所述可读存储介质中存储有计算机程序，所述计算机程序包括用于控制过程以执行过程的程序代码，所述过程包括根据权利要求1至6任一项所述的异常市政设施的定位方法。
PCT/CN2022/141315 2022-02-09 2022-12-23 异常市政设施的定位方法、装置及其应用 WO2023151401A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210119974.6 2022-02-09
CN202210119974.6A CN114170301B (zh) 2022-02-09 2022-02-09 异常市政设施的定位方法、装置及其应用

Publications (1)

Publication Number Publication Date
WO2023151401A1 true WO2023151401A1 (zh) 2023-08-17

Family

ID=80489636

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/141315 WO2023151401A1 (zh) 2022-02-09 2022-12-23 异常市政设施的定位方法、装置及其应用

Country Status (2)

Country Link
CN (1) CN114170301B (zh)
WO (1) WO2023151401A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114170301B (zh) * 2022-02-09 2022-05-17 城云科技（中国）有限公司 异常市政设施的定位方法、装置及其应用

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019008474A (ja) * 2017-06-22 2019-01-17 西日本高速道路株式会社 監視支援システム及び監視支援方法
CN112702568A (zh) * 2020-12-14 2021-04-23 广东荣文科技集团有限公司 异常检测方法及相关装置
CN113628164A (zh) * 2021-07-12 2021-11-09 北京科技大学 一种基于深度学习与web端定位的路面裂缝检测方法
CN113724259A (zh) * 2021-11-03 2021-11-30 城云科技（中国）有限公司 井盖异常检测方法、装置及其应用
CN114170301A (zh) * 2022-02-09 2022-03-11 城云科技（中国）有限公司 异常市政设施的定位方法、装置及其应用

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102063652A (zh) * 2010-12-01 2011-05-18 北京建筑工程学院 道路设施及养护监管方法
CN108205647A (zh) * 2016-12-20 2018-06-26 乐视汽车（北京）有限公司 道路状况监测方法、装置及电子设备
CN110062198B (zh) * 2018-01-18 2022-04-05 杭州海康威视数字技术股份有限公司 一种监控取证方法、装置、系统、电子设备及存储介质
CN109214280B (zh) * 2018-07-27 2021-10-01 北京三快在线科技有限公司 基于街景的店铺识别方法、装置、电子设备及存储介质
CN109547925A (zh) * 2018-12-07 2019-03-29 纳恩博（北京）科技有限公司 位置更新方法、位置及导航路线的显示方法、车辆和系统
JP7358762B2 (ja) * 2019-04-02 2023-10-11 トヨタ自動車株式会社 道路異常検知装置、道路異常検知方法および道路異常検知プログラム
CN113515968A (zh) * 2020-04-09 2021-10-19 华为技术有限公司 街道的异常事件检测方法、装置、设备及介质
CN111666971A (zh) * 2020-04-27 2020-09-15 深圳壹账通智能科技有限公司 基于位置定位的事件识别方法、装置、设备及存储介质
CN113865587A (zh) * 2020-06-30 2021-12-31 伊姆西Ip控股有限责任公司 用于监控现场设备的方法、电子设备和计算机程序产品
CN111832492B (zh) * 2020-07-16 2024-06-04 平安科技（深圳）有限公司 静态交通异常的判别方法、装置、计算机设备及存储介质
CN112668478B (zh) * 2020-12-29 2023-05-26 河南橡树智能科技有限公司 一种井盖监控方法、装置、电子设备和存储介质
CN113343779B (zh) * 2021-05-14 2024-03-12 南方电网调峰调频发电有限公司 环境异常检测方法、装置、计算机设备和存储介质
CN113887321B (zh) * 2021-09-09 2024-08-23 鹏城实验室 一种生成车道级仿真路网的方法、设备及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019008474A (ja) * 2017-06-22 2019-01-17 西日本高速道路株式会社 監視支援システム及び監視支援方法
CN112702568A (zh) * 2020-12-14 2021-04-23 广东荣文科技集团有限公司 异常检测方法及相关装置
CN113628164A (zh) * 2021-07-12 2021-11-09 北京科技大学 一种基于深度学习与web端定位的路面裂缝检测方法
CN113724259A (zh) * 2021-11-03 2021-11-30 城云科技（中国）有限公司 井盖异常检测方法、装置及其应用
CN114170301A (zh) * 2022-02-09 2022-03-11 城云科技（中国）有限公司 异常市政设施的定位方法、装置及其应用

Also Published As

Publication number Publication date
CN114170301A (zh) 2022-03-11
CN114170301B (zh) 2022-05-17

Similar Documents

Publication Publication Date Title
Arya et al. Transfer learning-based road damage detection for multiple countries
Ma et al. A real-time crack detection algorithm for pavement based on CNN with multiple feature layers
US20170186314A1 (en) Method, apparatus and computer program product for traffic lane and signal control identification and traffic flow management
Yan et al. A low-cost video-based pavement distress screening system for low-volume roads
JP2019056668A (ja) 情報処理装置、情報処理システム、情報処理方法、及び情報処理プログラム
WO2023151401A1 (zh) 异常市政设施的定位方法、装置及其应用
JP2011179874A (ja) 舗装路面のクラックの抽出と損傷レベルの評価方法
Tung et al. Large-scale object detection of images from network cameras in variable ambient lighting conditions
WO2018042208A1 (en) Street asset mapping
CN117351241B (zh) 隧道病害的智能检测评定方法、装置、终端及存储介质
Zhai et al. Automatic pavement crack detection using multimodal features fusion deep neural network
Roberts et al. Optimization and sensitivity analysis of existing deep learning models for pavement surface monitoring using low-quality images
JP6223256B2 (ja) マンホール蓋管理装置、マンホール蓋管理方法及びプログラム
Tsai et al. Automatic detection of deficient video log images using a histogram equity index and an adaptive Gaussian mixture model
Hara et al. An initial study of automatic curb ramp detection with crowdsourced verification using google street view images
JP2019057221A (ja) 情報処理装置、情報処理システム、情報処理方法、及び情報処理プログラム
CN114218383A (zh) 重复事件的判定方法、装置及应用
JP2005084930A (ja) 空間データマイニング方法及び空間データマイニングシステム
JP2022056219A (ja) 情報処理装置、情報処理方法及びプログラム
Park et al. PotholeEye--How Can We Effectively Maintain the Pavement Distress?
CN113807445B (zh) 案卷复核方法、装置及电子装置、可读存储介质
Riya et al. Pothole detection methods
Yang et al. Establishment of pavement information decision management platform via cloud computing technology
Chorada et al. CNN-based Real-time Pothole Detection for Avoidance Road Accident
KR102569366B1 (ko) 인공지능을 이용한 사용자 참여형 인식 플랫폼에서 시설물 설치 가능 지점 판별 방법 및 그 장치

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22925751

Country of ref document: EP

Kind code of ref document: A1