WO2022077264A1 - 一种物体识别方法、物体识别装置及电子设备 - Google Patents

一种物体识别方法、物体识别装置及电子设备 Download PDF

Info

Publication number
WO2022077264A1
WO2022077264A1 PCT/CN2020/120891 CN2020120891W WO2022077264A1 WO 2022077264 A1 WO2022077264 A1 WO 2022077264A1 CN 2020120891 W CN2020120891 W CN 2020120891W WO 2022077264 A1 WO2022077264 A1 WO 2022077264A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
road image
obstacle
road
camera
Prior art date
Application number
PCT/CN2020/120891
Other languages
English (en)
French (fr)
Inventor
高翔
高毅鹏
方昌銮
何洪刚
Original Assignee
深圳市锐明技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市锐明技术股份有限公司 filed Critical 深圳市锐明技术股份有限公司
Priority to CN202080002303.0A priority Critical patent/CN112424793B/zh
Priority to PCT/CN2020/120891 priority patent/WO2022077264A1/zh
Publication of WO2022077264A1 publication Critical patent/WO2022077264A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components

Definitions

  • the present application belongs to the technical field of image processing, and in particular, relates to an object recognition method, an object recognition device, an electronic device, and a computer-readable storage medium.
  • the present application provides an object identification method, an object identification device, an electronic device and a computer-readable storage medium, which can identify obstacles appearing on a road in time.
  • the present application provides an object recognition method, including:
  • an object recognition device comprising:
  • an acquisition unit configured to acquire road images collected by a preset camera, wherein the preset camera is installed on the monitored vehicle;
  • the identification unit is configured to perform obstacle identification on the above road image based on a preset algorithm, so as to obtain the obstacle image in the above road image.
  • the present application provides an electronic device, the electronic device includes a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the computer program described above when executing the computer program. The steps of the method of the first aspect.
  • the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, implements the steps of the method in the first aspect.
  • the present application provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by one or more processors, the steps of the method of the first aspect are implemented.
  • the present application has the following beneficial effects: the road is photographed by a camera installed on the vehicle, and the photographed road image is identified, so as to extract the obstacle image from the road image and discover the road in time possible obstacles. It can be understood that, for the beneficial effects of the second aspect to the fifth aspect, reference may be made to the relevant description in the first aspect, which is not repeated here.
  • Fig. 1 is the realization flow chart of the object recognition method provided by the embodiment of the present application.
  • FIG. 2 is a schematic diagram of a training process of a semantic segmentation model in the object recognition method provided by the embodiment of the present application;
  • FIG. 3 is a schematic diagram of a training process of a target detection model in the object recognition method provided by the embodiment of the present application;
  • FIG. 4 is a structural block diagram of an object recognition device provided by an embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • FIG. 1 shows an object recognition method provided by an embodiment of the present application, and the details are as follows:
  • Step 101 obtaining road images collected by a preset camera, wherein the preset camera is installed on the monitored vehicle;
  • a camera that is, a preset camera
  • a camera may be set on the monitored vehicle in advance, so as to collect road images through the preset camera.
  • a camera may be installed in front of and behind the monitored vehicle, that is, the preset cameras may include a first camera disposed in front of the monitored vehicle and a second camera disposed behind the monitored vehicle, the first camera One camera faces the front of the monitored vehicle for capturing images of the road in front of the monitored vehicle; the second camera faces the rear of the monitored vehicle for capturing images of the road behind the monitored camera.
  • the image of the road collected by the first camera may be recorded as the first road image, and it is generally considered that the obstacles appearing in the first road image are usually obstacles thrown by other vehicles.
  • a road image can be used to monitor the obstacle throwing situation of other vehicles; the image of the road collected by the second camera is recorded as the second road image, which can be used to monitor the obstacle throwing situation of the monitored vehicle itself.
  • the installation position of the above-mentioned preset camera is specifically determined by the situation of the monitored vehicle.
  • the first camera can be set at the front windshield of the vehicle to be monitored, the first camera is not blocked by other objects, and the angle of the first camera is adjusted to capture the subject.
  • the second camera can be set at the rear windshield of the monitored vehicle, and the second camera is not blocked by other objects , and by adjusting the angle of the second camera, a target within a second preset distance (for example, 10 meters) behind the monitored vehicle is captured.
  • a second preset distance for example, 10 meters
  • the specific process can be as follows: first, set the calibration mode, in the shooting mode of the camera, display the shooting preview interface, and in this Draw at least one calibration line in the shooting preview interface; after the camera is installed, enter the current distance of the camera from the ground in the calibration mode, that is, the height of the camera, and put it in front of the camera (that is, the actual camera currently shot by the camera). Scene) to set the calibration point, and by adjusting the camera, the calibration line in the shooting preview interface coincides with the calibration point in the actual scene to complete the calibration.
  • the preset camera can report the road image collected by itself to the electronic device with low delay, such as the host, and the host performs the steps proposed in the embodiments of the present application to detect obstacles in the road.
  • the host is also installed on the monitored vehicle, and the host uses the power supply of the monitored vehicle.
  • the electronic device and the preset camera may start running immediately after the monitored vehicle is ignited; or, the electronic device and the preset camera may also start running when the monitored vehicle is in a driving state, that is, , when the monitored vehicle is in a driving state, the electronic device acquires the road image collected by the preset camera.
  • the electronic device may have a built-in positioning module. When the positioning module detects that the position of the electronic device is constantly changing, that is, when the electronic device is in a moving state, it is considered that the monitored vehicle is currently in a driving state.
  • Step 102 Perform obstacle identification on the above road image based on a preset algorithm to obtain an obstacle image in the above road image.
  • the electronic device may first perform a preliminary analysis on a road image collected by a preset camera based on a preset algorithm locally, to determine whether an obstacle is captured in the road image.
  • a preset algorithm may be a semantic segmentation model; or, the above algorithm may also be a target detection model, and the type of the preset algorithm is not limited here.
  • the electronic device may implement step 102 based on the semantic segmentation model, and the above-mentioned step 102 may be embodied as:
  • the road image reported by the preset camera can be semantically segmented by using the trained semantic segmentation model, so as to obtain the image mask result.
  • whether the current pixel belongs to an obstacle can be determined pixel by pixel, and the above step 102 can be embodied as: segment the above road image by the semantic segmentation model passed first, and obtain the road image in the above road image.
  • the category to which each pixel belongs, and then the connected domain analysis is performed based on the category to which each pixel belongs to obtain at least one connected domain, and finally the image mask result is obtained according to the at least one connected domain.
  • the connected domain is determined as the result of the image mask, and the above specified category is at least one category to which the obstacle belongs.
  • the above-mentioned connected domain can be obtained by using a notation method, or other methods, which are not limited here. It should be noted that if there is no connected domain belonging to the specified category, it can be considered that there are no obstacles in the current road image, and no further operations are required.
  • Fig. 2 shows the training process of the semantic segmentation model, specifically: obtaining the original image for training, inputting the original image to the encoding network of the semantic segmentation model; After the encoding process, the encoding result is input to the decoding network of the semantic segmentation model; after decoding the encoding result, the decoding network fuses the decoding result and the annotated image based on the attention mechanism to obtain the training result; calculates the training result and Annotate the loss of the image, and perform gradient backhaul based on the loss to improve the encoding network.
  • the above process is repeated continuously until the training result and the loss of the labeled image converge, and the above process ends, and the trained semantic segmentation model is obtained.
  • the annotated image is an image obtained by performing mask annotation based on the obstacle region in the corresponding original image.
  • A2 Map the above-mentioned image mask result to the above-mentioned road image to obtain an obstacle image in the above-mentioned road image.
  • the image mask result can be mapped back to the corresponding road image according to the coordinates of each pixel in the image mask result, so as to obtain the obstacle image in the road image.
  • the electronic device can upload the obstacle image to a preset platform after obtaining the obstacle image; of course, the electronic device can also select the location where the obstacle image is located.
  • the road image is uploaded to the preset platform.
  • the platform can use a larger-scale network model to analyze the images (obstacle images or road images) uploaded by electronic devices with higher precision to determine whether the images contained in the road images are real obstacles; that is, , to determine whether the identified obstacle image represents the real obstacle.
  • the electronic device may implement step 102 based on the target detection model, and the above-mentioned step 102 may be embodied as:
  • the target detection is performed on the above road image through the trained target detection model to obtain the obstacle image in the above road image.
  • the semantic segmentation model may be replaced by an object detection model.
  • the general target detection model is not effective for detecting objects with a large scale range and irregular shapes
  • additional methods for targets of different scales are added.
  • Multiscale fusion module Please refer to FIG. 3.
  • FIG. 3 shows the training process of the target detection model, specifically: obtaining an original image for training, inputting the original image into the target detection model to be trained, and obtaining multiple data in the target detection model.
  • the detection frame output by the scale fusion module (that is, the training result based on the original image), where the multi-scale fusion operation performed by the multi-scale fusion module is specifically: using Convolutional Neural Networks (CNN) technology , establish a feature pyramid, predict the target contained in each layer (that is, the detection result of each layer), and finally fuse all the detection results together to obtain the final detection frame and output; calculate the loss of the training result and the labeled image, And according to this loss, the gradient backhaul is carried out to improve the above target detection model.
  • the above process is repeated continuously until the training result and the loss of the labeled image converge, and the above process ends, and the trained target detection model is obtained.
  • the labeled image is an image obtained after the obstacle region has been labeled in the corresponding original image.
  • the detection frame output by the above target detection model (the range selected by the detection frame is the obstacle image) has lower accuracy, but Running speed has been improved. That is, when the solution of the present application is implemented based on the semantic segmentation model, its precision is higher and the effect is better; when the solution of the present application is implemented based on the target detection model, the processing speed is faster; the user can select the algorithm to be used according to his own needs.
  • the electronic device can locally calculate the area of the above-mentioned obstacle image, and at the same time obtain the current positioning position of the monitored vehicle (that is, the position of the monitored vehicle at the moment when the image of the road where the obstacle image is located is captured. position), and upload the area and the positioning position to the preset platform for certification.
  • the platform confirms that the road image contains an image of a real obstacle, the platform will save the area and the positioning position to form a chain of evidence for subsequent evaluation of whether the vehicle is thrown by obstacles. .
  • the above-mentioned object recognition method may further include:
  • a reminder message is output to the monitored vehicle to remind the monitored vehicle that there is an obstacle throwing behavior.
  • the obstacles detected in the first road image can be considered as not the obstacles thrown by the monitored vehicle; , if an area is originally clean, the area is in front of the monitored vehicle when the monitored vehicle has not yet driven to the area, so the first road image including the area will be captured by the first camera. , there is no obstacle image in the first road image; when the monitored vehicle travels to the area, that is, when the monitored vehicle overlaps with the area, if the monitored vehicle is thrown, for example, the driver throws garbage or If the object transported by the monitored vehicle leaks, it will cause obstacles in the area; then the monitored vehicle continues to move forward, and the area will be behind the monitored vehicle.
  • the second road image of the area at this time, there are obstacles in the second road image. Based on the above process, it can be seen that when there is no obstacle image in the first road image and there is an obstacle image in the second road image, it is considered that the monitored vehicle has thrown obstacles, and a reminder can be output to the monitored vehicle at this time. information.
  • the first camera and the second camera it is obviously impossible for the first camera and the second camera to capture the same area at the same time; often the first camera captures the first road image including a certain area, and then waits for a preset time until the monitored vehicle is driving. After a certain distance, the second camera will capture the second road image including the same area. It can be considered that there is a time delay between the matching first road image and the second road image, that is, the first road image and the second road image including the same area (the same area as the shooting object), and the time delay is determined by the monitored It is determined by the speed of the vehicle: the faster the speed, the smaller the delay; the slower the speed, the greater the delay.
  • the above-mentioned step of detecting whether the vehicle has an obstacle throwing behavior may be specifically as follows: if there is no obstacle image in the first road image, and there is an obstacle image in the second road image matched with the first camera, then send the object to the victim.
  • the monitoring vehicle outputs a reminder message to remind the monitored vehicle that there is an obstacle throwing behavior.
  • the road is photographed by the camera erected on the vehicle, and the photographed road image is identified, so as to extract the obstacle image from the road image, so as to identify the road image. Identify obstacles that may appear on the road in time.
  • cameras can be set up at different positions of the vehicle to capture road images ahead and road images behind, so as to monitor whether there is an obstacle throwing behavior in the vehicle.
  • an embodiment of the present application provides an object recognition device, and the above object recognition device is integrated into an electronic device.
  • the object recognition apparatus 400 in the embodiment of the present application includes:
  • an acquisition unit 401 configured to acquire road images collected by a preset camera, wherein the preset camera is installed on the monitored vehicle;
  • the identification unit 402 is configured to perform obstacle identification on the above-mentioned road image based on a preset algorithm, so as to obtain an obstacle image in the above-mentioned road image.
  • the above-mentioned identifying unit 402 includes:
  • a segmentation subunit used for segmenting the above road image through the trained semantic segmentation model to obtain an image mask result
  • a mapping subunit configured to map the above-mentioned image mask result to the above-mentioned road image, and obtain the obstacle image in the above-mentioned road image.
  • the above-mentioned object recognition apparatus 400 further includes:
  • an extraction unit for inputting the above-mentioned road image into a trained feature extraction network to obtain image features of the above-mentioned road image
  • the above-mentioned segmentation subunit is specifically configured to segment the above-mentioned image features by using the trained semantic segmentation model to obtain an image mask result.
  • the above-mentioned segmentation subunit includes:
  • the category determination subunit is used to segment the above-mentioned road image through the trained semantic segmentation model, and obtain the category to which each pixel in the above-mentioned road image belongs:
  • the connected domain analysis subunit is used to analyze the connected domain based on the category to which each pixel point belongs to obtain at least one connected domain;
  • the image mask result obtaining subunit is used to obtain the image mask result according to the above at least one connected domain.
  • the above-mentioned identifying unit 402 includes:
  • the target detection subunit is used for performing target detection on the above road image through the trained target detection model, so as to obtain the obstacle image in the above road image.
  • the above-mentioned object recognition apparatus 400 further includes:
  • the first uploading unit is used for uploading the above-mentioned obstacle image to a preset platform, so as to instruct the above-mentioned platform to verify whether the above-mentioned obstacle image represents a real obstacle.
  • the above-mentioned object recognition apparatus 400 further includes:
  • a positioning unit used to obtain the positioning position of the above-mentioned monitored vehicle
  • the second uploading unit is used for uploading the above-mentioned area and the above-mentioned positioning position to a preset platform for certificate storage.
  • the above-mentioned preset camera includes a first camera and a second camera, the above-mentioned first camera faces the front of the monitored vehicle, the above-mentioned second camera faces the rear of the monitored vehicle, and the above-mentioned road image includes the image collected by the above-mentioned first camera.
  • the first road image and the second road image collected by the second camera; the object recognition device 400 further includes:
  • Reminder unit for if there is no obstacle image in the above-mentioned first road image, and there is an obstacle image in the above-mentioned second road image, then output a reminder message to the above-mentioned monitored vehicle to remind the above-mentioned monitored vehicle that there is an obstacle throwing Behavior.
  • the obtaining unit 401 is specifically configured to obtain a road image collected by the preset camera when the monitored vehicle is in a driving state.
  • the road is photographed by the camera erected on the vehicle, and the object recognition device can immediately recognize the photographed road image, thereby extracting obstacles from the road image. image of objects, and find possible obstacles in the road in time.
  • cameras can be set up at different positions of the vehicle to capture road images ahead and road images behind, so as to monitor whether there is an obstacle throwing behavior in the vehicle.
  • the electronic device 5 in the embodiment of the present application includes: a memory 501, one or more processors 502 (only one is shown in FIG. A computer program on memory 501 and executable on a processor.
  • the memory 501 is used to store software programs and units, and the processor 502 executes various functional applications and data processing by running the software programs and units stored in the memory 501 to obtain resources corresponding to the above preset events.
  • the processor 502 implements the following steps by running the above-mentioned computer program stored in the memory 501:
  • the above-mentioned obstacle recognition is performed on the above-mentioned road image based on the preset algorithm, so as to Obtain an image of obstacles in the above road image, including:
  • the above-mentioned image mask result is mapped to the above-mentioned road image, and the obstacle image in the above-mentioned road image is obtained.
  • the processor 502 runs the The above-mentioned computer program stored in the memory 501 also implements the following steps:
  • the above-mentioned road image is input into the trained feature extraction network to obtain the image features of the above-mentioned road image;
  • the above-mentioned road image is segmented by the trained semantic segmentation model to obtain an image mask result, including:
  • the above image features are segmented by the trained semantic segmentation model, and the image mask result is obtained.
  • the above-mentioned road image is segmented by the trained semantic segmentation model, and an image mask result is obtained, including:
  • the above road image is segmented by the trained semantic segmentation model to obtain the category to which each pixel in the above road image belongs:
  • An image mask result is obtained according to the above at least one connected domain.
  • the above-mentioned obstacle recognition is performed on the above-mentioned road image based on a preset algorithm, so as to obtain an obstacle image in the above-mentioned road image, including: :
  • the target detection is performed on the above road image through the trained target detection model to obtain the obstacle image in the above road image.
  • processing The device 502 also implements the following steps by running the above-mentioned computer program stored in the memory 501:
  • the device 502 Based on the above first possible implementation manner, or the above second possible implementation manner as a basis, or the above third possible implementation manner as a basis, or the above fourth possible implementation manner as a basis, or the above In the seventh possible implementation manner provided on the basis of the fifth possible implementation manner, after the above-mentioned road image is identified based on the preset algorithm to obtain the obstacle image in the above-mentioned road image, processing The device 502 also implements the following steps by running the above-mentioned computer program stored in the memory 501:
  • the preset camera includes a first camera and a second camera, the first camera is directed toward the front of the monitored vehicle, and the second camera is directed toward the front of the monitored vehicle.
  • the above-mentioned road image includes the first road image collected by the above-mentioned first camera and the second road image collected by the above-mentioned second camera; the processor 502 also realizes by running the above-mentioned computer program stored in the memory 501. The following steps:
  • a reminder message is output to the monitored vehicle to remind the monitored vehicle that there is an obstacle throwing behavior.
  • the above-mentioned acquisition of the road image collected by the preset camera includes:
  • the road image collected by the preset camera is acquired.
  • the processor 502 may be a central processing unit (Central Processing Unit, CPU), and the processor may also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP) , Application Specific Integrated Circuit (ASIC), off-the-shelf Programmable Gate Array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • Memory 501 may include read only memory and random access memory, and provides instructions and data to processor 502 . Part or all of memory 501 may also include non-volatile random access memory. For example, the memory 501 may also store information of device categories.
  • the road is photographed by the camera erected on the vehicle, and the electronic device can immediately identify the photographed road image, thereby extracting obstacles from the road image. images, and timely detection of possible obstacles in the road.
  • cameras can be set up at different positions of the vehicle to capture road images ahead and road images behind, so as to monitor whether there is an obstacle throwing behavior in the vehicle.
  • the disclosed apparatus and method may be implemented in other manners.
  • the system embodiments described above are only illustrative.
  • the division of the above-mentioned modules or units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined. Either it can be integrated into another system, or some features can be omitted, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.
  • the units described above as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • the above-mentioned integrated units are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium.
  • the present application can implement all or part of the processes in the methods of the above embodiments, and can also be completed by instructing the associated hardware through a computer program, and the above computer program can be stored in a computer-readable storage medium, the computer When the program is executed by the processor, the steps of the foregoing method embodiments can be implemented.
  • the above-mentioned computer program includes computer program code
  • the above-mentioned computer program code may be in the form of source code, object code form, executable file or some intermediate form.
  • the above-mentioned computer-readable storage medium may include: any entity or device capable of carrying the above-mentioned computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer-readable memory, a read-only memory (ROM, Read-Only Memory) ), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.
  • a recording medium a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer-readable memory, a read-only memory (ROM, Read-Only Memory) ), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.
  • ROM Read-Only Memory
  • RAM Random Access Memory
  • electric carrier signal telecommunication signal and software distribution medium, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Molecular Biology (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)

Abstract

一种物体识别方法、物体识别装置、电子设备及计算机可读存储介质，该方法包括：获取预设摄像头所采集的道路图像，其中，所述预设摄像头安装于被监控车辆（101）；基于预设的算法对所述道路图像进行障碍物识别，以获得所述道路图像中的障碍物图像（102）。该方法可以及时识别出道路中所出现的障碍物。

Description

一种物体识别方法、物体识别装置及电子设备 技术领域
本申请属于图像处理技术领域，尤其涉及一种物体识别方法、物体识别装置、电子设备及计算机可读存储介质。
 背景技术
随着汽车保有量越来越多，人们越来越关注与道路安全相关的问题。考虑到驾驶员的素质参差不齐，可能出现随手扔垃圾的情况；以及部分车辆在运输过程中也可能由于路面颠簸而导致车辆所运输的物体出现抛洒情况。
 技术问题
可能会导致行驶路面上出现障碍物，给道路安全带来风险。
 技术解决方案
本申请提供了一种物体识别方法、物体识别装置、电子设备及计算机可读存储介质，可以及时识别出道路中所出现的障碍物。
第一方面，本申请提供了一种物体识别方法，包括：
获取预设摄像头所采集的道路图像，其中，上述预设摄像头安装于被监控车辆；
基于预设的算法对上述道路图像进行障碍物识别，以获得上述道路图像中的障碍物图像。
第二方面，本申请提供了一种物体识别装置，包括：
获取单元，用于获取预设摄像头所采集的道路图像，其中，上述预设摄像头安装于被监控车辆；
识别单元，用于基于预设的算法对上述道路图像进行障碍物识别，以获得上述道路图像中的障碍物图像。
第三方面，本申请提供了一种电子设备，上述电子设备包括存储器、处理器以及存储在上述存储器中并可在上述处理器上运行的计算机程序，上述处理器执行上述计算机程序时实现如上述第一方面的方法的步骤。
第四方面，本申请提供了一种计算机可读存储介质，上述计算机可读存储介质存储有计算机程序，上述计算机程序被处理器执行时实现如上述第一方面的方法的步骤。
第五方面，本申请提供了一种计算机程序产品，上述计算机程序产品包括计算机程序，上述计算机程序被一个或多个处理器执行时实现如上述第一方面的方法的步骤。
 有益效果
本申请与现有技术相比存在的有益效果是：通过架设于车辆的摄像头对道路进行拍摄，并对拍摄到的道路图像进行识别，以此从道路图像提取出障碍物图像，以及时发现道路中所可能出现的障碍物。可以理解的是，上述第二方面至第五方面的有益效果可以参见上述第一方面中的相关描述，在此不再赘述。
 附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的物体识别方法的实现流程图；
图2是本申请实施例提供的物体识别方法中，语义分割模型的训练流程示意图；
图3是本申请实施例提供的物体识别方法中，目标检测模型的训练流程示意图；
图4是本申请实施例提供的物体识别装置的结构框图；
图5是本申请实施例提供的电子设备的结构示意图。
 本发明的实施方式
以下描述中，为了说明而不是为了限定，提出了诸如特定系统结构、技术之类的具体细节，以便透彻理解本申请实施例。然而，本领域的技术人员应当清楚，在没有这些具体细节的其它实施例中也可以实现本申请。在其它情况中，省略对众所周知的系统、装置、电路以及方法的详细说明，以免不必要的细节妨碍本申请的描述。
为了说明本申请所提出的技术方案，下面通过具体实施例来进行说明。
实施例1
请参阅图1，图1示出了本申请实施例提供的一种物体识别方法，详述如下：
步骤101，获取预设摄像头所采集的道路图像，其中，上述预设摄像头安装于被监控车辆；
在本申请实施例中，可预先在被监控车辆上设置摄像头，也即预设摄像头，以通过该预设摄像头来采集道路图像。示例性地，可以在被监控车辆的前方及后方均设置一摄像头，也即，预设摄像头可以包括设置于被监控车辆前方的第一摄像头及设置于被监控车辆后方的第二摄像头，该第一摄像头朝向被监控车辆的前方，用以采集在被监控车辆前面的道路的图像；第二摄像头朝向被监控车辆的后方，用以采集在被监控摄像头后面的道路的图像。为便于说明，可将通过第一摄像头所采集的道路的图像记为第一道路图像，一般认为该第一道路图像中所出现的障碍物通常为其它车辆所抛洒的障碍物，因而，该第一道路图像可用于监控其它车辆的障碍物抛洒情况；将通过第二摄像头所采集的道路的图像记为第二道路图像，可用于监控该被监控车辆自身的障碍物抛洒情况。
需要注意的是，上述预设摄像头的安装位置具体由被监控车辆的本身情况所决定。举例来说，对于第一摄像头来说，该第一摄像头可设置于被监控车辆的前风窗处，该第一摄像头不被其它物体所遮挡，并通过调整该第一摄像头的角度来拍摄被监控车辆前方第一预设距离（例如20米）内的目标；对于第二摄像头来说，该第二摄像头可设置于被监控车辆的后风窗处，该第二摄像头不被其它物体所遮挡，并通过调整该第二摄像头的角度来拍摄被监控车辆后方第二预设距离（例如10米）内的目标。除了安装上述预设摄像头外，还需要采用标定工具对该预设摄像头的内参进行标定，具体过程可以是：首先设定标定模式，在摄像头的拍摄模式下，显示出拍摄预览界面，并在该拍摄预览界面中画出至少一根标定线；在摄像头安装完毕后，在标定模式下输入摄像头当前距离地面的距离，也即摄像头的高度，并在摄像头的前方（也即摄像头当前所拍摄的实际场景）设定标定点，通过调节摄像头使拍摄预览界面中的标定线与实际场景下的标定点重合，以完成标定。
预设摄像头可将自己所采集到的道路图像低延时上报给电子设备，例如主机处，由主机执行本申请实施例所提出的各个步骤，实现对道路中障碍物的检测。其中，该主机也安装于被监控车辆上，且主机使用的是被监控车辆的电源。
在一些实施例中，电子设备及预设摄像头可以是在被监控车辆点火启动后即刻开始运行；或者，电子设备及预设摄像头也可以是在被监控车辆处于行驶状态时才开始运行，也即，当被监控车辆处于行驶状态时，电子设备才获取预设摄像头所采集的道路图像。该电子设备可以内置有定位模块，当通过定位模块检测到电子设备的位置一直在更改时，也即电子设备处于移动状态时，即认为被监控车辆当前正处于行驶状态。
步骤102，基于预设的算法对上述道路图像进行障碍物识别，以获得上述道路图像中的障碍物图像。
在本申请实施例中，电子设备可以先在其本地基于预设的算法对预设摄像头所采集到的道路图像进行初步分析，以确定该道路图像是否有拍摄到障碍物。示例性地，上述算法可以是语义分割模型；或者，上述算法也可以是目标检测模型，此处不对该预设的算法的类型作出限定。
在一种应用场景下，电子设备可基于语义分割模型实现步骤102，则上述步骤102可具体表现为：
A1、通过已训练的语义分割模型对上述道路图像进行分割，获得图像掩码结果；
在本申请实施例中，可通过已训练的语义分割模型对预设摄像头所上报的道路图像进行语义分割，用以获得图像掩码结果。需要注意的是，本申请实施例中，可逐个像素的判断当前像素是否属于障碍物，则上述步骤102可具体表现为：先通过的语义分割模型对上述道路图像进行分割，获得上述道路图像中每个像素点所属的类别，随后基于各个像素点所属的类别进行连通域分析，得到至少一个连通域，最后根据上述至少一个连通域获得图像掩码结果，具体为将所属的类别为指定类别的连通域确定为图像掩码结果，上述指定类别为障碍物所属的至少一个类别。上述连通域的求取可以采用标记法，或者可以采用其它方法，此处不作限定。需要注意的是，若不存在所属的类别为指定类别的连通域，即可认为当前的道路图像中不存在障碍物，无需再进行后续操作。
在一些实施例中，也可以是先将上述道路图像输入至已训练的特征提取网络，获得上述道路图像的图像特征，随后再通过已训练的语义分割模型对提取得到的图像特征进行分割，可一定程度降低语义分割模型的计算量，提升语义分割模型的工作效率。
在一些实施例中，在上述语义分割模型的基础上，还可添加注意力机制，使语义分割模型在训练时可以专注于存在障碍物的区域的训练，忽略其他区域。请参阅图2，图2示出了该语义分割模型的训练过程，具体为：获取用于训练的原始图像，将该原始图像输入至语义分割模型的编码网络；该编码网络在对原始图像进行编码处理后，将编码结果输入至语义分割模型的解码网络；该解码网络在对编码结果进行解码处理后，将解码结果与标注图像基于注意力机制进行融合，得到训练结果；计算该训练结果与标注图像的损失，并根据该损失进行梯度回传，对编码网络作出改进。上述过程不断重复，直至训练结果与标注图像的损失达到收敛时，结束上述过程，得到已训练的语义分割模型。其中，标注图像是基于对应的原始图像中的障碍物区域进行了掩码标注后所得到的图像。
A2，将上述图像掩码结果映射至上述道路图像，获得上述道路图像中的障碍物图像。
在本申请实施例中，可根据图像掩码结果中各个像素的坐标，将该图像掩码结果映射回对应的道路图像中，即可得到道路图像中的障碍物图像。在一些实施例中，为了保障检测结果的真实性，电子设备可以在获得障碍物图像后，将该障碍物图像上传至预设的平台；当然，电子设备也可以选择将该障碍物图像所在的道路图像上传至预设的平台。该平台可以使用更大规模的网络模型对电子设备所上传的图像（障碍物图像或道路图像）进行更高精度的分析，以确定道路图像中所包含的是否为真实障碍物的图像；也即，确定所识别出的障碍物图像是否表达了真实障碍物。
在另一种应用场景下，电子设备可基于目标检测模型实现步骤102，则上述步骤102可具体表现为：
通过已训练的目标检测模型对上述道路图像进行目标检测，以获得上述道路图像中的障碍物图像。
在本申请实施例中，出于算力的考虑，可使用目标检测模型替换语义分割模型。但考虑到一般的目标检测模型对于尺度范围较大，且形状不规则的物体进行检测时的检测效果不明显，因而在对目标检测模型进行训练的过程中，还添加了针对于不同尺度目标的多尺度融合模块。请参阅图3，图3示出了该目标检测模型的训练过程，具体为：获取用于训练的原始图像，将该原始图像输入至待训练的目标检测模型，获得该目标检测模型中的多尺度融合模块所输出的检测框（也即基于原始图像所得的训练结果），其中，该多尺度融合模块所进行的多尺度融合操作具体为：使用卷积神经网络（Convolutional Neural Networks，CNN）技术，建立特征金字塔，预测每一层含有的目标（也即每一层的检测结果），最后将所有检测结果融合在一起，得到最终的检测框并输出；计算该训练结果与标注图像的损失，并根据该损失进行梯度回传，对上述目标检测模型进行改进。上述过程不断重复，直至训练结果与标注图像的损失达到收敛时，结束上述过程，得到已训练的目标检测模型。其中，标注图像是在对应的原始图像中已对障碍物区域进行了标注后所得到的图像。
需要注意的是，相对于基于语义分割模型所识别出的障碍物图像，上述目标检测模型所输出的检测框（该检测框所框选的范围内即为障碍物图像）的精度较低，但运行速度有所提升。也即，基于语义分割模型实现本申请方案时，其精度更高，效果更好；而基于目标检测模型实现本申请方案时，处理速度更快；用户可根据自身需求，选取所要采用的算法。
在一些实施例中，电子设备可以在本地计算上述障碍物图像的面积，同时获取当前该被监控车辆的定位位置（也即拍摄该障碍物图像所在的道路图像的时刻下，被监控车辆的定位位置），并将该面积及该定位位置上传至预设的平台，以进行存证。具体地，可以是在平台确认道路图像中所包含的是真实障碍物的图像时，该平台才会保存该面积及该定位位置，以形成证据链，用以后续评估车辆是否出现障碍物抛洒情况。
在一些实施例中，在前文所给出的被监控车辆上安装有第一摄像头及第二摄像头的应用场景下，上述物体识别方法还可以包括：
若上述第一道路图像中不存在障碍物图像，且上述第二道路图像中存在障碍物图像，则向上述被监控车辆输出提醒消息，以提醒上述被监控车辆存在障碍物抛洒行为。
由于第一道路图像是被监控车辆前方的图像，考虑到车辆一般都是向前行驶，因而，在第一道路图像中所检测到的障碍物可以认为不是被监控车辆所抛洒的障碍物；反之，若某一区域本来是干净的，被监控车辆在还未行驶到该区域时，该区域处于被监控车辆前方，因而会先通过第一摄像头拍摄到包含该区域的第一道路图像，此时，第一道路图像不存在障碍物图像；当被监控车辆行驶到该区域时，也即，被监控车辆与该区域重合时，若被监控车辆出现抛洒现象，例如，驾驶员向外扔垃圾或者被监控车辆所运输的物体发生撒漏，则会导致该区域出现障碍物；之后被监控车辆继续向前行驶，该区域会处于被监控车辆后方，这时会再通过第二摄像头拍摄到包含该区域的第二道路图像，此时，第二道路图像存在障碍物。基于上述过程可知，当第一道路图像中不存在障碍物图像，且第二道路图像中存在障碍物图像时，认为是被监控车辆出现了障碍物抛洒行为，此时可向被监控车辆输出提醒消息。
需要注意的是，显然第一摄像头及第二摄像头不可能同时拍摄到同一区域；往往是第一摄像头先拍摄到包含某一区域的第一道路图像，之后等待预设时间，待被监控车辆行驶了一段距离后，第二摄像头才会拍摄到包含同一区域的第二道路图像。可以认为，相匹配的第一道路图像及第二道路图像，也即包含相同区域（拍摄对象为相同区域）的第一道路图像及第二道路图像之间存在时延，该时延由被监控车辆的车速所决定：车速越快，该时延越小；车速越慢，该时延越大。则上述检测车辆是否存在障碍物抛洒行为的步骤可具体为：若第一道路图像中不存在障碍物图像，且与该第一摄像头相匹配的第二道路图像中存在障碍物图像，则向被监控车辆输出提醒消息，以提醒被监控车辆存在障碍物抛洒行为。
由上可见，通过本申请实施例，在车辆行驶的过程中，通过架设于车辆的摄像头对道路进行拍摄，并对拍摄到的道路图像进行识别，以此从道路图像提取出障碍物图像，以及时发现道路中所可能出现的障碍物。此外，还可以在车辆的不同位置架设摄像头，以拍摄前方的道路图像及后方的道路图像，实现对本车是否存在障碍物抛洒行为的监控。
应理解，上述实施例中各步骤的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
实施例2
对应于前文所提出的物体识别方法，本申请实施例提供了一种物体识别装置，上述物体识别装置集成于电子设备。请参阅图4，本申请实施例中的物体识别装置400包括：
获取单元401，用于获取预设摄像头所采集的道路图像，其中，上述预设摄像头安装于被监控车辆；
识别单元402，用于基于预设的算法对上述道路图像进行障碍物识别，以获得上述道路图像中的障碍物图像。
可选地，上述识别单元402，包括：
分割子单元，用于通过已训练的语义分割模型对上述道路图像进行分割，获得图像掩码结果；
映射子单元，用于将上述图像掩码结果映射至上述道路图像，获得上述道路图像中的障碍物图像。
可选地，上述物体识别装置400还包括：
提取单元，用于将上述道路图像输入至已训练的特征提取网络，获得上述道路图像的图像特征；
相应地，上述分割子单元，具体用于通过已训练的语义分割模型对上述图像特征进行分割，获得图像掩码结果。
可选地，上述分割子单元，包括：
类别确定子单元，用于通过已训练的语义分割模型对上述道路图像进行分割，获得上述道路图像中每个像素点所属的类别：
连通域分析子单元，用于基于各个像素点所属的类别进行连通域分析，得到至少一个连通域；
图像掩码结果获取子单元，用于根据上述至少一个连通域获得图像掩码结果。
可选地，上述识别单元402，包括：
目标检测子单元，用于通过已训练的目标检测模型对上述道路图像进行目标检测，以获得上述道路图像中的障碍物图像。
可选地，上述物体识别装置400还包括：
第一上传单元，用于将上述障碍物图像上传至预设的平台，以指示上述平台对上述障碍物图像是否表达真实障碍物进行验证。
可选地，上述物体识别装置400还包括：
计算单元，用于计算上述障碍物图像的面积；
定位单元，用于获取上述被监控车辆的定位位置；
第二上传单元，用于将上述面积及上述定位位置上传至预设的平台，以进行存证。
可选地，上述预设摄像头包括第一摄像头及第二摄像头，上述第一摄像头朝向被监控车辆的前方，上述第二摄像头朝向被监控车辆的后方，上述道路图像包括上述第一摄像头所采集的第一道路图像及上述第二摄像头所采集的第二道路图像；上述物体识别装置400还包括：
提醒单元，用于若上述第一道路图像中不存在障碍物图像，且上述第二道路图像中存在障碍物图像，则向上述被监控车辆输出提醒消息，以提醒上述被监控车辆存在障碍物抛洒行为。
可选地，上述获取单元401，具体用于当上述被监控车辆处于行驶状态时，获取上述预设摄像头所采集的道路图像。
由上可见，通过本申请实施例，在车辆行驶的过程中，通过架设于车辆的摄像头对道路进行拍摄，物体识别装置随即可对拍摄到的道路图像进行识别，以此从道路图像提取出障碍物图像，以及时发现道路中所可能出现的障碍物。此外，还可以在车辆的不同位置架设摄像头，以拍摄前方的道路图像及后方的道路图像，实现对本车是否存在障碍物抛洒行为的监控。
实施例3
本申请实施例还提供了一种电子设备，请参阅图5，本申请实施例中的电子设备5包括：存储器501，一个或多个处理器502（图5中仅示出一个）及存储在存储器501上并可在处理器上运行的计算机程序。其中：存储器501用于存储软件程序以及单元，处理器502通过运行存储在存储器501的软件程序以及单元，从而执行各种功能应用以及数据处理，以获取上述预设事件对应的资源。具体地，处理器502通过运行存储在存储器501的上述计算机程序时实现以下步骤：
获取预设摄像头所采集的道路图像，其中，上述预设摄像头安装于被监控车辆；
基于预设的算法对上述道路图像进行障碍物识别，以获得上述道路图像中的障碍物图像。
假设上述为第一种可能的实施方式，则在第一种可能的实施方式作为基础而提供的第二种可能的实施方式中，上述基于预设的算法对上述道路图像进行障碍物识别，以获得上述道路图像中的障碍物图像，包括：
通过已训练的语义分割模型对上述道路图像进行分割，获得图像掩码结果；
将上述图像掩码结果映射至上述道路图像，获得上述道路图像中的障碍物图像。
在第二种可能的实施方式作为基础而提供的第三种可能的实施方式中，在上述通过已训练的语义分割模型对上述道路图像进行分割，获得图像掩码结果之前，处理器502通过运行存储在存储器501的上述计算机程序时还实现以下步骤：
将上述道路图像输入至已训练的特征提取网络，获得上述道路图像的图像特征；
相应地，上述通过已训练的语义分割模型对上述道路图像进行分割，获得图像掩码结果，包括：
通过已训练的语义分割模型对上述图像特征进行分割，获得图像掩码结果。
在上述第二种可能的实施方式作为基础而提供的第四种可能的实施方式中，上述通过已训练的语义分割模型对上述道路图像进行分割，获得图像掩码结果，包括：
通过已训练的语义分割模型对上述道路图像进行分割，获得上述道路图像中每个像素点所属的类别：
基于各个像素点所属的类别进行连通域分析，得到至少一个连通域；
根据上述至少一个连通域获得图像掩码结果。
在上述第一种可能的实施方式作为基础而提供的第五种可能的实施方式中，上述基于预设的算法对上述道路图像进行障碍物识别，以获得上述道路图像中的障碍物图像，包括：
通过已训练的目标检测模型对上述道路图像进行目标检测，以获得上述道路图像中的障碍物图像。
在上述第一种可能的实施方式作为基础，或者上述第二种可能的实施方式作为基础，或者上述第三种可能的实施方式作为基础，或者上述第四种可能的实施方式作为基础，或者上述第五种可能的实施方式作为基础而提供的第六种可能的实施方式中，在上述基于预设的算法对上述道路图像进行障碍物识别，以获得上述道路图像中的障碍物图像之后，处理器502通过运行存储在存储器501的上述计算机程序时还实现以下步骤：
将上述障碍物图像上传至预设的平台，以指示上述平台对上述障碍物图像是否表达真实障碍物进行验证。
在上述第一种可能的实施方式作为基础，或者上述第二种可能的实施方式作为基础，或者上述第三种可能的实施方式作为基础，或者上述第四种可能的实施方式作为基础，或者上述第五种可能的实施方式作为基础而提供的第七种可能的实施方式中，在上述基于预设的算法对上述道路图像进行障碍物识别，以获得上述道路图像中的障碍物图像之后，处理器502通过运行存储在存储器501的上述计算机程序时还实现以下步骤：
计算上述障碍物图像的面积；
获取上述被监控车辆的定位位置；
将上述面积及上述定位位置上传至预设的平台，以进行存证。
在上述第一种可能的实施方式作为基础，或者上述第二种可能的实施方式作为基础，或者上述第三种可能的实施方式作为基础，或者上述第四种可能的实施方式作为基础，或者上述第五种可能的实施方式作为基础而提供的第八种可能的实施方式中，上述预设摄像头包括第一摄像头及第二摄像头，上述第一摄像头朝向被监控车辆的前方，上述第二摄像头朝向被监控车辆的后方，上述道路图像包括上述第一摄像头所采集的第一道路图像及上述第二摄像头所采集的第二道路图像；处理器502通过运行存储在存储器501的上述计算机程序时还实现以下步骤：
若上述第一道路图像中不存在障碍物图像，且上述第二道路图像中存在障碍物图像，则向上述被监控车辆输出提醒消息，以提醒上述被监控车辆存在障碍物抛洒行为。
在上述第一种可能的实施方式作为基础，或者上述第二种可能的实施方式作为基础，或者上述第三种可能的实施方式作为基础，或者上述第四种可能的实施方式作为基础，或者上述第五种可能的实施方式作为基础而提供的第九种可能的实施方式中，上述获取预设摄像头所采集的道路图像，包括：
当上述被监控车辆处于行驶状态时，获取上述预设摄像头所采集的道路图像。
应当理解，在本申请实施例中，所称处理器502可以是中央处理单元(Central Processing Unit，CPU)，该处理器还可以是其他通用处理器、数字信号处理器 (Digital Signal Processor，DSP)、专用集成电路 (Application Specific Integrated Circuit，ASIC)、现成可编程门阵列 (Field-Programmable Gate Array，FPGA) 或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
存储器501可以包括只读存储器和随机存取存储器，并向处理器502 提供指令和数据。存储器501的一部分或全部还可以包括非易失性随机存取存储器。例如，存储器501还可以存储设备类别的信息。
由上可见，通过本申请实施例，在车辆行驶的过程中，通过架设于车辆的摄像头对道路进行拍摄，电子设备随即可对拍摄到的道路图像进行识别，以此从道路图像提取出障碍物图像，以及时发现道路中所可能出现的障碍物。此外，还可以在车辆的不同位置架设摄像头，以拍摄前方的道路图像及后方的道路图像，实现对本车是否存在障碍物抛洒行为的监控。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元、模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即将上述装置的内部结构划分成不同的功能单元或模块，以完成以上描述的全部或者部分功能。实施例中的各功能单元、模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中，上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。另外，各功能单元、模块的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。上述系统中单元、模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者外部设备软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
在本申请所提供的实施例中，应该理解到，所揭露的装置和方法，可以通过其它的方式实现。例如，以上所描述的系统实施例仅仅是示意性的，例如，上述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通讯连接可以是通过一些接口，装置或单元的间接耦合或通讯连接，可以是电性，机械或其它的形式。
上述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
上述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读存储介质中。基于这样的理解，本申请实现上述实施例方法中的全部或部分流程，也可以通过计算机程序来指令相关联的硬件来完成，上述的计算机程序可存储于一计算机可读存储介质中，该计算机程序在被处理器执行时，可实现上述各个方法实施例的步骤。其中，上述计算机程序包括计算机程序代码，上述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。上述计算机可读存储介质可以包括：能够携带上述计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机可读存储器、只读存储器（ROM，Read-Only Memory）、随机存取存储器（RAM，Random Access Memory）、电载波信号、电信信号以及软件分发介质等。需要说明的是，上述计算机可读存储介质包含的内容可以根据司法管辖区内立法和专利实践的要求进行适当的增减，例如在某些司法管辖区，根据立法和专利实践，计算机可读存储介质不包括是电载波信号和电信信号。
以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围，均应包含在本申请的保护范围之内。

Claims (20)

  1. 一种物体识别方法，其特征在于，包括：
    获取预设摄像头所采集的道路图像，其中，所述预设摄像头安装于被监控车辆；
    基于预设的算法对所述道路图像进行障碍物识别，以获得所述道路图像中的障碍物图像。
  2. 如权利要求1所述的物体识别方法，其特征在于，所述基于预设的算法对所述道路图像进行障碍物识别，以获得所述道路图像中的障碍物图像，包括：
    通过已训练的语义分割模型对所述道路图像进行分割，获得图像掩码结果；
    将所述图像掩码结果映射至所述道路图像，以获得所述道路图像中的障碍物图像。
  3. 如权利要求2所述的物体识别方法，其特征在于，在所述通过已训练的语义分割模型对所述道路图像进行分割，获得图像掩码结果之前，所述物体识别方法还包括：
    将所述道路图像输入至已训练的特征提取网络，获得所述道路图像的图像特征；
    相应地，所述通过已训练的语义分割模型对所述道路图像进行分割，获得图像掩码结果，包括：
    通过已训练的语义分割模型对所述图像特征进行分割，获得图像掩码结果。
  4. 如权利要求2所述的物体识别方法，其特征在于，所述通过已训练的语义分割模型对所述道路图像进行分割，获得图像掩码结果，包括：
    通过已训练的语义分割模型对所述道路图像进行分割，获得所述道路图像中每个像素点所属的类别：
    基于各个像素点所属的类别进行连通域分析，得到至少一个连通域；
    根据所述至少一个连通域获得图像掩码结果。
  5. 如权利要求1所述的物体识别方法，其特征在于，所述基于预设的算法对所述道路图像进行障碍物识别，以获得所述道路图像中的障碍物图像，包括：
    通过已训练的目标检测模型对所述道路图像进行目标检测，以获得所述道路图像中的障碍物图像。
  6. 如权利要求1至5任一项所述的物体识别方法，其特征在于，在所述基于预设的算法对所述道路图像进行障碍物识别，以获得所述道路图像中的障碍物图像之后，所述物体识别方法还包括：
    将所述障碍物图像上传至预设的平台，以指示所述平台对所述障碍物图像是否表达真实障碍物进行验证。
  7. 如权利要求1至5任一项所述的物体识别方法，其特征在于，在所述基于预设的算法对所述道路图像进行障碍物识别，以获得所述道路图像中的障碍物图像之后，所述物体识别方法还包括：
    计算所述障碍物图像的面积；
    获取所述被监控车辆的定位位置；
    将所述面积及所述定位位置上传至预设的平台，以进行存证。
  8. 如权利要求1至5任一项所述的物体识别方法，其特征在于，所述预设摄像头包括第一摄像头及第二摄像头，所述第一摄像头朝向被监控车辆的前方，所述第二摄像头朝向被监控车辆的后方，所述道路图像包括所述第一摄像头所采集的第一道路图像及所述第二摄像头所采集的第二道路图像；所述物体识别方法还包括：
    若所述第一道路图像中不存在障碍物图像，且所述第二道路图像中存在障碍物图像，则向所述被监控车辆输出提醒消息，以提醒所述被监控车辆存在障碍物抛洒行为。
  9. 如权利要求1至5任一项所述的物体识别方法，其特征在于，所述获取预设摄像头所采集的道路图像，包括：
    当所述被监控车辆处于行驶状态时，获取所述预设摄像头所采集的道路图像。
  10. 一种物体识别装置，其特征在于，包括：
    获取单元，用于获取预设摄像头所采集的道路图像，其中，所述预设摄像头安装于被监控车辆；
    识别单元，用于基于预设的算法对所述道路图像进行障碍物识别，以获得所述道路图像中的障碍物图像。
  11. 一种电子设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现如下步骤：
    获取预设摄像头所采集的道路图像，其中，所述预设摄像头安装于被监控车辆；
    基于预设的算法对所述道路图像进行障碍物识别，以获得所述道路图像中的障碍物图像。
  12. 如权利要求11所述的电子设备，其特征在于，所述处理器执行所述计算机程序时，所述基于预设的算法对所述道路图像进行障碍物识别，以获得所述道路图像中的障碍物图像，包括：
    通过已训练的语义分割模型对所述道路图像进行分割，获得图像掩码结果；
    将所述图像掩码结果映射至所述道路图像，以获得所述道路图像中的障碍物图像。
  13. 如权利要求12所述的电子设备，其特征在于，在所述通过已训练的语义分割模型对所述道路图像进行分割，获得图像掩码结果之前，所述处理器执行所述计算机程序时还实现以下步骤：
    将所述道路图像输入至已训练的特征提取网络，获得所述道路图像的图像特征；
    相应地，所述通过已训练的语义分割模型对所述道路图像进行分割，获得图像掩码结果，包括：
    通过已训练的语义分割模型对所述图像特征进行分割，获得图像掩码结果。
  14. 如权利要求12所述的电子设备，其特征在于，所述处理器执行所述计算机程序时，所述通过已训练的语义分割模型对所述道路图像进行分割，获得图像掩码结果，包括：
    通过已训练的语义分割模型对所述道路图像进行分割，获得所述道路图像中每个像素点所属的类别：
    基于各个像素点所属的类别进行连通域分析，得到至少一个连通域；
    根据所述至少一个连通域获得图像掩码结果。
  15. 如权利要求11所述的电子设备，其特征在于，所述处理器执行所述计算机程序时，所述基于预设的算法对所述道路图像进行障碍物识别，以获得所述道路图像中的障碍物图像，包括：
    通过已训练的目标检测模型对所述道路图像进行目标检测，以获得所述道路图像中的障碍物图像。
  16. 如权利要求11至15任一项所述的电子设备，其特征在于，在所述基于预设的算法对所述道路图像进行障碍物识别，以获得所述道路图像中的障碍物图像之后，所述处理器执行所述计算机程序时还实现以下步骤：
    将所述障碍物图像上传至预设的平台，以指示所述平台对所述障碍物图像是否表达真实障碍物进行验证。
  17. 如权利要求11至15任一项所述的电子设备，其特征在于，在所述基于预设的算法对所述道路图像进行障碍物识别，以获得所述道路图像中的障碍物图像之后，所述处理器执行所述计算机程序时还实现以下步骤：
    计算所述障碍物图像的面积；
    获取所述被监控车辆的定位位置；
    将所述面积及所述定位位置上传至预设的平台，以进行存证。
  18. 如权利要求11至15任一项所述的电子设备，其特征在于，所述预设摄像头包括第一摄像头及第二摄像头，所述第一摄像头朝向被监控车辆的前方，所述第二摄像头朝向被监控车辆的后方，所述道路图像包括所述第一摄像头所采集的第一道路图像及所述第二摄像头所采集的第二道路图像；所述处理器执行所述计算机程序时还实现以下步骤：
    若所述第一道路图像中不存在障碍物图像，且所述第二道路图像中存在障碍物图像，则向所述被监控车辆输出提醒消息，以提醒所述被监控车辆存在障碍物抛洒行为。
  19. 如权利要求11至15任一项所述的电子设备，其特征在于，所述处理器执行所述计算机程序时，所述获取预设摄像头所采集的道路图像，包括：
    当所述被监控车辆处于行驶状态时，获取所述预设摄像头所采集的道路图像。
  20. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至9任一项所述的方法。
PCT/CN2020/120891 2020-10-14 2020-10-14 一种物体识别方法、物体识别装置及电子设备 WO2022077264A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202080002303.0A CN112424793B (zh) 2020-10-14 2020-10-14 一种物体识别方法、物体识别装置及电子设备
PCT/CN2020/120891 WO2022077264A1 (zh) 2020-10-14 2020-10-14 一种物体识别方法、物体识别装置及电子设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/120891 WO2022077264A1 (zh) 2020-10-14 2020-10-14 一种物体识别方法、物体识别装置及电子设备

Publications (1)

Publication Number Publication Date
WO2022077264A1 true WO2022077264A1 (zh) 2022-04-21

Family

ID=74782983

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/120891 WO2022077264A1 (zh) 2020-10-14 2020-10-14 一种物体识别方法、物体识别装置及电子设备

Country Status (2)

Country Link
CN (1) CN112424793B (zh)
WO (1) WO2022077264A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115100633A (zh) * 2022-08-24 2022-09-23 广东中科凯泽信息科技有限公司 基于机器学习的障碍物识别方法
WO2023245615A1 (zh) * 2022-06-24 2023-12-28 中国科学院深圳先进技术研究院 导盲方法、装置及可读存储介质
WO2024007570A1 (zh) * 2022-07-04 2024-01-11 惠州市德赛西威汽车电子股份有限公司 障碍物识别方法、装置、电子设备及存储介质
CN117593890A (zh) * 2024-01-18 2024-02-23 交通运输部公路科学研究所 一种道路遗撒物体的检测方法、装置、电子设备及存储介质
CN118097625A (zh) * 2024-04-24 2024-05-28 广汽埃安新能源汽车股份有限公司 一种障碍物识别方法及装置

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113160231A (zh) * 2021-03-29 2021-07-23 深圳市优必选科技股份有限公司 一种样本生成方法、样本生成装置及电子设备
CN113255439B (zh) * 2021-04-13 2024-01-12 深圳市锐明技术股份有限公司 一种障碍物识别方法、装置、系统、终端及云端
CN113128386B (zh) * 2021-04-13 2024-02-09 深圳市锐明技术股份有限公司 一种障碍物识别方法、障碍物识别装置及电子设备
CN113160217B (zh) * 2021-05-12 2024-08-20 北京京东乾石科技有限公司 一种线路异物的检测方法、装置、设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006050451A (ja) * 2004-08-06 2006-02-16 Sumitomo Electric Ind Ltd 障害物警告システム及び画像処理装置
CN107169468A (zh) * 2017-05-31 2017-09-15 北京京东尚科信息技术有限公司 用于控制车辆的方法和装置
CN110246142A (zh) * 2019-06-14 2019-09-17 深圳前海达闼云端智能科技有限公司 一种检测障碍物的方法、终端和可读存储介质
CN111353337A (zh) * 2018-12-21 2020-06-30 厦门歌乐电子企业有限公司 一种障碍物识别装置及方法
CN111666921A (zh) * 2020-06-30 2020-09-15 腾讯科技（深圳）有限公司 车辆控制方法、装置、计算机设备和计算机可读存储介质
CN111753612A (zh) * 2019-09-11 2020-10-09 上海高德威智能交通系统有限公司 一种抛洒物检测方法、装置及存储介质

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109740484A (zh) * 2018-12-27 2019-05-10 斑马网络技术有限公司 道路障碍物识别的方法、装置及系统
CN110097109B (zh) * 2019-04-25 2022-12-16 湖北工业大学 一种基于深度学习的道路环境障碍物检测系统及方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006050451A (ja) * 2004-08-06 2006-02-16 Sumitomo Electric Ind Ltd 障害物警告システム及び画像処理装置
CN107169468A (zh) * 2017-05-31 2017-09-15 北京京东尚科信息技术有限公司 用于控制车辆的方法和装置
CN111353337A (zh) * 2018-12-21 2020-06-30 厦门歌乐电子企业有限公司 一种障碍物识别装置及方法
CN110246142A (zh) * 2019-06-14 2019-09-17 深圳前海达闼云端智能科技有限公司 一种检测障碍物的方法、终端和可读存储介质
CN111753612A (zh) * 2019-09-11 2020-10-09 上海高德威智能交通系统有限公司 一种抛洒物检测方法、装置及存储介质
CN111666921A (zh) * 2020-06-30 2020-09-15 腾讯科技（深圳）有限公司 车辆控制方法、装置、计算机设备和计算机可读存储介质

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023245615A1 (zh) * 2022-06-24 2023-12-28 中国科学院深圳先进技术研究院 导盲方法、装置及可读存储介质
WO2024007570A1 (zh) * 2022-07-04 2024-01-11 惠州市德赛西威汽车电子股份有限公司 障碍物识别方法、装置、电子设备及存储介质
CN115100633A (zh) * 2022-08-24 2022-09-23 广东中科凯泽信息科技有限公司 基于机器学习的障碍物识别方法
CN117593890A (zh) * 2024-01-18 2024-02-23 交通运输部公路科学研究所 一种道路遗撒物体的检测方法、装置、电子设备及存储介质
CN117593890B (zh) * 2024-01-18 2024-03-29 交通运输部公路科学研究所 一种道路遗撒物体的检测方法、装置、电子设备及存储介质
CN118097625A (zh) * 2024-04-24 2024-05-28 广汽埃安新能源汽车股份有限公司 一种障碍物识别方法及装置

Also Published As

Publication number Publication date
CN112424793A (zh) 2021-02-26
CN112424793B (zh) 2024-09-20

Similar Documents

Publication Publication Date Title
WO2022077264A1 (zh) 一种物体识别方法、物体识别装置及电子设备
US11704631B2 (en) Analyzing images and videos of damaged vehicles to determine damaged vehicle parts and vehicle asymmetries
CN107738612B (zh) 基于全景视觉辅助系统的自动泊车停车位检测与识别系统
Dhiman et al. Pothole detection using computer vision and learning
WO2020192122A1 (zh) 一种闯红灯车辆的非现场执法图片智能审核方法和系统
CN108877269B (zh) 一种交叉路口车辆状态检测及v2x广播方法
CN110738150B (zh) 相机联动抓拍方法、装置以及计算机存储介质
CN104517111A (zh) 车道线检测方法、系统、车道偏离预警方法及系统
WO2019165614A1 (zh) 一种车辆周边环境分析方法、装置及车载设备
CN113128386B (zh) 一种障碍物识别方法、障碍物识别装置及电子设备
CN113160272B (zh) 目标跟踪方法、装置、电子设备及存储介质
CN113255439B (zh) 一种障碍物识别方法、装置、系统、终端及云端
CN111967396A (zh) 障碍物检测的处理方法、装置、设备及存储介质
CN106713761A (zh) 一种图像处理方法和装置
CN109840454A (zh) 目标物定位方法、装置、存储介质以及设备
CN112597807A (zh) 一种违章检测系统、方法、装置、图像采集设备及介质
CN113793413A (zh) 三维重建方法、装置、电子设备及存储介质
CN115527178A (zh) 路面病害检测方法和装置、电子设备和存储介质
Yebes et al. Learning to automatically catch potholes in worldwide road scene images
CN115240148A (zh) 车辆行为的检测方法、装置、存储介质及电子装置
CN115564937A (zh) 一种车辆侧面图像实时拼接方法、计算设备及存储介质
CN114255428A (zh) 一种基于边缘智能和路边高位监控视频的车辆停车识别方法
CN115188077A (zh) 电池违规充电的检测方法、装置、设备及存储介质
CN110490117B (zh) 一种基于图像深度信息的停车事件确定方法及系统
CN117058912A (zh) 巡检车检测异常停车方法、装置、存储介质及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20957045

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20957045

Country of ref document: EP

Kind code of ref document: A1